Robotic arm for picture and video capture

Abstract

The present invention relates to a modular robotic arm for mounting a camera for picture and video capture. The modular robotic arm has six degrees of freedom and comes with a wrist portion for holding a camera capable of moving in six degrees of freedom. The wrist portion has different parts for different configurations allowing use with a variety of camera types and weights. A universal camera holder makes the modular robotic arm compatible with different camera types. The modular robotic arm is programmable for movement in predetermined paths and can focus a camera on a single point while moving.

Description

ROBOTIC ARM FOR PICTURE AND VIDEO CAPTUREBACKGROUND

[0001] Picture and video capturing quality depends highly on the positioning of the camera. Various mechanisms for camera positioning are available.

[0002] The most popular camera extension is known as a selfie stick, which allows a camera to attach to a clamp at one end of the selfie stick. A user can hold one end of the selfie stick and press a button to take a picture using the camera attached to the other end of the selfie stick. A wire is available at the camera attachment end, which is plugged into the camera for the button of at the other end of the camera to command the camera. Alternatively, the button is operatively connected to the camera through wireless communication, such as by Bluetooth antenna.

[0003] Carriers and extensions for camera movement and positioning are available for use in various settings. All robotic arms are defined by “degree of freedom”, which is how many points around which a robotic arm can move. A higher degree of freedom allows a wider range of movement but is also harder to control for precision. Carriers and extensions for camera movement and positioning are also subject to this challenge in design and operation.

[0004] Moving a camera on a pathway while pointing at a certain location is a desirable characteristic in video taking. This is often referred to as “360-degree view” and is used commonly in recording product videos for product introduction and listing in e-commerce. It is also a popular technique in picture and video capture of other types.

[0005] Video and picture capturing invariably involves moving to different locations. Video recording equipment, in particular carriers and extensions for cameras, are often heavy and cumbersome. The ability of any camera carrier or extension to move on a certain terrain is also a challenge, especially for taking videos and pictures in natural settings. Providing a means for achieving these objectives while maintaining a low cost remains a challenge.

[0006] The present invention addresses these and other needs in the art.SUMMARY

[0007] These and other embodiments, features, and advantages will become apparent to those skilled in the art when taken with reference to the following more detailed description of various exemplary embodiments of the present disclosure in conjunction with the accompanying drawings.

[0008] According to frequently included embodiments, there is provided a modular apparatus to move a camera during video recording or picture taking, the modular apparatus comprises: a base with a plurality of casters configured to move on different surfaces; a pedestal attached to the base at one end of the pedestal; upper ami portion attached to the pedestal at a shoulder joint, the shoulder joint having a shoulder motor configured to rotate the upper arm around the shoulder joint; a pedestal motor contained in the pedestal and connected to the shoulder joint, the pedestal motor configured to rotate the shoulder joint around the vertical central axis of the pedestal; a forearm portion attached to the upper arm portion at an elbow joint, the elbow joint having an elbow motor configured to rotate the forearm around the elbow joint; a wrist assembly attached to the forearm at a wrist joint, the wrist assembly having at least one wrist motor configured to rotate the wrist assembly around the wrist joint; an end effector attached to the wrist assembly and configured to hold a camera; a central processing unit configured to control the pedestal motor, the shoulder motor, the elbow motor, and the at least one wrist motor, a microcontroller unit having Bluetooth connection capability and configured to trigger a camera; at least one power supply to supply power to the central processing unit, the pedestal motor, the shoulder motor, the elbow motor, and the at least one wrist motor; a computer programming product operable on the central processing unit, comprising: a robotic operating system configured to command and control the robotic arm; a motion planning framework configured to plan movement of the robotic arm between way points; a hardware abstraction layer configured to interface with the motion planning framework and software within motors;a web application operable on the central processing unit and configured to be accessible from a separate computing article; and a Representation State Transfer Application Programming Interface configured to interface between the robotic operating system and the web application, wherein each motor has software configured to interact with the hardware abstraction layer within the computer programming product, wherein the web application is configured to interact with robotic operating system via the Representation State Transfer Application Programming Interface, and wherein the robotic operating system is configured to command the microcontroller unit to trigger the camera

[0009] Often according to embodiments described herein, there is provided a modular apparatus as above, wherein the wrist assembly comprises: a roll motor having a roll motor output shaft attached to a roll motor horn, the roll motor horn attached a roll motor output shaft through a roll motor bearing assembly, the roll motor output shaft rotates around the central axis of the forearm; a pitch motor mounting bracket attached to roll motor output shaft; a pitch motor having a pitch motor output shaft attached to a pitch motor horn, the pitch motor horn attached to a pitch motor output shaft through a pitch motor bearing assembly, the pitch motor output shaft rotates around an axis perpendicular to the central axis of the forearm; a yaw motor mounting bracket attached to the pitch motor output shaft; a yaw motor having a yaw motor output shaft attached to a yaw motor horn, the yaw motor horn attached to a yaw motor output shaft through a yaw motor bearing assembly, the yaw motor output shaft rotates around an axis perpendicular to the central axis of the forearm and the pitch motor output shaft axis; and a universal attachment attached to the yaw motor mounting bracket and is operatively connected to the yaw motor output shaft, the universal attachment is capable of rotating by the yaw motor output shaft, wherein the universal attachment is attachable to the end effector.

[0010] Frequently according to embodiments described herein, there is provided a modular apparatus as above, wherein the roll motor, the pitch motor, and the yaw motor arc servo motors.

[0011] Often according to embodiments described herein, there is provided a modular apparatus as above, wherein the pedestal motor, the shoulder motor, and the elbow motor are stepper motors.

[0012] Frequently according to embodiments described herein, there is provided a modular apparatus as above, wherein the end effector comprises: a housing with a space configured to hold a camera; at least one LED light on the housing, the at least on LED light is embedded into the housing; electronic components to control the at least one LED light; wherein the housing is configured to attached to the universal attachment, wherein the electronic components are operatively connected to the central processing unit.

[0013] Often according to embodiments, there is provided a modular apparatus as above, wherein the end effector further comprises a plastic diffuser to protect the at least one LED light.

[0014] Frequently according to embodiments, there is provided a modular apparatus as above, wherein the end effector further comprises a clamp configured to hold cameras of different sizes.

[0015] Often according to embodiments described herein, there is provided a modular apparatus as above, wherein the pedestal comprises concentric plates at the top attached to the shoulder joint via a dowel pin.

[0016] Frequently according to embodiments described herein, there is provided a modular apparatus as above, wherein the central processing unit is configured to communicate with the microcontroller unit via serial communication.

[0017] Often according to embodiments described herein, there is provided a modular' apparatus as above, further comprising a pedestal motor brake configured to provide braking function to the pedestal motor.

[0018] Frequently according to embodiments described herein, there is provided a modular apparatus as above, further comprising a communication hub configured to receive commands from the central processing unit and send commands to the motors and the end effector.

[0019] Often according to embodiments described herein, there is provided a modular apparatus as above, further comprising an emergency stop switch capable of being activated to stop power supplied to the apparatus.

[0020] Frequently according to embodiments described herein, there is provided a modular apparatus as above, wherein the separate computing article comprises desktop, a laptop, a smartphone, or a tablet.

[0021] Often according to embodiments described herein, there is provided a modular apparatus as above, wherein the web application is accessible from the separate computing article via Ethernet or a wireless network.

[0022] Frequently according to embodiments described herein, there is provided a modular apparatus as above, the motion planning framework is configured to avoid self-collision.

[0023] Often according to embodiments described herein, there is provided a modular apparatus as above, wherein the computer programming product is configured to carry out preplanned movement path for the robotic arm.

[0024] Frequently according to embodiments described herein, there is provided a modular apparatus as above, wherein each of the base, the pedestal, the shoulder joint, the upper arm portion, the forearm portion, the wrist assembly, and the end effector is attached to other parts of the modular apparatus using removable attachments.

[0025] Often according to embodiments described herein, there is provided a modular apparatus as above, wherein the removable attachments are removable screws.

[0026] Frequently according to embodiments described herein, there is provided a modular apparatus as above, wherein the motion planning framework is further configured to avoid selfcollision of the modular apparatus.

[0027] Often according to embodiments described herein, there is provided a computer programming product operable on a computing article, the computer programming product comprises: a robotic operating system configured to command and control a robotic arm; a motion planning framework configured to plan movement of the robotic arm between way points; a hardware abstraction layer configured to interface with the motion planning framework and motor software;a web application operable on the central processing unit and configured to be accessible from a separate computing article; and a Representation State Transfer Application Programming Interface configured to interface between the robotic operating system and the web application; wherein the web application is configured to interact with robotic operating system via the Representation State Transfer Application Programming Interface, and wherein the robotic operating system is configured to command the microcontroller unit to trigger the camera.

[0028] Frequently according to embodiments described herein, there is provided a computer programming product as above, wherein the web application is accessible from the separate computing article via Ethernet or a wireless network.

[0029] Often according to embodiments described herein, there is provided a computer programming product as above, wherein the motion planning framework is configured to avoid self-collision.

[0030] Frequently according to embodiments described herein, there is provided a computer programming product as above, wherein the computer programming product is configured to carry out pre-planned movement path for the robotic arm.

[0031] Often according to embodiments described herein, there is provided a method to take a video recording, comprising: positioning a camera in the end effector of the modular apparatus as above; activating a computer programming product configured to communicate with the central processing unit in the robotic arm and control the robotic arm; choosing a pathway from a collection of pathways available in the computer programming product; commanding the robotic arm to move according to the chosen pathway; and recording a video using the camera in the end effector.

[0032] Frequently according to embodiments described herein, there is provided a method to take a video recording as above, wherein activating the computer programming product is carried out on the separate computing article.

[0033] In addition, methods of use of the modular apparatus as described herein with a camera for capturing one or more images or video according to the description provided herein are also specifically contemplated. The capturing of images is static or dynamic. For example, according to certain embodiments described herein each of the one or more images is captured from a single position of the base, pedestal, upper ami, forearm and / or wrist assembly. According to more frequent embodiments, at least one of the one or more images is captured at a position of the base, pedestal, upper arm, forearm and / or wrist assembly that is different than the position of the base, pedestal, upper arm, forearm and / or wrist assembly when at least one other of the one or more images is captured. According to other frequent embodiments, during the time that the video is captured by the camera, the position of the base, pedestal, upper arm, forearm and / or wrist assembly changes from one position to one or more different positions. In each of these image or video capture embodiments, the camera maintains its area of focus or target area on a specific static or dynamic / moving target location.BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The skilled person in the art will understand that the drawings, described below, are for illustration purposes only.

[0035] FIG. 1 illustrates a robotic arm according to embodiments herein, with the robotic arm extending straight.

[0036] FIG. 2 illustrates a robotic arm according to embodiments herein, with the robotic arm bent at a joint and part of the inside of the pedestal showing.

[0037] FIG. 3 is a perspective view of the base of the robotic arm.

[0038] FIG. 4 is a top view of the base of the robotic arm.

[0039] FIG. 5 is a perspective of the base of the robotic arm and the pedestal, with the pedestal being in exploded view from the base.

[0040] FIG. 6 is a top view of the base and the pedestal with wires shown.

[0041] FIG. 7 is a schematic view of components inside the pedestal of the robotic arm.

[0042] FIG. 8 is a perspective view of the shoulder area of the robotic arm.

[0043] FIG. 9 is an exploded view of the elbow joint in the robotic arm.

[0044] FIG. 10 is an exploded view of the wrist assembly as attached to the forearm.

[0045] FIG. 11 is a perspective view of the wrist assembly as attached to the forearm.

[0046] FIG. 12 is an exploded view of the wrist assembly showing the roll and the pitch motor.

[0047] FIG. 13 is an exploded view of the wrist assembly output, the end effector, and the phone in the end effector.

[0048] FIG. 14 is the power supply and command chart for the robotic arm.

[0049] FIG. 15 illustrates the software architecture of the computer programming product used for control and operation of the robotic arm.DETAILED DESCRIPTION

[0050] For clarity of disclosure, and not by way of limitation, the detailed description of the invention is divided into the subsections that follow.

[0051] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as is commonly understood by one of ordinary skill in the art to which this invention belongs. All patents, applications, published applications and other publications referred to herein are incorporated by reference in their entirety. If a definition set forth in this section is contrary to or otherwise inconsistent with a definition set forth in the patents, applications, published applications and other publications that are herein incorporated by reference, the definition set forth in this section prevails over the definition that is incorporated herein by reference.

[0052] As used herein, “a” or “an” means “at least one” or “one or more.”

[0053] As used herein, the term “and / or” may mean “and,” it may mean “or,” it may mean “exclusive-or,” it may mean “one,” it may mean “some, but not all,” it may mean “neither,” and / or it may mean “both.”

[0054] As used herein, “end effector” refers to a frame with clamps and other parts to hold onto a camera, which may be a camera by itself or a camera available on a computing article or mobile device such as a smartphone or a tablet. The term “camera” as used herein is intended to encompass computing articles or mobile devices such as smartphones or tablets featuring a camera.The term “end effector” encompasses any number of structures or apparatus that act to attach the camera to the robotic arm in a manner that permits operation of the arm in the manner described and contemplated herein and is not limited to the specific structural implementations described and depicted herein.

[0055] As used herein, “pedestal” refers to a, for example tower-like, structure housing various electronic and mechanical equipment while providing a platform which other components of the modular robotic arm may attach.

[0056] Embodiments of this invention relate to a modular robotic aim for picture and video taking. The robotic arm is operable by electronic command received at a user interface. The robotic arm physically attaches to a camera, which as noted may be available by itself or as part of an electronic device and is capable of moving such that the camera is pointed at a fixed location during the robotic arm movement. The fixed location is often predetermined and includes one or more locations, including movement between one fixed location to one or more other fixed location. The robotic arm is often provided in different parts that can be assembled and disassembled for movement from place to place.

[0057] FIG. 1 illustrates a modular robotic arm according to embodiments herein. The robotic arm 100 comprises a base 200 with casters 204, a pedestal 300 on top of base 200 and connected to an arm-like extension 400, the extension 400 comprises an upper arm portion 403, a forearm portion 404, a wrist portion 500, and an end effector 700. Cameras are attachable to end effector 700. The robotic arm 100 and its parts can rotate around the joints, including wrist joints, an elbow joint, and shoulder joints. Robotic arm 100 as a whole is also movable by movement of the base 200 using casters 204.

[0058] FIG. 2 illustrates the robotic arm 100 with motors in the pedestal 300 shown and various joints. Inside the body of pedestal 300 is a stepper motor 306 which operates to pan side- to-side the shoulder portion 315 thereby allowing shoulder portion 315 to rotate around the longitudinal center axis of pedestal 300, optionally in increments.

[0059] At the top of pedestal 300 is shoulder portion 315 upon which extension 400 connects to pedestal 300. Shoulder joint 307 fits to pedestal 300 at a plate and is configured to be capable of rotating in a circle for a full 360°. Extension 400 connects to shoulder joint 307 at a joint operated by motor 601, enabling extension 400 to lift and rotate around shoulder joint 307.Extension 400 connects to shoulder joint 307 at the far end of upper arm 403. Shoulder motor 601 is a stepper motor, but other suitable types of motors can be used.

[0060] The opposite end of upper arm 403 connects to forearm 404 by an elbow joint 405 operable by a motor 406, which operates to lift and rotate forearm 404 around elbow joint 405. Motor at elbow 406 is a stepper motor, but other types of motors suitable for the purpose can be used. At the end of forearm 404 is wrist assembly 500 attached to end effector 700, which carries a camera.

[0061] In embodiments, the height of the robotic arm reaches 100 inches, even though the robotic arm height can be larger than 100 inches. In an exemplary embodiment, the maximum height of the robotic arm is 99 inches. The max length of the extension is 66 inches, even though it can be longer than 66 inches if desired. The forearm and the upper arm are of approximately equal length. By way of an example only, the total length of the wrist assembly and the end effector when the wrist is at a straight position on the same line with forearm is 20 - 25 inches, and in an embodiment is 23 inches. The width of the end effector is 5-9 inches, and in an embodiment is 7 inches. The robotic arm is configured to carry a camera, and the weight of the camera attached to the end effector can be up to 3 kilograms. These parameters are for exemplary purposes only. However, it is contemplated that the size of the robotic arm is sufficient to take pictures and videos of human subjects.

[0062] FIG. 3 illustrates a perspective view of base 200 of the robotic arm. Base 200 comprises two bars situated parallel on four casters 204 holding a pan-like structure between the two bars. Casters 204 allow the robotic arm as a whole to be movable on hard surfaces. Casters 204 are located on the ends of each bar to allow movement of the robotic arm in different directions. In some cases, a different structure can be used to hold the pan-like structure, such as a substantially circular plate, and casters can be placed on those structures to fit the movement of the specific structure. Casters 204 can rotate 360°. The robotic arm according to embodiments herein thus can be used on various surfaces such as asphalt, concrete, decomposed granite, AstroTurf, carpet, or hardwood, among other surface types, and can be changed in movement direction with ease. By way of an example only, the base is approximately 61 centimeters on each side.

[0063] To enable movement of the robotic arm, other means of movement can be used. Apart from casters, wheels of different sizes and rotation capabilities can be attached to the base for moving the robotic arm or a flat surface. Wheels with a jack assembly at the top can be used toattach to the base, such that the jack assembly can push up the base to create a larger distance between the base and the ground. This configuration can increase mobility of the robotic arm when used on outside surfaces, such as a ground with rocks. Other ways to change the distance between the base and the ground are contemplated, such as by an extension / withdrawal of a rod.

[0064] On both sides of the pan-like structure are fittings for screws to attach and stabilize the pedestal. FIG. 4 illustrates a top view of base 200. A center processing unit (CPU) 201 is provided and is situated in a pan-like structure at base 200. CPU 201 provides computing power and control for the robotic arm. Also, in the pan-like structure is a power supply unit 202, which supplies power to the wrist assembly and the shoulder brake. In an exemplary embodiment, power supply unit 202 is a 24 VDC (volts of direct current) power supply unit. Other configurations for the power supply unit are contemplated.

[0065] FIG. 5 illustrates an exploded view of pedestal 300 and base 200, with pedestal 300 shown with screws 203 attachable to base 200 at fittings. Two screws are available at each comer of the pedestal bottom to slot into fittings on base 200. Wires with electrical plugs are shown at the bottom of pedestal 300. By way of an example only, the pedestal is 74cm in height and 20cm each in width and length.

[0066] FIG. 6 illustrates a top perspective view of pedestal 300 as attached to base 200. At the top of pedestal 300 are concentric plates 309 stacked on top of each other and held together by screws. At the center of the concentric plates 309 is a dowel pin 308, which aligns the pedestal top with the shoulder portion.

[0067] FIG. 7 illustrates components present in pedestal 300. Components in pedestal control and operate motors in the robotic arm, thereby moving a camera attached to the robotic arm. At the bottom of pedestal 300 is a stepper motor controller 301, which controls stepper motors in pedestal (306), in shoulder (601), and in elbow joint (406). An intake fan 303 is present in pedestal 300 and operates to keep the temperature of the area near stepper motor controller 301 and a power and signal assembly 310 within a safe operating range. In embodiments, a power signal and assembly 310 is used to supply power to and command motors in the robotic arm. A power supply 304 supplies power to the motor in the shoulder, while another power supply 305 supplies power to the motors in the elbow and the shoulder. A case fan 312 is provided to keep the temperature of power supply 305 down.

[0068] The pedestal motor is configured to rotate the shoulder joint at a speed of up to 90° per second. With the length of the extension and the speed of shoulder joint rotation, the shooting speed of the camera is up to 3 feet per second. In other words, the camera can move on a path at the speed of 3 ft / s.

[0069] FIG. 8 illustrates the pedestal to extension connection at shoulder joint 307. The motor inside pedestal 300 connects to shoulder portion 315 at dowel pin (not viewable). Concentric plates 309 surround an output shaft (not viewable) and provide a base on which shoulder housing 314 rests. Forearm 403 attaches to shoulder motor 601 at shoulder housing 314. The stepper motor in pedestal 300 rotates shoulder joint 307 via a bolted connection while shoulder motor 601 rotates and lifts the extension via forearm 403.

[0070] FIG. 9 is an exploded view of the elbow motor 406 assembly to forearm 404. The elbow motor 406 as used in embodiments herein is a NEMA23 motor, but other suitable motors are contemplated. Elbow motor 406 has a shaft 408 (not shown), which connects to harmonic gearbox 409 and turns the gears therein. A harmonic gearbox 409 is used to control and smooth movement of the motor and to add torque capability for sufficient payload capability. Between elbow motor 406 and harmonic gearbox 409 is a motor adapter plate 407 to facilitate a good fitting connection. Forearm 404 connects to harmonic gearbox 409 through an arm spacer 401 and arm plate 402 for integration and assembly spacing. Screws physically connects forearm 404 with harmonic gearbox 409 by screwing into fittings on harmonic gearbox 409. Movement of shaft 408 within elbow motor 406 is transmitted to forearm 404 through harmonic gearbox 409. Harmonic gearbox 409 allows the motor to move forearm 404 in a smooth manner, thereby preventing unstable movement of forearm 404 which results in unstable video.

[0071] FIG. 10 illustrates an embodiment with a wrist assembly with the connection between wrist roll motor 501 and forearm 404 shown in exploded view. Wrist roll motor 501 fits into a gap present on forearm 404 secured by screws and connected to a wrist pitch mounting bracket 508 that carries wrist pitch motor 502. Wrist pitch mounting bracket 508 carries wrist pitch motor 502 and attaches to wrist connecting bracket 514. Wrist connecting bracket 514 attaches to wrist yaw bracket 509, which carries wrist yaw motor 503. At the bottom of wrist yaw bracket 509 is universal attachment 504, wherein end effector 700 is mounted. As shown in FIG. 10, end effector 700 is mounted onto universal attachment 504 by screws. FIG. 11 illustrates the same wrist assembly 500 as in FIG. 10 but with forearm 404 and wrist assembly 500 shown as attached.

[0072] FIG. 12 is an exploded view of wrist assembly 500 with wrist roll motor 501 and wrist pitch motor 502 shown. Output shaft 513 of wrist roll motor 501 is fastened to a custom horn 510, which acts as an adapter from the motor splines to an output shaft 513 through a bolt pattern for fixation to the next wrist bracket 508. A wrist bearing assembly 512 surrounds output shaft 513 and hom 510 to bear output shaft loading to protect wrist roll motor 501. The next wrist bracket, or wrist pitch bracket 508 as shown in FIG. 12, connects to another hom 511, which in turn connects to wrist yaw bracket (not shown in FIG. 12) in a similar way.

[0073] Wrist assembly 500 can be detached from the robotic arm for transportation. Wrist assembly 500 can also be detached at any of the motors and replaced with other parts for versatility in use. The robotic arm can be provided with different parts for exchange, such that different options for wrist rotation and camera fitting and / or positioning can be accommodated. The forearm and upper arm are of approximately the same length, even though forearm and upper arm of different lengths are contemplated. With wrist assembly as disclosed herein, the robotic arm has a reach of up to 66 inches in radius. As the arm-like extension extends horizontally, the camera attached to the end effector is 66 inches away from the center of the pedestal and can rotate 360 degrees around the pedestal.

[0074] FIG. 13 illustrates an embodiment of the end effector, or the component of the robotic arm attached to a camera, as attached to the wrist assembly. End effector 700 is a component physically and operatively attached to universal attachment 504 to hold camera during operation of the robotic arm. End effector 700 shown in FIG. 13 has a generally oblong shape dimensioned to fit common cameras, including but not limited to smartphones. Other shapes for the end effector are contemplated. End effector 700 comprises a housing to house components and camera 800. End effector 700 optionally further comprises LED light 702 for decoration and / or to produce a higher quality output video. Electronics in the nature of a controller 701 are present to control LED light 702 on the end effector 700. As shown in FIG. 13, a plastic diffuser 703 is present to overlay LED light 702 for further entertainment and improved lighting quality. Cameras suitable for use with the end effector include, but are not limited to, smartphones, GoPro, DSLR and MIRRORLESS cameras.

[0075] End effector 700 has a space to fit camera 800 therein and a clamp (not shown) to secure the camera, such that cameras of different sizes can be attached to end effector 700 and used with the robotic arm. Since end effector 700 can be removably attached to universal attachment504, multiple end effectors with different sizes and / or different configurations may be provided with the robotic arm to increase versatility. Various adapters arc also available for use with different cameras and different end effectors.

[0076] In embodiments herein, removeable attachments, in particular removable screws, are used to attach various parts, such that the robotic arm can be disassembled at all joints. Additionally, the pedestal can be detached from the shoulder portion and the base, while the base can be detached from the casters. Screws attaching the base to the pedestal, the pedestal to the shoulder joint, the shoulder joint to the upper ami, the upper arm to the forearm, the forearm to the wrist assembly, and the wrist assembly to the end effectors are removable and the robotic arm can be disassembled. The robotic arm can be disassembled within 30 minutes and transported as parts. Once at a different location, the robotic arm can be reassembled for use, also within 30 minutes. The transportability of the modular robotic arm is of great utility for movement between film sets, events, or locations.

[0077] FIG. 14 illustrates the operation block diagram of the robotic arm according to embodiments herein. A Central Processing Unit (CPU), which commands a driver for wrist motors, is provided. The CPU receives commands from user input or from pre-programmed commands and sends commands to a communication hub. The communication hub is configured to drive motors present in the wrist assembly. Commands from CPU 201 translate to movement of motors present in the wrist assembly, which in turn moves the universal attachment on the robotic arm, which is attached to the end effector carrying a camera, and consequently moves the camera in the end effector.

[0078] The CPU also commands a communication hub, which commands motors in the pedestal, the shoulder, and the elbow, as well as the shoulder brake. The shoulder is provided with a brake for safety reasons. When there is power, the brake is released. When power is lost, the brake engages, and the joint cannot move. If power is lost, the shoulder brake keeps the arm-like extension from falling down, thereby assuring safety for humans and objects around the robotic arm.

[0079] In embodiments described herein, the robotic arm operates on power supplied by a battery or by an outside source, such as an electrical grid. FIG. 14 also illustrates a power supply diagram for various pails within the robotic arm. A power source provides power in five directions, which goes to three (3) Power Supplies, a CPU and a recording device, which is a camera, and caninclude LED light. A first power supply provides power to the elbow motor and the shoulder motor. A second power supply provides power to the pedestal motor. A third power supply provides power to the motor control hub, the motor brake, the motors in the wrist assembly, and fans, if present. Fans may be provided within the pedestal, including intake and exhaust fans and case fans.

[0080] In embodiments described herein, the robotic arm is provided together with a computer programming product operable on a computing article. The computing article can be a CPU present in the pedestal or other computing articles remotely available. The computer programming product includes an application or a software with a user interface available on a computing article such as a computer, a smartphone, a tablet, or other computing articles. The CPU in the base of the robotic arm can work in conjunction with an application on a remote computing article. A user can run the software or the application and choose a pathway, over which the robotic arm will move while taking pictures and / or videos. A command is sent from computing article to the CPU, which sends commands to the communication hub. Commands are then sent to the various motors in the robotic arm to bring a camera present in the end effector to a start position corresponding to the starting point of the selected pathway. In operation, once at the start location, the robotic arm will automatically enter a brief pause to prepare the subjects for the recording.

[0081] Once the subjects are ready for recording, the CPU sends a command to the computer programming product to start recording. Simultaneously, the CPU sends commands to the motors and through the respective communication hubs to move robotic arm through the selected pathway. Software with commands operable on the CPU articulates the motors throughout the pathway to maintain focus on the target while stabilizing the camera to create the desired output video. At the end of the pathway, recording will end, and the user can navigate on the application or software interface through options for recording and start another recording.

[0082] The robotic arm is configured to point the camera in a dynamic position in the end effector, therefore the orientation of the camera can be adjusted accordingly during the video taking process. The camera also can travel on a pre-programmed fixed path. With the camera pointing at a specific location and remaining in locked orientation, cinematic effects can be obtained.

[0083] The computer programming product for the control and operation of the robotic arm according to embodiments herein have multiple layers to interact with the hardware and operators. A part of the computer programming product is a Web Application operating on a host computer, which can be accessed by devices such as an operator iPad connected to the samenetwork. This allows operators to command the robot through the operator iPad. Another part of the computer programming product runs on the CPU and is configured to plan motion paths and control the motors to carry out the planned motion paths, as well as to trigger the camera. The motors are connected to the CPU by wires, while the camera communicates with the camera trigger by Bluetooth communication.

[0084] FIG. 15 illustrates the software architecture of the computer programming product used for control and operation of the robotic arm. A Robot Operating System (ROS) (for information: https / / www.ros.org) software is integrated into the computer programming product and is the main command center for the robotic arm. A software package within the ROS system, which may be Moveit2, integrates with a Motion Planning Framework (MPF), which plans movement of the robotic arm. The MPF interacts with a Hardware Abstraction Layer (HAL), which provides a unified interface for interacting with different motors. The Web Application also interfaces with a camera trigger module, which triggers the camera to start and stop the recording process. The operator commands the robot using a Web Application (“Web App”), which interfaces with the software stack via a Representation State Transfer (REST) Application Programming Interface (API).

[0085] The Web Application is available on the CPU in the robotic arm. The operator pad can be operatively connected to the CPU wirelessly through wireless communication, such as by Ethernet (LAN), by wireless internet (WiFi), BLUETOOTH®, or other wired or wireless communication, thereby allowing an operator to access the Web Application from a separate computing article, such as a desktop, a laptop, a smartphone, a tablet, or a computing interface available on an electronic device. Other suitable communication modes can also be used.

[0086] The ROS Software is present and operates on the CPU inside the robotic arm. Interaction between the ROS Software and the Web App is via the REST API. The REST API endpoints provide a standardized interface for controlling and monitoring the robotic arm, enhancing interoperability and enabling integration with web-based interfaces. Data sent between the Web App and the ROS Software via REST API is over a network, which can be a wireless network, or by wires. The ROS Software commands the camera trigger module through a microcontroller unit (MCU), which connects to the camera by BLUETOOTH® connection. The ROS Software running on the CPU communicates with the MCU by serial communication, andthe MCU sends a command to the camera over BLUETOOTH® connection to start or stop recording. In some embodiments, the MCU can be an ESP32 MCU by ESPRESSIE

[0087] The MPF plans a collision free path, generates a trajectory path, and executes the path for the robotic arm while avoiding self-collision. Using Movelt, the Motion Planning Framework generates a collision free path, which is a waypoint from point A to point B, point A being the starting point and point B being the ending point. The MPF then generates a trajectory for the collision free path. In the trajectory, timing information on how to execute the path in a shortest and / or optimal amount of time is included. Each of the waypoints from point A to point B is assigned timing information. Time-Optimal Trajectory Generation may be used as the algorithm to plan the trajectories. The trajectories are designed to move the camera in 3D space while pointing the camera towards a fixed focus point in space. For a position of the camera in space, there is a specific quaternion that would point the camera towards the target focus point. A spline interpolation is performed between waypoints on the path to obtain the trajectory with the orientation constraint defined by the position of the camera in 3D space. Other methods may also be used to generate a trajectory for a path. The MPF receives commands from the ROS Application and plans for a specific path. After a path is planned, the Movelt software component executes the planned path under the control of a Joint Trajectory Controller software component. This ensures the robotic arm does not steer off the planned path. With a collision free path planned by the MPF, the robotic arm according to embodiments herein avoids collision with itself.

[0088] The hardware to be controlled in this robotic arm includes a motor (or a plurality of motors) and aspects of the system operatively connected with identified motors are controlled by operating the motor. Motors in the presently described system can be grouped, for example, into two families, according to their functionalities or manufacturing sources. Motor family 1 includes, for example, the motor in the pedestal for shoulder panning, the shoulder lift motor, and the elbow motor. Motor family 2 includes, for example, three (3) motors at the wrist. The motors herein are divided into families according to their manufacturers and consequently their communication configuration. Each motor in the same family has software for control, and the software may be obtained from a Software Development Kit (SDK) library. Each motor in the same family has or may have the same or similar software developed from the SDK and capable of interacting with the HAL in the same manner. The HAL is an interface between the MPF and software from the SDKs within motor families. The HAL ensures consistent control commandsand data exchange among the different motor hardware, such that interoperability between the motors is enhanced.

[0089] The ROS Application commands the camera trigger to start and interacts with the MPF and the Web App. The camera trigger is a module receiving commands from the ROS Application and communicating with the camera held by the robotic arm at the end effector via Bluetooth communication. The camera trigger commands the camera to start recording.

[0090] The Web Application contains pre-determined pathways, which are displayed at the operator pad on a separate computing article and can be selected by a user. Pre-determined pathways define pathways in which the robotic arm moves and consequently moves the camera in the end effector with it. As the camera moves along the pre-defined pathway, pictures and / or videos are taken and recorded.

[0091] In operation, the operator sends commands for video recording on the operator pad, such as on a tablet, the commands include a chosen path of movement for the camera. The commands are received by the Web App and transmitted to the ROS, using a REST API. The REST API transmits the commands to the ROS, which commands the MFP to plan for the path. Upon completion of planning, the MFP outputs the control signals for the motors which are transmitted to the motors through the HAL to control the motors to move to carry out the path. The ROS Software also communicates with the camera trigger module to send signal via Bluetooth to the camera on the end effector.

[0092] The CPU is connected to the various motors by wires, which may be hidden inside the pedestal and the robotic arm for safety, security and aesthetics reasons. Wire connection ensures that data and commands are transmitted quickly, and the robotic arm’s operation is not affected by strength of WiFi or wireless network.

[0093] In a first embodiment, a modular apparatus to move a camera during video recording or picture taking is provided, comprising: a base with a plurality of casters configured to move on different surfaces; a pedestal attached to the base at one end of the pedestal; upper arm portion attached to the pedestal at a shoulder joint, the shoulder joint having a shoulder motor configured to rotate the upper arm around the shoulder joint; a pedestal motor contained in the pedestal and connected to the shoulder joint, the pedestal motor configured to rotate the shoulder joint around the vertical central axis of the pedestal;a forearm portion attached to the upper arm portion at an elbow joint, the elbow joint having an elbow motor configured to rotate the forearm around the elbow joint; a wrist assembly attached to the forearm at a wrist joint, the wrist assembly having at least one wrist motor configured to rotate the wrist assembly around the wrist joint; an end effector attached to the wrist assembly and configured to hold a camera; a central processing unit configured to control the pedestal motor, the shoulder motor, the elbow motor, and the at least one wrist motor, a microcontroller unit having Bluetooth connection capability and configured to trigger a camera; at least one power supply to supply power to the central processing unit, the pedestal motor, the shoulder motor, the elbow motor, and the at least one wrist motor; a computer programming product operable on the central processing unit, comprising: a robotic operating system configured to command and control the robotic arm; a motion planning framework configured to plan movement of the robotic arm between way points; a hardware abstraction layer configured to interface with the motion planning framework and software within motors; a web application operable on the central processing unit and configured to be accessible from a separate computing article; and a Representation State Transfer Application Programming Interface configured to interface between the robotic operating system and the web application, wherein each motor has software configured to interact with the hardware abstraction layer within the computer programming product, wherein the web application is configured to interact with robotic operating system via the Representation State Transfer Application Programming Interface, and wherein the robotic operating system is configured to command the microcontroller unit to trigger the camera.

[0094] In a second embodiment, the first embodiment includes wherein the wrist assembly comprises:a roll motor having a roll motor output shaft attached to a roll motor horn, the roll motor horn attached a roll motor output shaft through a roll motor bearing assembly, the roll motor output shaft rotates around the central axis of the forearm; a pitch motor mounting bracket attached to roll motor output shaft; a pitch motor having a pitch motor output shaft attached to a pitch motor horn, the pitch motor horn attached to a pitch motor output shaft through a pitch motor bearing assembly, the pitch motor output shaft rotates around an axis perpendicular to the central axis of the forearm; a yaw motor mounting bracket attached to the pitch motor output shaft; a yaw motor having a yaw motor output shaft attached to a yaw motor horn, the yaw motor horn attached to a yaw motor output shaft through a yaw motor bearing assembly, the yaw motor output shaft rotates around an axis perpendicular to the central axis of the forearm and the pitch motor output shaft axis; and a universal attachment attached to the yaw motor mounting bracket and is operatively connected to the yaw motor output shaft, the universal attachment is capable of rotating by the yaw motor output shaft, wherein the universal attachment is attachable to the end effector.

[0095] In a third embodiment, the second embodiment includes wherein the roll motor, the pitch motor, and the yaw motor are servo motors.

[0096] In a fourth embodiment, the first embodiment includes wherein the pedestal motor, the shoulder motor, and the elbow motor are stepper motors.

[0097] In a fifth embodiment, the first embodiment includes wherein the end effector comprises: a housing with a space configured to hold a camera; at least one LED light on the housing, the at least on LED light is embedded into the housing; electronic components to control the at least one LED light; wherein the housing is configured to attached to the universal attachment, wherein the electronic components are operatively connected to the central processing unit.

[0098] In a sixth embodiment, the fifth embodiment includes wherein the end effector further comprises a plastic diffuser to protect the at least one LED light.

[0099] In a seventh embodiment, the fifth embodiment includes wherein the end effector further comprises a clamp configured to hold cameras of different sizes.

[0100] In an eighth embodiment, the first embodiment includes wherein the pedestal comprises concentric plates at the top attached to the shoulder joint via a dowel pin.

[0101] In a ninth embodiment, the first embodiment includes wherein the central processing unit is configured to communicate with the microcontroller unit via serial communication.

[0102] In a tenth embodiment, the first embodiment further comprises a pedestal motor brake configured to provide braking function to the pedestal motor.

[0103] In an eleventh embodiment, the first embodiment further comprises a communication hub configured to receive commands from the central processing unit and send commands to the motors and the end effector.

[0104] In a twelfth embodiment, the eleventh embodiment further comprises an emergency stop switch capable of being activated to stop power supplied to the apparatus.

[0105] In a thirteenth embodiment, the first embodiment includes wherein the separate computing article comprises desktop, a laptop, a smartphone, or a tablet.

[0106] In a fourteenth embodiment, the first embodiment includes wherein the web application is accessible from the separate computing article via Ethernet or a wireless network.

[0107] In a fifteenth embodiment, the first embodiment includes wherein the motion planning framework is configured to avoid self-collision.

[0108] In a sixteenth embodiment, the first embodiment includes wherein the computer programming product is configured to carry out pre-planned movement path for the robotic arm.

[0109] In a seventeenth embodiment, the first embodiment includes wherein each of the base, the pedestal, the shoulder joint, the upper arm portion, the forearm portion, the wrist assembly, and the end effector is attached to other parts of the modular apparatus using removable attachments.

[0110] In an eighteenth embodiment, the seventeenth embodiment includes wherein the removable attachments are removable screws.

[0111] In a nineteenth embodiment, the eighteenth embodiment includes wherein the motion planning framework is further configured to avoid self-collision of the modular apparatus.

[0112] In a twentieth embodiment, a computer programming product operable on a computing article is provided, the computer programming product comprises: a robotic operating system configured to command and control a robotic arm; a motion planning framework configured to plan movement of the robotic arm between way points; a hardware abstraction layer configured to interface with the motion planning framework and motor software; a web application operable on the central processing unit and configured to be accessible from a separate computing article; and a Representation State Transfer Application Programming Interface configured to interface between the robotic operating system and the web application; wherein the web application is configured to interact with robotic operating system via the Representation State Transfer Application Programming Interface, and wherein the robotic operating system is configured to command the microcontroller unit to trigger the camera.

[0113] In a twenty first embodiment, the twentieth embodiment includes wherein the web application is accessible from the separate computing article via Ethernet or a wireless network.

[0114] In a twenty second embodiment, the twentieth embodiment includes wherein the motion planning framework is configured to avoid self-collision.

[0115] In a twenty third embodiment, the twentieth embodiment includes wherein the computer programming product is configured to carry out pre-planned movement path for the robotic arm.

[0116] In a twenty fourth embodiment, a method to take a video recording is provided, comprising: positioning a camera in the end effector of the modular apparatus according to the nineteenth embodiment; activating a computer programming product configured to communicate with the central processing unit in the robotic arm and control the robotic arm; choosing a pathway from a collection of pathways available in the computer programming product;commanding the robotic arm to move according to the chosen pathway; and recording a video using the camera in the end effector.

[0117] In a twenty fifth embodiment, the twenty fourth embodiment includes wherein activating the computer programming product is carried out on the separate computing article.

[0118] Citation of the above publications or documents is not intended as an admission that any of the foregoing is pertinent prior art, nor does it constitute any admission as to the contents or date of these publications or documents.

Claims

CLAIMSWe claim:

1. A modular apparatus to move a camera during video recording or picture taking, comprising: a base with a plurality of casters configured to move on different surfaces; a pedestal attached to the base at one end of the pedestal; an upper arm portion attached to the pedestal at a shoulder joint, the shoulder joint having a shoulder motor configured to rotate the upper arm around the shoulder joint; a pedestal motor contained in the pedestal and connected to the shoulder joint, the pedestal motor configured to rotate the shoulder joint around the vertical central axis of the pedestal; a forearm portion attached to the upper arm portion at an elbow joint, the elbow joint having an elbow motor configured to rotate the forearm around the elbow joint; a wrist assembly attached to the forearm at a wrist joint, the wrist assembly having at least one wrist motor configured to rotate the wrist assembly around the wrist joint; an end effector attached to the wrist assembly and configured to hold a camera; a central processing unit configured to control the pedestal motor, the shoulder motor, the elbow motor, and the at least one wrist motor, a microcontroller unit having Bluetooth connection capability and configured to trigger a camera; at least one power supply to supply power to the central processing unit, the pedestal motor, the shoulder motor, the elbow motor, and the at least one wrist motor; a computer programming product operable on the central processing unit, comprising: a robotic operating system configured to command and control the robotic arm; a motion planning framework configured to plan movement of the robotic arm between way points; a hardware abstraction layer configured to interface with the motion planning framework and software within motors; a web application operable on the central processing unit and configured to be accessible from a separate computing article; and a Representation State Transfer Application Programming Interface configured to interface between the robotic operating system and the web application,wherein each motor has software configured to interact with the hardware abstraction layer within the computer programming product, wherein the web application is configured to interact with robotic operating system via the Representation State Transfer Application Programming Interface, and wherein the robotic operating system is configured to command the microcontroller unit to trigger the camera.

2. The modular apparatus of claim 1, wherein the wrist assembly comprises: a roll motor having a roll motor output shaft attached to a roll motor hom, the roll motor horn attached a roll motor output shaft through a roll motor bearing assembly, the roll motor output shaft rotates around the central axis of the forearm; a pitch motor mounting bracket attached to roll motor output shaft; a pitch motor having a pitch motor output shaft attached to a pitch motor horn, the pitch motor horn attached to a pitch motor output shaft through a pitch motor bearing assembly, the pitch motor output shaft rotates around an axis perpendicular to the central axis of the forearm; a yaw motor mounting bracket attached to the pitch motor output shaft; a yaw motor having a yaw motor output shaft attached to a yaw motor hom, the yaw motor hom attached to a yaw motor output shaft through a yaw motor bearing assembly, the yaw motor output shaft rotates around an axis perpendicular to the central axis of the forearm and the pitch motor output shaft axis; and a universal attachment attached to the yaw motor mounting bracket and is operatively connected to the yaw motor output shaft, the universal attachment is capable of rotating by the yaw motor output shaft, wherein the universal attachment is attachable to the end effector.

3. The modular apparatus of claim 2, wherein the roll motor, the pitch motor, and the yaw motor are servo motors.

4. The modular apparatus of claim 1, wherein the pedestal motor, the shoulder motor, and the elbow motor are stepper motors.

5. The modular of claim 1 , wherein the end effector comprises: a housing with a space configured to hold a camera; at least one LED light on the housing, the at least on LED light is embedded into the housing; electronic components to control the at least one LED light; wherein the housing is configured to attached to the universal attachment, wherein the electronic components are operatively connected to the central processing unit.

6. The modular apparatus of claim 5, wherein the end effector further comprises a plastic diffuser to protect the at least one LED light.

7. The modular apparatus of claim 5, wherein the end effector further comprises a clamp configured to hold cameras of different sizes.

8. The modular apparatus of claim 1 , wherein the pedestal comprises concentric plates at the top attached to the shoulder joint via a dowel pin.

9. The modular apparatus of claim 1 , wherein the central processing unit is configured to communicate with the microcontroller unit via serial communication.

10. The modular apparatus of claim 1, further comprising a pedestal motor brake configured to provide braking function to the pedestal motor.

11. The modular apparatus of claim 1, further comprising a communication hub configured to receive commands from the central processing unit and send commands to the motors and the end effector.

12. The modular apparatus of claim 11, further comprising an emergency stop switch capable of being activated to stop power supplied to the apparatus.

13. The modular apparatus of claim 1 , wherein the separate computing article comprises desktop, a laptop, a smartphone, or a tablet.

14. The modular apparatus of claim 1, wherein the web application is accessible from the separate computing article via Ethernet or a wireless network.

15. The modular apparatus of claim 1, wherein the motion planning framework is configured to avoid self-collision.

16. The modular apparatus of claim 1, wherein the computer programming product is configured to carry out pre-planned movement path for the robotic arm.

17. The modular apparatus of claim 1, wherein each of the base, the pedestal, the shoulder joint, the upper arm portion, the forearm portion, the wrist assembly, and the end effector is attached to other parts of the modular apparatus using removable attachments.

18. The modular apparatus of claim 17, wherein the removable attachments arc removable screws.

19. The modular apparatus of claim 18, wherein the motion planning framework is further configured to avoid self-collision of the modular apparatus.

20. A computer programming product operable on a computing article, the computer programming product comprises: a robotic operating system configured to command and control a robotic arm; a motion planning framework configured to plan movement of the robotic arm between way points;a hardware abstraction layer configured to interface with the motion planning framework and motor software; a web application operable on the central processing unit and configured to be accessible from a separate computing article; and a Representation State Transfer Application Programming Interface configured to interface between the robotic operating system and the web application; wherein the web application is configured to interact with robotic operating system via the Representation State Transfer Application Programming Interface, and wherein the robotic operating system is configured to command the microcontroller unit to trigger the camera.

21. The computer programming product of claim 20, wherein the web application is accessible from the separate computing article via Ethernet or a wireless network.

22. The computer programming product of claim 20, wherein the motion planning framework is configured to avoid self-collision.

23. The computer programming product of claim 20, wherein the computer programming product is configured to carry out pre-planned movement path for the robotic arm.

24. A method to take a video recording, comprising: positioning a camera in the end effector of the modular apparatus according to claim 19; activating a computer programming product configured to communicate with the central processing unit in the robotic arm and control the robotic arm; choosing a pathway from a collection of pathways available in the computer programming product; commanding the robotic arm to move according to the chosen pathway; and recording a video using the camera in the end effector.

25. The method of claim 24, wherein activating the computer programming product is carried out on the separate computing article.

Images