Artificial intelligence robotics platform

Abstract

Methods and systems for an artificial intelligence robotics platform are provided. The methods and systems include digitizing data determined by a robotic sensing patch system of a robotic body part, the robotic sensing patch system includes a sensing patch and a protective membrane. The methods and systems also include determining, based on the digitized data, first information that includes a total amount of forces or a total number of forces applied to the object by one or more phalanges of the robotic body part. The methods and systems further include determining, based on criteria and the first information, one or more positions of the one or more phalanges to augment the total amount of forces or the total number of forces. The methods and systems further include transmitting instructions to control the one or more phalanges to further augment the total amount of forces or the total number of forces.

Description

[0001] ARTIFICIAL INTELLIGENCE ROBOTICS PLATFORM

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims benefit of and priority to U.S. provisional patent application Ser. No. 63 / 663,605 filed June 24, 2024.

[0004] TECHNOLOGICAL FIELD

[0005] Exemplary embodiments of this disclosure relate generally tomethods, apparatuses, or computer program products for an artificial intelligence robotics platform.

[0006] BACKGROUND

[0007] Robotics and artificial intelligence (Al) have revolutionized numerous industries and aspects of people’s lives, transforming the way people work, interact, and approach complex problems. Robotics has enabled the automation of tasks that were previously too dangerous, difficult, or repetitive for humans, freeing us up to focus on higher-value tasks. Meanwhile, Al has enabled machines to learn from data, make decisions autonomously, and improve their performance over time. The integration of Al and robotics has given rise to applications such as autonomous vehicles, robotic assistants, and smart homes.

[0008] SUMMARY

[0009] Disclosed herein are methods, apparatuses, and / or systems for creating and using a technology platform for robotics and artificial intelligence. In an example, the disclosed platform may allow for connections with and interactions with robotic sensors and robotic manipulators in a way that may optimize deployments with artificial intelligence.

[0010] According to an aspect, there is provided a robotic sensing patch system comprising: at least one processor; a sensing patch comprising an array of sensing taxels, wherein the sensing patch is configured to communicate sensor data, determined in response to interactions of a robotic body part with an object, to the at least one processor; and a membrane configured to convert a physical signal to an electromagnetic signal.

[0011] In one embodiment, the robotic sensing patch system further comprises: a sensing spacer coupled with the sensing patch and the membrane; wherein: the sensing spacer protects at least one sensing taxel; and the membrane comprises an artificial membrane mimicking human skin.

[0012] In one embodiment, the array of sensing taxels includes at least one of a capacitive based sensor, a magnetic based sensor, a resistive based sensor, or other electromagnetic based sensor.

[0013] In one embodiment, the array of sensing taxels includes at least one of a pressure based sensor or an audio based sensor.

[0014] According to another aspect, there is provided an apparatus comprising: a first sensing patch comprising a plurality of first sensing taxels; a second sensing patch comprising a plurality of second sensing taxels; a first robotic processor communicatively connected with the first sensing patch; a second robotic processor communicatively connected with the second sensing patch; a communication bus connected with the second local robotic processor; and an appendage processing component configured to receive or process sensor data from one or more areas of a robotic appendage.

[0015] In one embodiment, the robotic appendage comprises a robotic hand.

[0016] In one embodiment, the robotic appendage comprises a robotic foot.

[0017] In one embodiment: the appendage processing component comprises an artificial intelligence (Al) processing component configured to control at least a first robotic part and a second robotic part of the robotic appendage; and the first robotic processor and the second robotic processor are embedded within, or associated with the first robotic part.

[0018] In one embodiment: the appendage processing component comprises an expandable input / output component; and the expandable input / output component is configured to control a peripheral of a robot.

[0019] In one embodiment, the apparatus further comprises a robotic sensing patch system comprising a robotic membrane and the first sensing patch.

[0020] In one embodiment, the robotic sensing patch system further comprises a sensing spacer coupled with the first sensing patch.

[0021] In one embodiment: the sensing spacer is coupled with the first sensing patch; and the sensing spacer protects at least one sensing taxel of the first sensing patch.

[0022] According to a further aspect, there is provided a method comprising: digitizing data determined by a robotic sensing patch system of a robotic body part that interacts with an object, wherein the robotic sensing patch system comprises a sensing patch and a protective membrane; determining, based on the digitized data, first information that comprises a total amount of forces or a total number of forces being applied to the object by one or more phalanges of the robotic body part; determining, based on criteria and the first information, one or more positions of the one or more phalanges of the robotic body part to augment the total amount of forces or the total number of forces being applied to the object; and transmitting instructions to control the one or more phalanges of the robotic body part to further augment the total amount of forces or the total number of forces being applied to the object.

[0023] In one embodiment, the determining the one or more positions of the one or more phalanges of the robotic body part to augment the total amount of forces or the total number of forces being applied to the object is executed by a machine learning model.

[0024] In one embodiment, the machine learning model utilizes data from the sensing patch to facilitate training of the machine learning model.

[0025] In one embodiment, the criteria comprises prevention of the object from dropping.

[0026] In one embodiment, the criteria comprises rotation of the object on one or more fingertips of the robotic body part.

[0027] In one embodiment, the robotic body part comprises a robotic hand.

[0028] In one embodiment, the robotic body part comprises a robotic foot.

[0029] In one embodiment, the method further comprises determining the total amount of forces or the total number of forces being applied to the object, wherein the one or more phalanges manipulate the object based on the total amount of forces or the total number of forces.

[0030] In one embodiment, the data from the sensing patch is associated with movement of the robotic body part.

[0031] In some example aspects, a robotic sensing patch system is provided. The robotic sensing patch system may include at least one processor and may include a sensing patch including an array of sensing taxels. The sensing patch may be configured to communicate sensor data, determined in response to interactions of a robotic body part with an object, to the at least one processor. The robotic sensing patch may further include a membrane configured to convert a physical signal to an electromagnetic signal.

[0032] In some other example aspects, an apparatus is provided. The apparatus may include a first sensing patch including a plurality of first sensing taxels. The apparatus may also include a second sensing patch including a plurality of second sensing taxels. The apparatus may also include a first robotic processor communicatively connected with the first sensing patch. The apparatus may also include a second robotic processor communicatively connected with the second sensing patch. The apparatus may also include an appendage processing component configured to receive or process sensor data from one or more areas of a robotic appendage.

[0033] In yet some other example aspects, a method is provided. The method may include digitizing data determined by a robotic sensing patch system of a robotic body part that interacts with an object. The robotic sensing patch system may include a sensing patch and a protective membrane. The method may further include determining, based on the digitized data, first information that includes a total amount of forces or a total number of forces being applied to the object by one or more phalanges of the robotic body part. The method may further include determining, based on criteria and the first information, one or more positions of the one or more phalanges of the robotic body part to augment the total amount of forces or the total number of forces being applied to the object. The method may further include transmitting instructions to control the one or more phalanges of the robotic body part to further augment the total amount of forces or the total number of forces being applied to the object.

[0034] In an example, an apparatus may include a first sensing patch, wherein the first sensing patch may include a plurality of first sensing taxels; a second sensing patch, wherein the second sensing patch may include a plurality of second sensing taxels; a first local robotic processor communicatively connected with the first sensing patch; a second local robotic processor communicatively connected with the second sensing patch, wherein the first local robotic processor and the second local robotic processor are connected in series; a communication bus connected with the second local robotic processor; and an appendage processing component communicatively connected with the communication bus, the appendage processing component configured to receive or process sensor data from one or more areas of a robotic appendage.

[0035] In another example, a robotic sensing patch system may include a printed circuit board; a sensing patch coupled with the printed circuit board, wherein the sensing patch comprises an array of sensing taxels, wherein the printed circuit board may be configured to communicate sensor data from the sensing patch to a processor; and a membrane configured to convert a physical signal to an electromagnetic signal.

[0036] In one example of the present disclosure, a robotic sensing patch system is provided. The robotic sensing patch system may include at least one processor and a printed circuit board. The robotic sensing patch system may also include a sensing patch coupled with the printed circuit board. The sensing patch may include an array of sensing taxels. The printed circuit board may be configured to communicate sensor data from the sensing patch to the at least one processor. The robotic sensing patch system may also include a membrane configured to convert a physical signal to an electromagnetic signal, or to provide an interface between an environment and one or more sensing electronics. The environment may be a real-world environment. In some examples, the printed circuit board may be flexible. In other examples, the printed circuit board may be rigid (i.e., non-flexible). In yet some other examples, the printed circuit board may be a rigid-flexible printed circuit board (e.g., a combination of a rigid and flexible printed circuit board).

[0037] In another example of the present disclosure, an apparatus is provided. The apparatus may include a first sensing patch that may include a plurality of first sensing taxels. The apparatus may further include a second sensing patch that may include a plurality of second sensing taxels. The apparatus may further include a first local robotic processor communicatively connected with the first sensing patch. The apparatus may further include a second local robotic processor communicatively connected with the second sensing patch. The first local robotic processor and the second local robotic processor may be connected in series. The apparatus may further include a communication bus connected with the second local robotic processor. The apparatus may further include an appendage processing component communicatively connected with the communication bus. The appendage processing component may be configured to receive or process sensor data from one or more areas of a robotic appendage.

[0038] In yet another example of the present disclosure, a method is provided. The method may include digitizing data obtained by a robotic sensing patch system of a robotic hand that interacts with an object. The robotic sensing patch system may include a printed circuit board, a sensing patch, and a protective membrane. The method may further include determining, based on the digitized data, first information that may include a total amount of forces or a total number of forces being applied to the object by one or more phalanges of the robotic hand. The method may further include determining, based on criteria and the first information, one or more positions of the one or more phalanges of the robotic hand to augment the total amount of forces or the total number of forces being applied to the object. The method may further include transmitting instructions to control the one or more phalanges of the robotic hand to further augment the total amount of forces or the total number of forces being applied to the object. In some examples, the printed circuit board may be flexible. In other examples, the printed circuit board may be rigid (i.e., non-flexible). In yet some other examples, the printed circuit board may be a rigid-flexible printed circuit board. It will be appreciated that any features described herein as being suitable for incorporation into one or more aspects or embodiments of the present disclosure are intended to be generalizable across any and all aspects and embodiments of the present disclosure. Other aspects of the present disclosure can be understood by those skilled in the art in light of the description, the claims, and the drawings of the present disclosure. The foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the claims.

[0039] DESCRIPTION OF THE DRAWINGS

[0040] The summary, as well as the following detailed description, is further understood when read in conjunction with the appended drawings. For the purpose of illustrating the disclosed subject matter, there are shown in the drawings exemplary embodiments of the disclosed subject matter; however, the disclosed subject matter is not limited to the specific methods, compositions, and devices disclosed. In addition, the drawings are not necessarily drawn to scale. In the drawings:

[0041] FIG. 1 illustrates an example robotic appendage in accordance with an example of the present disclosure.

[0042] FIG. 2A illustrates an example component associated with a robotic sensing patch system in accordance with an example of the present disclosure.

[0043] FIG. 2B illustrates an example component associated with a robotic sensing patch system in accordance with an example of the present disclosure.

[0044] FIG. 2C illustrates an example component associated with a robotic sensing patch system in accordance with an example of the present disclosure.

[0045] FIG. 3A illustrates an example artificial intelligence robotics platform in accordance with an example of the present disclosure.

[0046] FIG. 3B illustrates another example artificial intelligence robotics platform in accordance with an example of the present disclosure.

[0047] FIG. 4 illustrates a machine learning and training model in accordance with various examples of the present disclosure.

[0048] FIG. 5 illustrates an example method associated with a robotics platform in accordance with an example of the present disclosure.

[0049] FIG. 6 illustrates an example diagram of an exemplary computing system in accordance with an example of the present disclosure.

[0050] FIG. 7 illustrates another example method associated with a robotics platform in accordance with an example of the present disclosure.

[0051] The figures depict various embodiments for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles described herein.

[0052] DETAILED DESCRIPTION

[0053] Some embodiments of the present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all embodiments of the invention are shown. Various embodiments of the invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Like reference numerals refer to like elements throughout.

[0054] It is to be understood that the methods and systems described herein are not limited to specific methods, specific components, or to particular implementations. It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting.

[0055] As defined herein a “computer-readable storage medium,” which refers to a non- transitory, physical or tangible storage medium (e.g., volatile or non-volatile memory device), may be differentiated from a “computer-readable transmission medium,” which refers to an electromagnetic signal.

[0056] As referred to herein, phalanges may refer to one or more articulated segments that mimic or represent bones of a body part(s) such a robotic body part(s) (e.g., a robotic hand, a robotic foot, etc.) mimicking / representing a body part(s) of a person. In this regard, for example, a robotic hand (e.g., mechanical representations of bones of a human hand’s fingers) and / or a robotic foot (e.g., mechanical representations of bones of a human foot’s toes) may have phalanges. In some examples, a robotic hand may include, but is not limited to, phalanges such as, for example, proximal phalanges, middle phalanges and / or distal phalanges. Additionally, in some examples a robotic foot may have phalanges, including, but not limited to, a robotic big toe (e.g., a robotic hallux) having a proximal phalange and / or a distal phalange and one or more other toes of a robotic foot may, but need not, include a proximal phalange, a middle phalange, and / or a distal phalange.

[0057] Robotic dexterous manipulation conventionally is an unsolved problem. Fingertip or the like sensing while digitizing the world is a step towards general useful manipulation between embodied artificial intelligence (Al) and a physical environment. In physical embodied Al, conventional methods may lack a way to collect large amounts of data for training purposes across different use cases. While alternative methods may be used, such as through simulation, the simulation to real gap may be large due to the intricate detail in contact dynamics in soft- body interactions. While this may aid in successfully completing task specific actions, a method for stable grasp to dexterous manipulation is desired.

[0058] The disclosed Al robotics platform may provide standardization to connect and interact with robotic sensors (e.g., robotic fingertips, robotic membranes, or other sensors) and robotic manipulators (e g., robotic hands or actuators). In some examples, an Al robotics platform may encompass a robotic sensing patch system, as further described herein, and may implement Al robotics. The Al robotics platform may be a composition of hardware and software and may be responsible for interfacing with multiple signal paths into a single processor which may encode the data and may supply this data to a host computer. In some examples, the Al robotics platform may be implemented, based in part, on an Al model(s) and / or a machine learning model(s) (e.g., machine learning model(s) 410 of FIG. 4).

[0059] FIG. 1 illustrates an example robotic hand. Robotic hand 10 may be designed to mimic the functionality and dexterity of a human hand. Robotic hand 10 may include palm 15, finger 20 (e.g., small finger), finger 30 (e.g., ring or middle finger), finger 40 (e.g., index finger), or finger 50 (e.g., thumb), which may be interconnected by a series of joints (e.g., joint 11) or actuators. As in a human hand, palm 15 may serve as a base of robotic hand 10 and palm 15 may provide a stable platform for fingers 20, 30, 40, 50 to move in relation to. Fingers or the like (e.g., toes) generally number between 2 and 5 but may include more.

[0060] As described herein, which may be similar to a human hand, the fingers of robotic hand 10 may include multiple phalanges (e.g., in area 22 - 24 (e.g., areas 22, 23, 24), area 32 - 34 (e.g., areas 32, 33, 34), or area 42 - 44 (e.g., areas 42, 43, 44), etc.), each connected by a joint (e.g., joint 11), allowing for flexion, extension, and / or rotation. In some examples, area 22, area 23 and area 24 may be referred to herein as phalange 22, phalange 23, and phalange 24. Similarly, area 32, area 33, and area 34 may be referred to herein as phalange 32, phalange 33, and phalange 34 and area 42, area 43, and area 44 may be referred to herein as phalange 42, phalange 43, and phalange 44. Finger 20 may include area 21, area 22, area 23, or area 24. Finger 30 may include area 31, area 32, area 33, or area 34. Finger 40 may include area 41, area

[0061] 42, area 43, or area 44. Finger 50 may include area 51, area 52, or area 53. Palm 15 may include area 25, area 35, area 45, area 46, or area 54. As described herein, each area may include one or more processors, actuators, or sensors.

[0062] For additional perspective, robotic hand 10 is further described in relation to a human hand. Fingertips may include area 21, area, 31, area 41, and / or area 51. Area 22, area 32, area 42, and / or area 52 may be associated with the distal phalanx (e.g., of a hand). Area 23, area 33, area

[0063] 43, and / or area 53 may be associated with the middle phalanx (e.g., of a hand). Area 24, area 34, and / or area 44 may be associated with the proximal phalanx (e.g., of a hand). Area 54 may be associated with the thenar (e.g., of a hand). Area 25, area 35, area 45, and / or area 46 may be associated with the distal palmar or proximal palmar (e.g., of a hand). Area 13 may be associated with the hypothenar (e.g., of a hand).

[0064] The movement of robotic hand 10, or a robotic foot, may be facilitated by a sophisticated control system (e.g., Al robotics platform 100), which integrates sensor feedback, motor control algorithms, kinematic models, and / or machine learning models to ensure precise and coordinated movement, such as with reference to FIG. 3A. The control system may be a hierarchical architecture, comprising low-level motor control, mid-level kinematic control, or high-level task control.

[0065] FIG. 2A, FIG. 2B, and FIG. 2C illustrate example sensors that may be associated with robotic hand 10, a robotic foot, and / or other robotic body parts. These sensors may be located in the areas described herein with reference to FIG. 1 or finely layered on one or more sides of robotic hand 10. The sensory capabilities of robotic hand 10 may be provided by a network of sensors, including tactile sensors, force sensors, or position sensors, which collectively may provide a rich source of feedback on the hand’s (or foot’s) environment and movement. The tactile sensors, embedded in the fingers (e g., finger 20) and palm 15, may detect pressure, temperature, or texture, allowing robotic hand 10 (or a robotic foot) to feel and respond to its surroundings. Force sensors, located at the joints or fingertips, may measure the force applied by the hand, and may implement measures to prevent damage to the robotic hand 10 (or a robotic foot) or an object 7 being manipulated. Position sensors, integrated into the joints and actuators, may track the hand's movement and orientation, which may enable precise control and movement.

[0066] The main sensing modality of vision-based tactile sensors may capture the geometry of the object (e.g., object 7) being touched. From that geometric data it may be possible to reconstruct normal and shear forces. However, that particular modality may fall short of the multimodal nature of human skin, which uses many different types of receptors (such as mechanoreceptors, thermoreceptors or nociceptors). In contrast with the conventional efforts, the disclosed exemplary subject matter of the present disclosure, as shown in FIG. 2A and FIG. 2B, may include multiple sensors which may be within a sensing patch 60. The disclosed sensing system (e.g., robotic sensing patch system) may include a multimodal sensor system, metal-oxide semiconductor (MOS), an image capture system, processing, an artificial intelligence module, or data transfer system.

[0067] FIG. 2A illustrates an example sensing patch 60. Sensing patch 60 may include a plurality of sensing taxels (e.g., sensor points), such as sensing taxel 61, sensing taxel 62, or sensing taxel 63. Sensing patch 60 may be considered a vessel that may carry an array of sensing taxels and may be bound to a type of robotic membrane 71 (e.g., robotic skin). Sensing patch 60 may include one type or a combination of different types of sensors. In an example, robotic hand 10 may include high-resolution sensors (e.g., millions of taxels) that respond to omnidirectional touch, capture multimodal signals, and use artificial intelligence to process the data in real time. Sensing taxels 61 may include capacitive based sensors, magnetic based sensors, resistive based sensors, pressure based sensors, or audio based sensors, among other sensors. In an example with reference to an audio sensor, microphones may be analog which differ in their sensing frequency ranges to cover a larger bandwidth. These microphones may provide surface audio textures similar to what a human’s fingertip may perceive when scratching the surface of an object (e.g., object 7), sampling object-to-object interactions, or object-to-environment interactions.

[0068] In an example with reference to a resistive sensor, piezoresistive materials may be analog which differ in their internal resistance and material properties to cover a range of force sensitivities. Resistive sensors may be used for force sensing in situations that may include ferrous materials due to resistive sensing being less susceptible to magnetic ferrous materials. Capacitive sensors may be used for normal force, while magnetic sensors may be used for multiaxis normal and shear force. A combination of these sensors may be useful to obtain or collect multi-modal data which may be used in downstream machine learning (ML) models (e.g., machine learning model(s) 410) which learn cross-modal effects of sensor noise to sensor signal, which may increase the signal to noise ratio (SNR). The combination of multiple underlying modes may be used to introduce redundancy to self-training and react to the variety of stimuli found in varying environments.

[0069] FIG. 2B illustrates an example robotic sensing patch system 70 that may be implemented with a robotic hand 10 or other parts of a robot (e.g., a robotic foot, a robotic arm, etc.). Robotic sensing patch system 70 illustrates an example component stack that may include robotic membrane 71, sensing spacer 72, sensing patch 60, and / or a printed circuit board 73. It is contemplated that components may not be stacked in the order as shown in FIG. 2B. In some examples, the printed circuit board 73 may be a flexible printed circuit board. In other examples, the printed circuit board 73 may be a rigid printed circuit board. In some other examples, the printed circuit board 73 may be a rigid-flexible printed circuit board (e.g., a combination of a rigid and flexible printed circuit board).

[0070] Robotic membrane 71 may be an overmolded material (e.g., silicone, rubber, or plastic) that may include particulate that induce electromagnetic perturbations in a local field. The sensors perceive a change in the electromagnetic field, however, in order to create changes in the electromagnetic field, a medium may be used to convert physical impressions into electromagnetic changes by altering the volumetric shape or physical properties of the sensing medium. For example, mechanically coupling an audio sensor (e.g., microphone) to a rubberplastic-based medium, may enable mechanically coupling to mechanical waves which are perceived by the audio sensor. Robotic membrane 71 has material properties that may allow the conversion of a physical signal into some electromagnetic signal, such as a physical signal into a pressure signal. In addition, robotic membrane 71 (also referred herein as a protective membrane) may provide a protective surface for sensors (e.g., sensing patch 60 or sensor taxels 61) that are sensitive, such that corrosive or abrasive materials or liquids may not damage the sensors. Whether robotic membrane 71 is soft or hard may be based on the end application of robotic hand 10. In some examples, the end application may include, but is not limited to, the robotic hand 10 interacting with an environment and / or performing specific tasks.

[0071] Sensing spacer 72 may be a mechanism for the robotic membrane 71 to attach to (e.g., attach to during overmolding process) or may protect electronics of sensing patch 60. Sensing spacer 72 may be based on the dimensions of sensing patches 60. Sensing spacer 72 may be a rigid or non-rigid material to support later application of the robotic membrane 71 to the sensing patches 60. Sensing spacer 72 may allow for a post-processing step to be applied to the sensing medium without affecting the sensitive electronics which may be damaged by post-processing steps. For example, sensing spacer 72 may prevent damage to electronics during a chemical process (e.g., chemical bath or electrochemical material deposition). The printed circuit board 73 may include electronics associated with sensing patch 60 that may allow flexible and / or non- rigid installation on curved surfaces. FIG. 2C illustrates an example implementation of sensing patch 60. The disclosed sensing patch 60 may be used in a sensing patch system 70 that includes a multi-array design (e.g., as shown in FIG. 2A), local co-processing unit (e.g., local robotic coprocessor 102 of FIG. 3A), and the ability to add different sensing modalities (e.g., sensing patch 60 or sensors 101).

[0072] FIG. 3 A illustrates an example Al robotics platform 100. As shown in FIG. 3 A, there may be a sensor 101 (e.g., sensing patch 60 or sensing taxel 61) and local robotic coprocessor 102 in area 51 which is associated with finger 50. There may be a plurality of local robotic coprocessors 102 connected to one or more respective sensors 101. The plurality of local robotic coprocessors 102 may be connected with each other serially and connected with communication bus 103. It is contemplated that the local robotic coprocessors 102 may have parallel connections with communication bus 103. Sensors 101 may be connected with a particular local robotic processor 102 and sensors 101 may be grouped together in one or more ways, such as by geographic location (e.g., sensors of area 51, sensors of area 52, sensors of finger 50, etc.), by type of sensors (e.g., pressure sensor grouping, or audio pressure sensor grouping), or the like. Sensors 101 may include any subset or collection of sensors, such as electromagnetic sensitive devices, audio, pressure, capacitive, or resistive based array, among other sensors.

[0073] Each group of sensors may be associated with a digit (e.g., a finger(s) (e.g., fingers 20, 30, 40, 50), a toe(s)) or other robotic part (e.g., palm 15). Each group of sensors may have one or more local processors 102 connected with communication bus 103.

[0074] Communication buses 103 (e.g., for data or control information) may connect the component blocks of Al robotics platform 100 together through a unified (e.g., standardized) communication bus. Communication buses 103 may include a communication bus that provides power or data / control communication, and such communication buses may include as universal serial bus (USB), inter-integrated circuit (I2C), universal asynchronous receiver / transmitter (UART), universal synchronous / asynchronous receiver / transmitter (US ART), controller area network (CAN), or general-purpose input / output (GPIO), among other communication components. Communication buses 103, as shown, may be connected with local robotic coprocessor 102 or appendage processing component 105. It is contemplated herein that other communication buses 103 may be implemented.

[0075] Local robotic coprocessor 102 may configure sensors 101 based on communication from appendage processing component 105. Appendage processing component 105 may be located locally on robotic hand 10, such as in the area of palm 15. Local robotic coprocessor 102 may receive or send information to sensor 101 or appendage processing component 105. Local robotic coprocessor 102 may obtain or collect data and sample data. Local robotic coprocessor 102 may sample the data in order to analyze a subset of the data to uncover the meaningful information in a larger data set. For purposes of illustration and not of limitation, for example, if sensing patch 60 has 100 hundred sensors, processing the sensor information may take significant time or computing resources. Therefore, sampling data may allow for processing the sensor information using less time and computing resources. A subset of data (e.g., 20%, 25%, etc.) may be transmitted or processed (e.g., by local robotic coprocessor 102) based on the implementation (e.g., holding an object (e.g., object 7) in place versus rotating an object on fingertips).

[0076] Local robotic coprocessor 102 may deserialize and serialize the data for communication bus 103. When data is serialized as disclosed, the components may understand (or be conversant with) the same language and allow for changes to be implemented, otherwise an entire new system may need to be built for a different use case. Local robotic coprocessor 102 may take a first language the sensor block is using and convert that language to an appropriate second language the communication bus speaks. In an example, if communication bus 103 is defined as understanding USB, then local robotic coprocessor 102 may understand USB as well. Local robotic coprocessor 102 may translate from sensor 101 to data using that same language that is defined with the communication bus 103.

[0077] Appendage processing component 105 may include expandable input / output (VO) 108, control and data processor 107, and / or Al processing component 106. One or more blocks of appendage processing component 105 may be located locally on robotic hand 10 or remotely. Expandable I / O 108 may be used to connect, control, or configure auxiliary devices, such as an actuator, a manipulator, robotic hand, robotic foot, robotic arm, or other robotic peripherals. Connection or controls may be based on the data collected through expandable I / O 108 and other information, such as determinations by control and data processor 107 or determinations by Al processing component 106. Expandable VO 108 may sample data and / or obtain / collect data.

[0078] Control and data processor 107 may connect with communication bus 103 and may configure a plurality (e.g., some or all) of the local robotic coprocessors 102 of robotic hand 10. Control and data processor 107 may deserialize or serialize data from local coprocessors 102 or may multiplex data streams to expandable I / O 108 or Al processing component 106. In addition, control and data processor 107 may transmit instructions (e.g., commands) based on determinations of Al processing component 106 or expandable I / O 108.

[0079] Al processing component 106 may have one or more ML models (e.g., machine learning model(s) 410 of FIG. 4) provisioned for operating a robotic hand 10 or other robotic body part (e.g., robotic foot, robotic arm, or other robotic peripherals). Al processing component 106 may obtain data (e.g., training data 420 of FIG. 4) for training or other analysis from data received on communication bus 103 (e.g., sensor data), received from expandable I / O 108 (e.g., peripheral data), or received from control and data processor 107. The obtained data for training may be implemented models, such as force regression (e.g., normal and shear force), slip detection, or contact patch estimation (e.g., what does the contact area look like or how the contact area is represented). Al model (e.g., machine learning model(s) 410) outputs from Al processing component 106 may be used by control and data processor 107 to provide commands and actions to I / O devices associated with expandable I / O 108.

[0080] Data aggregation block 109 may deserialize or serialize data from or to Al robotics platform 100 or transmit bidirectional data from robotic hand 10 to a device external to robotic hand 10. The device external to robotic hand 10 may provide instructions (e.g., commands) for actions or configurations to sensors, actuators, manipulators, or the like for robotic hand 10.

[0081] With continued reference to FIG. 3 A, the disclosed Al robotics platform 100 allows for streamlined implementation and evolution of the usage of a robotic body part, such as for example robotic hand 10, feet, legs, or arms (e.g., robotic feet, robotic legs, or robotic arms). There may be any number of n sensors tiled across robotic hand 10 associated with Al robotics platform 100. Sensor 101 may digitize data being obtained / collected while robotic hand 10 interacts with an object (e.g., object 7). The object may be any suitable tangible object (e.g., a ball, a can, a box, a package, an appliance, a hardware tool, any other suitable object(s)). The data may be serialized and sent across the communication bus 103 to control and data processor 107. This data may be sent in parallel to a device that is external to robotic hand 10, and also to Al processing component 106. In an example, based on the end application, for dexterous manipulation, a custom developed Al model may be uploaded to Al processing component 106. Al processing component 106 may be local to robotic hand 10 (e.g., on-device), and Al processing component 106 may make decisions based on the data being obtained / collected by sensors 101. In an example, if the Al model (e.g., machine learning model(s) 410) may be configured to detect the total amount of forces (or the total number of forces) being applied by each link or phalange of the hand, and to augment these forces based on the criteria that may include preventing an object (e.g., object 7) from dropping. Al processing component 106 may directly transmit instructions for these actions or provide augmentation instructions through expandable I / O 108 to directly control an actuator or manipulator. The ability of Al processing component 106 to control robotic hand 10, and / or a robotic foot, may occur with lower latency than that of commands coming from a host device or connection that is external to robotic hand 10, and / or a robotic foot. FIG. 3B is another illustration that depicts the Al platform (e.g., Al platform 300) which may be associated with a robotic body part 310 such as, for example, a robotic hand (e.g., robotic hand 10) or another robotic body part(s). In some examples, the robotic body part 310 may be a robotic foot, a robotic hand, or other robotic body part (e g., a robotic arm, etc.).

[0082] Various aspects, examples, and techniques discussed herein may utilize one or more machine learning models (e.g., machine learning model 410 of FIG. 4), such as neural networks, deep learning models, and other techniques such as object recognition, or the like, to assist in one or more of determining objects (e.g., one or more objects 7) that are being interacted with (e.g., held by robotic hand 10), determining force to be applied to hold or manipulate an object (e.g., object 7), or the like. Such machine learning models (e g., machine learning model 410) may be trained on data sets associated with manipulation of objects by various forces applied by fingers, palms, feet, or the like.

[0083] FIG. 4 illustrates a framework 400 employed by a software application (e.g., computer code, a computer program) to generate robotic part actions, in accordance with aspects discussed herein. The framework 400 may be hosted remotely. Alternatively, the framework 400 may reside within a robotic body part, such as robotic hand 10 shown in FIG. 1 and / or may be processed by the computing system 600 shown in FIG. 6. Machine learning model(s) 410 may be operably coupled with the stored training data 420 in a database (e.g., training database 430). Machine learning (ML) and Al are generally used interchangeably herein. In some examples, the machine learning model(s) 410 may be associated with operations (or performing operations) of FIG. 5 and / or FIG. 7. In some other examples, the machine learning model(s) 410 may be associated with other operations.

[0084] In an example, the training data 420 may include attributes of thousands of objects. For example, the object(s) may be identified or associated with user profiles, posts, photographs / images, videos, augmented reality data, sensor data (e.g., capacitive based sensors, magnetic based sensors, resistive based sensors, pressure based sensors, or audio based sensors), teleoperation of a robotic body part(s), telerobotics, movement / motion of a robotic body part(s), and / or the like. Attributes may include but are not limited to the resistance, capacitance, audio, pressures, size, shape, orientation, position of an object. The training data 420 employed by machine learning model 410 may be fixed or updated periodically. Alternatively, training data 420 may be updated in real-time or near real-time based upon the evaluations performed by machine learning model(s) 410 in a non-training mode.

[0085] In operation, the machine learning model(s) 410 may evaluate attributes of images, audio, videos, capacitance, resistance, or other information obtained by hardware (e.g., sensors 101, peripherals, etc.), robotic movements of robotic body parts (e.g., robotic hands, robotic feet, robotic arms, robotic legs, etc.). For example, aspects of a user profile, posts, images, resistance, capacitance, audio, pressures, size, shape, orientation, position of an object(s), movements of one or more robotic body parts and / or the like may be ingested and analyzed. The attributes of any of the above (e.g., captured image of an object(s), captured capacitance(s), characteristic(s), movements of one or more robotic body parts, etc.) may then be compared with respective attributes of stored training data 420 (e.g., prestored objects, prestored data). The likelihood of similarity between each of the obtained attributes (e.g., of a captured audio or resistance) and the stored training data 420 (e.g., prestored objects, prestored data) may be given a determined confidence score. In one example, in an instance in which the confidence score exceeds a predetermined threshold, the attribute(s) is included in an instruction that is ultimately communicated to the robotic body part (e.g., robotic hand 10, robotic body part 310, control and data processor 107, a peripheral, etc.) or user via a user interface of a computing device (e.g., computing system 600). In another example, the description may include a certain number of attributes which exceed a predetermined threshold to share with the robotic body part or a user interface. The sensitivity of sharing more or less attributes may be customized based upon the needs of the particular device.

[0086] FIG. 5 illustrates an example method 520 associated with a robotics platform. At block 521, interactions of a robotic appendage (e.g., a robotic hand 10) with an object (e.g., object 7) may be digitized based on one or more sensors (e.g., sensing patch 60). The digitized interaction may be referred herein as sensor data. In an example, sensor 101 may receive pressure data (e.g., sensor data) and send the pressure data to local robotic coprocessor 102. Local robotic coprocessor 102 may serialize such sensor data for transmission to appendage processing component 105. The sensor data may serially be transmitted through a plurality of local robotic coprocessors 102 or other devices before ultimately being transmitted to appendage processing component 105. The sensor data may be transmitted to a machine learning component (e.g., Al processing component 106) that uses a machine learning model (e.g., machine learning model 410). The Al processing component 106 may be logically or physically within appendage processing component 105. The machine learning model (e.g., machine learning model(s) 410) may be specifically trained for robotic functions associated with movement of one or more appendages.

[0087] At block 522, based on the sensor data, information associated with the object and robotic hand 10 may be determined. The information associated with the object and robotic hand 10 may include a total amount of forces or a total number of forces being applied to the object by one or more phalanges or other parts (e.g., fingertips) of robotic hand (e.g., robotic hand 10, robotic body part 310).

[0088] At block 523, Al processing component 106 may determine one or more positions of the one or more phalanges or other parts of the robotic hand to augment the total amount of forces or the total number of forces being applied to the object. The determination may be based on a criterion and sensor data. Example criteria may include prevention of the object from dropping or rotation of the object on one or more fingertips of the robotic hand (e.g., robotic hand 10, robotic body part 310). At block 524, instructions may be sent to control the one or more phalanges or other parts / devices of the robotic hand to augment the total amount of forces, or the total number of forces being applied to the object. In an example, the instructions may be sent to local robotic coprocessor 102 to activate or deactivate certain sensors for optimal sampling of data. In another example, local robotic coprocessor 102 may receive instructions to move joints or actuators to control the position of the one or more phalanges of robotic hand 10. It is contemplated that method 520 or other methods may apply to other robotic appendages, such as arms or feet. The steps disclosed herein may be executed on one device or distributed over multiple devices.

[0089] The Al robotics platform may significantly reduce the number of connections and cables implemented compared to conventional implementation, such that points in the system may be controlled by the system over a single wire. The systems may be controlled with software and may use bi-directional communication from the Al robotics platform to a local or remote computing system. The Al robotics platform may be useful in advancing automatic, autonomous, and / or teleoperation robotics because it may allow the adoption of Al based models to be used on the same standardized platform (e.g., Al robotics platform). A standardized platform may provide for training across different implementations (e.g., uses) of a robotic system. The Al robotics platform may include on-device Al capabilities to direct processing of the robotic signals coming from the hand (e.g., palm, fingers, fingertips, etc.) sensors to make faster decisions for tasks involved with interacting with the environment.

[0090] Use cases for the disclosed Al robotics platform or robotic sensing patch system may include robotic hand, robotic actuator, robotic manipulator, robotic feet, robotic arms, robotic legs, or other robotic body parts (which may or may not mimic a body part of a human). An example of robotic sensing patch may be “skin” (e.g., robotic skin). In a robotic foot example, sensors 101 may be based on a printed circuit board, due to this, sensors 101 (e.g., sensor array) may adhere to a set of curvatures and may allow for the application to robotic feet. In an arm (e.g., robotic arm) example, sensors 101 may be based on a printed circuit board, due to this, sensors 101 (e.g., sensor array) may be able to tile across a large area and may adhere to a set of curvatures, which may allow for the application to a robotic arm. The sensor block array, flexible substrate, scalable communication bus 103, the ability to tile sensors (e.g., sensing patch 60), or the ability for sensors 101 to be distributed across the body (e.g., a robotic body) may allow for other robotic body parts use cases to be applicable. The disclosed subject matter may provide methods, systems, or apparatuses for a rich sensing appendage (e.g., multi-finger robotic hand, arm, foot, or leg) which captures palm, finger, fingertip, toe, or the like multi-modal information that reaches beyond visual perception. The interaction with surfaces may be physically modeled to mimic that of human skin (e.g., soft and flexible), for example, rather than hard low-friction surfaces often found on robotic endeffectors. In an example, rich sensing general task specific actions may be accomplished through a system that enables fingertip-based object rotation along multiple axes by leveraging multimodal sensory inputs. Moving from fingertip actions to full hand actions, which enables general manipulation of objects (e.g., one or more objects 7), may engage the use of full hand gestures, including stable grasps provided by the palm. The Al robotics platform (e.g., Al robotics platform 100, Al platform 300) may provide these rich inputs through a modular platform which provides multi-modal signals including vision and touch. In an example, the high fidelity signals may be limited to the fingertip (or similar parts) for precision manipulation while including non-vision data through the artificial membrane (or skin) in order to provide higher frequency data than an image system, and in order to reduce the computational complexity of solving occlusion and localization problems encountered with fingertip touch and visions systems.

[0091] The disclosed subject matter provides a robotic appendage that is capable of performing a wide range of tasks, including grasping and manipulation of objects, assembly, stability of movement of a robotic body, and material handling. The design and control capabilities of the robotic appendage may enable precise and flexible movement, making it an invaluable tool in various fields, including manufacturing, healthcare, or space exploration. The robotic hands or other components may offer enhanced dexterity, flexibility, or sensory capabilities when compared to conventional robotic components.

[0092] Methods, systems, computer readable storage medium, or apparatus, among other things as described herein may provide for an Al robotic platform or robotic sensing patch system. In an example, an apparatus may include a first sensing patch, wherein the first sensing patch comprises a plurality of first sensing taxels; a second sensing patch, wherein the second sensing patch comprises a plurality of second sensing taxels; a first local robotic processor communicatively connected with the first sensing patch; a second local robotic processor communicatively connected with the second sensing patch, wherein the first local robotic processor and the second local robotic processor are connected in series; a communication bus connected with the second local robotic processor; and an appendage processing component communicatively connected with the communication bus, the appendage processing component configured to receive or process sensor data from one or more areas of a robotic appendage. The robotic appendage may include a robotic hand or a robotic foot. The appendage processing component may include an Al processing component. The Al processing component may be configured to control a plurality of devices of a first digit and a second digit of the robotic appendage (e.g., first finger and second finger). The first local robotic processor and the second local robotic processor may be on the first digit. The second digit may have a third local robotic processor which may be connected in parallel to the second local robotic processor via the communication bus. The appendage processing component may include an expandable input / output component. The expandable input / output component may be configured to control a peripheral of a robot. All combinations (including the removal or addition of steps) in this paragraph and the below paragraphs may be contemplated in a manner that is consistent with the other portions of the detailed description.

[0093] A system, such as a robotic sensing patch system, may include a printed circuit board; a sensing patch coupled with the printed circuit board, wherein the sensing patch comprises an array of sensing taxels, wherein the printed circuit board may be configured to communicate sensor data from the sensing patch to a processor; and a membrane configured to convert a physical signal to an electromagnetic signal. The system may further include a sensing spacer coupled with the sensing patch and the membrane. The array of sensing taxels may include a capacitive based sensor, a magnetic based sensor, a pressure based sensor, an audio based sensor, and / or a resistive based sensor. A method, system, or apparatus may include digitize data being collected while robotic hand (or other appendage) interacts with an object; serialize the digitized data; transmit the serialized data across the communication bus to an appendage centralized processor; detect, by an Al processing component, the total amount of forces (or the total number of forces) being applied by each link or phalange of the robotic hand (or other appendage); determine augmentation of these forces based on the criteria that includes preventing an object from dropping or the like; and sending instructions based on the determined augmentation, wherein the instructions may cause movement of one or more digits of the robotic hand (or other appendage). All combinations (including the removal or addition of steps) in this paragraph and the above paragraphs may be contemplated in a manner that is consistent with the other portions of the detailed description.

[0094] The system, methods, or apparatus, among other things provide for digitizing data determined or obtained / collected by a sensor (e.g., sensing patch) of a robotic hand that interacts with an object; serializing the digitized data; transmitting the serialized data across a communication bus to an appendage processing component communicatively connected with the communication bus, wherein the data comprises a total amount of forces or a total number of forces being applied to the object by one or more phalanges of the robotic hand; determining, based on criteria and the serialized data, one or more positions of the one or more phalanges of the robotic hand to augment the total amount of forces or the total number of forces being applied to the object; and transmitting instructions to control the one or more phalanges of the robotic hand to augment the total amount of forces or the total number of forces being applied to the object. The determining of one or more positions of the one or more phalanges of the robotic hand to augment the total amount of forces or the total number of forces being applied to the object may be executed by a machine learning model. The data from the sensing patch may be used to train the ML model. The criteria may include prevention of the object from dropping or rotation of the object on one or more fingertips of the robotic hand. All combinations (including the removal or addition of steps) in this paragraph and the above paragraphs may be contemplated in a manner that is consistent with the other portions of the detailed description.

[0095] FIG. 6 illustrates an example computer system 600. In examples, one or more computer systems 600 perform one or more steps of one or more methods described or illustrated herein. In particular embodiments, one or more computer systems 600 provide functionality described or illustrated herein. In examples, software running on one or more computer systems 600 performs one or more steps of one or more methods described or illustrated herein or provides functionality described or illustrated herein. Examples include one or more portions of one or more computer systems 600. Herein, reference to a computer system may encompass a computing device, and vice versa, where appropriate. Moreover, reference to a computer system may encompass one or more computer systems, where appropriate.

[0096] This disclosure contemplates any suitable number of computer systems 600. This disclosure contemplates computer system 600 taking any suitable physical form. As example and not by way of limitation, computer system 600 may be an embedded computer system, a system- on-chip (SOC), a single-board computer system (SBC) (such as, for example, a computer-on- module (COM) or system-on-module (SOM)), a desktop computer system, a laptop or notebook computer system, an interactive kiosk, a mainframe, a mesh of computer systems, a mobile telephone, a personal digital assistant (PDA), a server, a tablet computer system, or a combination of two or more of these. Where appropriate, computer system 600 may include one or more computer systems 600; be unitary or distributed; span multiple locations; span multiple machines; span multiple data centers; or reside in a cloud, which may include one or more cloud components in one or more networks. Where appropriate, one or more computer systems 600 may perform without substantial spatial or temporal limitation one or more steps of one or more methods described or illustrated herein. As an example, and not by way of limitation, one or more computer systems 600 may perform in real time or in batch mode one or more steps of one or more methods described or illustrated herein. One or more computer systems 600 may perform at different times or at different locations one or more steps of one or more methods described or illustrated herein, where appropriate.

[0097] In examples, computer system 600 includes a processor 602, memory 604, storage 606, an input / output (I / O) interface 608, a communication interface 610, and a bus 612 (e.g., communication bus 103). Although this disclosure describes and illustrates a particular computer system having a particular number of particular components in a particular arrangement, this disclosure contemplates any suitable computer system having any suitable number of any suitable components in any suitable arrangement.

[0098] In examples, processor 602 includes hardware for executing instructions, such as those making up a computer program. As an example and not by way of limitation, to execute instructions, processor 602 may retrieve (or fetch) the instructions from an internal register, an internal cache, memory 604, or storage 606; decode and execute them; and then write one or more results to an internal register, an internal cache, memory 604, or storage 606. In particular embodiments, processor 602 may include one or more internal caches for data, instructions, or addresses. This disclosure contemplates processor 602 including any suitable number of any suitable internal caches, where appropriate. As an example, and not by way of limitation, processor 602 may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffers (TLBs). Instructions in the instruction caches may be copies of instructions in memory 604 or storage 606, and the instruction caches may speed up retrieval of those instructions by processor 602. Data in the data caches may be copies of data in memory 604 or storage 606 for instructions executing at processor 602 to operate on; the results of previous instructions executed at processor 602 for access by subsequent instructions executing at processor 602 or for writing to memory 604 or storage 606; or other suitable data. The data caches may speed up read or write operations by processor 602. The TLBs may speed up virtual- address translation for processor 602. In particular embodiments, processor 602 may include one or more internal registers for data, instructions, or addresses. This disclosure contemplates processor 602 including any suitable number of any suitable internal registers, where appropriate. Where appropriate, processor 602 may include one or more arithmetic logic units (ALUs); be a multi-core processor; or include one or more processors 602. Although this disclosure describes and illustrates a particular processor, this disclosure contemplates any suitable processor.

[0099] In examples, memory 604 includes main memory for storing instructions for processor 602 to execute or data for processor 602 to operate on. As an example, and not by way of limitation, computer system 600 may load instructions from storage 606 or another source (such as, for example, another computer system 600) to memory 604. Processor 602 may then load the instructions from memory 604 to an internal register or internal cache. To execute the instructions, processor 602 may retrieve the instructions from the internal register or internal cache and decode them. During or after execution of the instructions, processor 602 may write one or more results (which may be intermediate or final results) to the internal register or internal cache. Processor 602 may then write one or more of those results to memory 604. In particular embodiments, processor 602 executes only instructions in one or more internal registers or internal caches or in memory 604 (as opposed to storage 606 or elsewhere) and operates only on data in one or more internal registers or internal caches or in memory 604 (as opposed to storage 606 or elsewhere). One or more memory buses (which may each include an address bus and a data bus) may couple processor 602 to memory 604. Bus 612 may include one or more memory buses, as described below. In examples, one or more memory management units (MMUs) reside between processor 602 and memory 604 and facilitate accesses to memory 604 requested by processor 602. In particular embodiments, memory 604 includes random access memory (RAM). This RAM may be volatile memory, where appropriate. Where appropriate, this RAM may be dynamic RAM (DRAM) or static RAM (SRAM). Moreover, where appropriate, this RAM may be single-ported or multi-ported RAM. This disclosure contemplates any suitable RAM. Memory 604 may include one or more memories 604, where appropriate. Although this disclosure describes and illustrates particular memory, this disclosure contemplates any suitable memory.

[0100] In examples, storage 606 includes mass storage for data or instructions. As an example, and not by way of limitation, storage 606 may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disc, a magneto-optical disc, magnetic tape, or a Universal Serial Bus (USB) drive or a combination of two or more of these. Storage 606 may include removable or non-removable (or fixed) media, where appropriate. Storage 606 may be internal or external to computer system 600, where appropriate. In examples, storage 606 is non-volatile, solid-state memory. In particular embodiments, storage 606 includes read-only memory (ROM). Where appropriate, this ROM may be mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory or a combination of two or more of these. This disclosure contemplates mass storage 606 taking any suitable physical form. Storage 606 may include one or more storage control units facilitating communication between processor 602 and storage 606, where appropriate. Where appropriate, storage 606 may include one or more storages 606. Although this disclosure describes and illustrates particular storage, this disclosure contemplates any suitable storage.

[0101] In examples, EO interface 608 includes hardware, software, or both, providing one or more interfaces for communication between computer system 600 and one or more RO devices. Computer system 600 may include one or more of these I / O devices, where appropriate. One or more of these VO devices may enable communication between a person and computer system 600. As an example, and not by way of limitation, an I / O device may include a keyboard, keypad, microphone, monitor, mouse, printer, scanner, speaker, still camera, stylus, tablet, touch screen, trackball, video camera, another suitable VO device or a combination of two or more of these. An I / O device may include one or more sensors. This disclosure contemplates any suitable I / O devices and any suitable VO interfaces 608 for them. Where appropriate, VO interface 608 may include one or more device or software drivers enabling processor 602 to drive one or more of these I / O devices. I / O interface 608 may include one or more VO interfaces 608, where appropriate. Although this disclosure describes and illustrates a particular I / O interface, this disclosure contemplates any suitable I / O interface. In examples, communication interface 610 includes hardware, software, or both providing one or more interfaces for communication (such as, for example, packet-based communication) between computer system 600 and one or more other computer systems 600 or one or more networks. As an example, and not by way of limitation, communication interface 610 may include a network interface controller (NIC) or network adapter for communicating with an Ethernet or other wire-based network or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network, such as a WI-FI network. This disclosure contemplates any suitable network and any suitable communication interface 610 for it. As an example, and not by way of limitation, computer system 600 may communicate with an ad hoc network, a personal area network (PAN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), or one or more portions of the Internet or a combination of two or more of these. One or more portions of one or more of these networks may be wired or wireless. As an example, computer system 600 may communicate with a wireless PAN (WPAN) (such as, for example, a BLUETOOTH WPAN), a WI-FI network, a WI-MAX network, a cellular telephone network (such as, for example, a Global System for Mobile Communications (GSM) network), or other suitable wireless network or a combination of two or more of these. Computer system 600 may include any suitable communication interface 610 for any of these networks, where appropriate. Communication interface 610 may include one or more communication interfaces 610, where appropriate. Although this disclosure describes and illustrates a particular communication interface, this disclosure contemplates any suitable communication interface.

[0102] In particular embodiments, bus 612 includes hardware, software, or both coupling components of computer system 600 to each other. As an example and not by way of limitation, bus 612 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a front-side bus (FSB), a HYPERTRANSPORT (HT) interconnect, an Industry Standard Architecture (ISA) bus, an INFINIBAND interconnect, a low-pin-count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCIe) bus, a serial advanced technology attachment (SATA) bus, a Video Electronics Standards Association local (VLB) bus, or another suitable bus or a combination of two or more of these. Bus 612 may include one or more buses 612, where appropriate. Although this disclosure describes and illustrates a particular bus, this disclosure contemplates any suitable bus or interconnect.

[0103] FIG. 7 illustrates an example method associated with a robotics platform. At operation 700, a device (e.g., Al robotics platform 100) may digitize data determined by a robotic sensing patch system of a robotic body part (e.g., robotic hand 10) that interacts with an object (e.g., object 7). The robotic sensing patch system (e.g., robotic sensing patch system 70) may include a sensing patch, and a protective membrane. The robotic sensing patch system may also include a rigid or rigid-flexible printed circuit board. At operation 702, a device (e.g., Al robotics platform 100) may determine, based on the digitized data, first information that includes a total amount of forces or a total number of forces being applied to the object by one or more phalanges of the robotic body part (e.g., robotic hand 10).

[0104] At operation 704, a device (e.g., Al robotics platform 100) may determine, based on criteria and the first information, one or more positions of the one or more phalanges of the robotic body part to augment the total amount of forces or the total number of forces being applied to the object. At operation 706, a device (e.g., Al robotics platform 100) may transmit instructions to control the one or more phalanges of the robotic body part to further augment the total amount of forces or the total number of forces being applied to the object.

[0105] Herein, a computer-readable non-transitory storage medium or media may include one or more semiconductor-based or other integrated circuits (ICs) (such, as for example, field- programmable gate arrays (FPGAs) or application-specific ICs (ASICs)), hard disk drives (HDDs), hybrid hard drives (HHDs), optical discs, optical disc drives (ODDs), magneto-optical discs, magneto-optical drives, floppy diskettes, floppy disk drives (FDDs), magnetic tapes, solid- state drives (SSDs), RAM-drives, SECURE DIGITAL cards or drives, any other suitable computer-readable non-transitory storage media, computer readable medium or any suitable combination of two or more of these, where appropriate. A computer-readable non-transitory storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile, where appropriate.

[0106] Herein, “or” is inclusive and not exclusive, unless expressly indicated otherwise or indicated otherwise by context. Therefore, herein, “A or B” means “A, B, or both,” unless expressly indicated otherwise or indicated otherwise by context. Moreover, “and” is both joint and several, unless expressly indicated otherwise or indicated otherwise by context. Therefore, herein, “A and B” means “A and B, jointly or severally,” unless expressly indicated otherwise or indicated otherwise by context.

[0107] While the disclosed systems have been described in connection with the various examples of the various figures, it is to be understood that other similar implementations may be used or modifications and additions may be made to the described examples of a robotic skin or Al robotics platform, among other things as disclosed herein. For example, one skilled in the art will recognize that robotic skin or an Al robotics platform, among other things as disclosed herein in the instant application may apply to any environment, whether wired or wireless, and may be applied to any number of such devices connected via a communications network and interacting across the network. Therefore, the disclosed systems as described herein should not be limited to any single example, but rather should be construed in breadth and scope in accordance with the appended claims.

[0108] In describing preferred methods, systems, or apparatuses of the subject matter of the present disclosure - robotic skin or an Al robotics platform - as illustrated in the Figures, specific terminology is employed for the sake of clarity. The claimed subject matter, however, is not intended to be limited to the specific terminology so selected.

[0109] Also, as used in the specification including the appended claims, the singular forms “a,” “an,” and “the” include the plural, and reference to a particular numerical value includes at least that particular value, unless the context clearly dictates otherwise. The term “plurality”, as used herein, means more than one. When a range of values is expressed, another embodiment includes from the one particular value or to the other particular value. Similarly, when values are expressed as approximations, by use of the antecedent “about,” it will be understood that the particular value forms another embodiment. All ranges are inclusive and combinable. It is to be understood that the terminology used herein is for the purpose of describing particular aspects only and is not intended to be limiting.

[0110] This written description uses examples to enable any person skilled in the art to practice the claimed subject matter, including making and using any devices or systems and performing any incorporated methods. Other variations of the examples are contemplated herein. It is to be appreciated that certain features of the disclosed subject matter which are, for clarity, described herein in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the disclosed subject matter that are, for brevity, described in the context of a single embodiment, may also be provided separately or in any sub- combination. Further, any reference to values stated in ranges includes each and every value within that range.

[0111] The scope of this disclosure encompasses all changes, substitutions, variations, alterations, and modifications to the example embodiments described or illustrated herein that a person having ordinary skill in the art would comprehend. The scope of this disclosure is not limited to the examples described or illustrated herein. Moreover, although this disclosure describes and illustrates respective embodiments herein as including particular components, elements, feature, functions, operations, or steps, any of these embodiments may include any combination or permutation of any of the components, elements, features, functions, operations, or steps described or illustrated anywhere herein that a person having ordinary skill in the art would comprehend. Furthermore, reference in the appended claims to an apparatus or system or a component of an apparatus or system being adapted to, arranged to, capable of, configured to, enabled to, operable to, or operative to perform a particular function encompasses that apparatus, system, component, whether or not it or that particular function is activated, turned on, or unlocked, as long as that apparatus, system, or component is so adapted, arranged, capable, configured, enabled, operable, or operative. Additionally, although this disclosure describes or illustrates particular embodiments as providing particular advantages, particular embodiments may provide none, some, or all of these advantages.

Claims

CLAIMS1. A robotic sensing patch system comprising: at least one processor; a sensing patch comprising an array of sensing taxels, wherein the sensing patch is configured to communicate sensor data, determined in response to interactions of a robotic body part with an object, to the at least one processor; and a membrane configured to convert a physical signal to an electromagnetic signal.

2. The robotic sensing patch system of claim 1, further comprising: a sensing spacer coupled with the sensing patch and the membrane; wherein: the sensing spacer protects at least one sensing taxel; and the membrane comprises an artificial membrane mimicking human skin.

3. The robotic sensing patch system of claim 1 or claim 2, wherein the array of sensing taxels includes at least one of a capacitive based sensor, a magnetic based sensor, a resistive based sensor, or other electromagnetic based sensor.

4. The robotic sensing patch system of any preceding claim, wherein the array of sensing taxels includes at least one of a pressure based sensor or an audio based sensor.

5. An apparatus comprising: a first sensing patch comprising a plurality of first sensing taxels; a second sensing patch comprising a plurality of second sensing taxels; a first robotic processor communicatively connected with the first sensing patch; a second robotic processor communicatively connected with the second sensing patch; a communication bus connected with the second local robotic processor; and an appendage processing component configured to receive or process sensor data from one or more areas of a robotic appendage.

6. The apparatus of claim 5, wherein: the robotic appendage comprises a robotic hand, or the robotic appendage comprises a robotic foot.

7. The apparatus of claim 5 or claim 6, wherein: the appendage processing component comprises an artificial intelligence (Al) processing component configured to control at least a first robotic part and a secondrobotic part of the robotic appendage; and the first robotic processor and the second robotic processor are embedded within, or associated with the first robotic part.

8. The apparatus of any of claims 5 to 7, wherein: the appendage processing component comprises an expandable input / output component; and the expandable input / output component is configured to control a peripheral of a robot.

9. The apparatus of any of claims 5 to 8, further comprising: a robotic sensing patch system comprising a robotic membrane and the first sensing patch, and optionally wherein the robotic sensing patch system further comprises a sensing spacer coupled with the first sensing patch, and further optionally wherein: the sensing spacer is coupled with the first sensing patch; and the sensing spacer protects at least one sensing taxel of the first sensing patch.

10. A method comprising: digitizing data determined by a robotic sensing patch system of a robotic body part that interacts with an object, wherein the robotic sensing patch system comprises a sensing patch and a protective membrane; determining, based on the digitized data, first information that comprises a total amount of forces or a total number of forces being applied to the object by one or more phalanges of the robotic body part; determining, based on criteria and the first information, one or more positions of the one or more phalanges of the robotic body part to augment the total amount of forces or the total number of forces being applied to the object; and transmitting instructions to control the one or more phalanges of the robotic body part to further augment the total amount of forces or the total number of forces being applied to the object.

11. The method of claim 10, wherein the determining the one or more positions of the one ormore phalanges of the robotic body part to augment the total amount of forces or the total number of forces being applied to the object is executed by a machine learning model, and optionally wherein the machine learning model utilizes data from the sensing patch to facilitate training of the machine learning model.

12. The method of claim 10 or claim 11, wherein: the criteria comprises prevention of the object from dropping, and / or the criteria comprises rotation of the object on one or more fingertips of the robotic body part.

13. The method of any of claims 10 to 12, wherein the robotic body part comprises a robotic hand or a robotic foot.

14. The method of any of claims 10 to 13, further comprising determining the total amount of forces or the total number of forces being applied to the object, wherein the one or more phalanges manipulate the object based on the total amount of forces or the total number of forces .

15. The method of any of claims 10 to 14, wherein the data from the sensing patch is associated with movement of the robotic body part.

Images