System for measuring position using a robot with deformable sensors.

Abstract

To provide systems and methods for determining a location of a robot.SOLUTION: A method includes receiving, by a processor, a signal from a deformable sensor including data with respect to a deformation region in a deformable membrane of the deformable sensor resulting from contact with a first object. The data associated with contact with the first object is compared, by the processor, to details associated with contact with the first object in information associated with a plurality of objects stored in a database. The first object is identified, by the processor, as a first identified object of the plurality of objects stored in the database. The first identified object is an object of the plurality of objects stored in the database that is most similar to the first object. The location of the robot is measured, by the processor, based on a location of the first identified object.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] This application claims priority to co-pending U.S. Provisional Patent Application No. 62 / 977,466, filed February 17, 2020, for “System for Measuring Position Using a Robot with a Bubble Sensor,” and co-pending U.S. Provisional Patent Application No. 62 / 984,083, filed March 2, 2020, for “Bubble Sensor Gripper for Force Manipulation and Manipuland State Estimation,” each of which is incorporated by reference in its entirety, including any drawings.

[0002] The embodiments described herein relate generally to systems and methods for determining the position of a robot in space, and in particular to a robot having a deformable contact and shape / pose sensor capable of detecting the contact and shape of an object to determine the position of the robot. [Background technology]

[0003] The human sense of touch allows us to gauge the shape of objects without seeing them, which helps us identify objects and allows us to locate where we are in space based on our knowledge of the location of touched objects without using vision.

[0004] Robots generally include end effectors that are configured to perform a specific task. However, robots do not have the variable level of tactile sensation that humans have. End effectors can include sensors such as pressure sensors, but such sensors provide limited information about the object in contact with the end effector. Thus, the robot may damage the target object by using too much force or may drop the object due to not properly grasping it. Therefore, in some applications, a deformable / compliant end effector may be preferred.

[0005] However, robots cannot currently touch objects in this way, and as a result, they cannot identify objects. Therefore, to measure the position or location of the robot, the robot relies on other technologies, such as GPS sensors or visual sensors to identify the robot's location, however, these can be inaccurate in small-scale environments. Summary of the Invention

[0006] In one embodiment, a method for determining a position of a robot including a deformable sensor includes receiving, by a processor, a signal from the deformable sensor including data regarding a deformation area of ​​a deformable membrane of the deformable sensor resulting from contact with a first object. The data associated with the contact with the first object is compared by the processor to information associated with a plurality of objects stored in a database, details associated with the contact with the first object. The first object is identified by the processor as a first identified object (first identified object) of the plurality of objects stored in the database. The first identified object is the object of the plurality of objects stored in the database that is most similar to the first object. The position of the robot is determined by the processor based on the position of the first identified object.

[0007] In another embodiment, a robot for measuring a position in space includes a casing including an upper surface, an opposing lower surface, and an edge surface extending between the upper surface and the lower surface. At least one deformable sensor is provided on the casing. The at least one deformable sensor includes an internal sensor and a deformable membrane. The internal sensor is configured to output a deformation area in the deformable membrane resulting from contact with a first object. The robot includes one or more processors and one or more memory modules including a computer readable medium storing computer readable instructions, which when executed by the one or more processors cause the one or more processors to receive data from the internal sensor representative of the deformation area upon contact with the first object. The data associated with the contact of the first object is compared by the processor to details associated with a plurality of objects stored in a database. The first identified object is an object of the plurality of objects stored in the database that is most similar to the first object. The processor identifies the first object as the first identified object of the plurality of objects stored in the database. The position of the robot is determined by the processor based on the position of the first identified object.

[0008] In yet another embodiment, a system for measuring a position of a robot including a deformable sensor includes a robot including an upper surface, an opposing lower surface, and an edge surface extending between the upper surface and the lower surface. At least one deformable sensor is provided on the robot. The at least one deformable sensor includes a housing, a deformable membrane coupled to an upper portion of the housing, an enclosure configured to be filled with a medium, and an internal sensor disposed within the enclosure having a field of view configured to pass through the medium toward a bottom surface of the deformable membrane. The internal sensor is configured to output a deformation area in the deformable membrane resulting from contact with a first object. The system includes one or more processors and one or more memory modules including a computer readable medium storing computer readable instructions, which when executed by the one or more processors cause the one or more processors to receive data from the internal sensor representative of the deformation area upon contact with the first object. The data associated with the contact of the first object is compared by the processor to details associated with the plurality of objects stored in a database. The first identified object is an object of the plurality of objects stored in the database that is most similar to the first object. The processor identifies the first object as the first identified object of the plurality of objects stored in the database. The position of the robot is determined based on the position of the first identified object.

[0009] These and additional features provided by the embodiments described herein can be better understood by reading the following detailed description in conjunction with the accompanying drawings.

[0010] The embodiments illustrated in the drawings are exemplary and representative and are not intended to limit the subject matter defined by the claims. The following detailed description of exemplary embodiments can be understood when read in conjunction with the drawings, in which like structures are indicated with like reference numerals. [Brief description of the drawings]

[0011] [Figure 1]FIG. 1 is a schematic perspective view of an example robot including multiple deformable sensors according to one or more embodiments described and illustrated herein. [Diagram 2] FIG. 2 is a block diagram of hardware utilized by the robot of FIG. 1 to implement various processes and systems according to one or more embodiments described and illustrated herein. [Diagram 3] 1 is a schematic cross-sectional view of an example deformable sensor according to one or more embodiments described and illustrated herein. [Figure 4] FIG. 4 is a schematic top perspective view of the example deformable sensor of FIG. 3 according to one or more embodiments described and illustrated herein. [Diagram 5] 1 is a schematic cross-sectional view of an example deformable sensor according to one or more embodiments described and illustrated herein. [Figure 6] FIG. 6 is a schematic rear perspective view of the bubble module of the example deformable sensor of FIG. 5 according to one or more embodiments described and illustrated herein. [Figure 7] 7 is a schematic exploded view of the bubble sensor of FIG. 6 according to one or more embodiments described and illustrated herein. [Figure 8] FIG. 2 is a schematic diagram of a filter layer coupled to a deformable membrane of a deformable sensor according to one or more embodiments described and illustrated herein. [Figure 9] 1 is a schematic diagram of a filter within the field of view of an internal sensor of a deformable sensor according to one or more embodiments described and illustrated herein. [Figure 10] FIG. 2 is a schematic diagram of a dot pattern on the bottom surface of a deformable membrane of a deformable sensor according to one or more embodiments described and illustrated herein. [Figure 11] FIG. 2 is a schematic diagram of a grid pattern on the bottom surface of a deformable membrane of a deformable sensor according to one or more embodiments described and illustrated herein. [Figure 12] FIG. 1 is a schematic diagram of a composite internal sensor having multiple internal sensors according to one or more embodiments described and illustrated herein. [Figure 13] 1 is an image of the output of a deformable sensor on an electronic display according to one or more embodiments described and illustrated herein. [Figure 14] 1 is a flowchart of an example process for measuring pose and forces associated with an object in contact with a deformable sensor in accordance with one or more embodiments described and illustrated herein. [Figure 15] FIG. 1 is a schematic overhead view of a space in which a robot is utilized to perform a task in accordance with one or more embodiments described and illustrated herein. [Figure 16] 1 is a flowchart of an example process for determining the position of a robot in space in accordance with one or more embodiments described and illustrated herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Embodiments of the present disclosure relate to robots that include deformable sensors, and in particular to deformable / compliant contact and / or shape sensors (hereinafter "deformable sensors") that not only detect contact with a target object, but also detect the shape, pose and contact force of the target object to identify the location of the robot. In particular, the deformable sensors described herein comprise a deformable membrane coupled to a housing, which carries a sensor capable of detecting the displacement of the deformable membrane due to contact with an object. Thus, the deformable sensors described herein provide a robot (or other device) with a sense of touch when manipulating or contacting an object.

[0013] Autonomous robots can be used to perform a variety of tasks and efficiently navigate spaces such as buildings or cubicles. Such tasks can include retrieving an item, delivering an item, or, in the case of an autonomous vacuum cleaner, cleaning a room. Known autonomous vacuum cleaners survey a space, such as a room in a house, by traveling in a straight line until they encounter an object, such as a wall, step, or other obstacle, and the robot stores the location information of the contacted object in its memory as an object to be avoided in future cleaning tasks. As additional objects are contacted, the robot continually adds the location information of these objects to its memory in order to more accurately survey the room and avoid these objects in the future.

[0014] 1, the example robot 100 is illustrated as an autonomous vacuum cleaner, and the robotic task herein is a cleaning task. However, it should be appreciated that the robot 100 may be any other suitable robot other than an autonomous vacuum cleaner for performing other tasks without departing from the scope of this disclosure. For example, the robot 100 may be an object retrieval or delivery robot that receives instructions to navigate a space from a start point to a destination point.

[0015] The robot 100 includes a casing 102 generally defined by an upper surface 104, an opposing lower surface 106, and an edge surface 108 extending between the upper surface 104 and the lower surface 106. The casing 102 houses the internal components of the robot 100 described herein. The lower surface 106 of the casing 102 faces downward toward a floor surface F, and the upper surface 104 faces upward in the opposite direction. The embodiments are not limited to the casing 102 configuration of FIG. 1, and the various surfaces can take other forms. At least one wheel 110 is provided on the lower surface 106 of the casing 102 to enable the robot 100 to traverse the floor surface F. In some embodiments, the robot 100 includes a pair of wheels 110. In another embodiment, the robot 100 can include legs that include joints, skis, rails, or flying components for moving or transporting the robot 100.

[0016] Additionally, the robot 100 includes at least one deformable sensor 112 disposed on the edge surface 108 of the casing 102. In some embodiments, the robot 100 includes multiple deformable sensors 112 spaced apart from one another along the edge surface 108 of the casing 102. In embodiments in which multiple deformable sensors 112 are included, the multiple deformable sensors 112 can be spaced apart and arranged along the edge surface 108 of the casing 102 in any suitable manner. However, in some embodiments, the deformable sensor 112 can be disposed on the top surface 104 and / or the bottom surface 106 of the casing 102. When the deformable sensor 112 is disposed on the bottom surface 106 of the casing 102, the deformable sensor 112 is adapted to sense the type of floor the robot 100 is rolling over, a height threshold, or the like. In other embodiments, the robot 100 can include any number of arms including joints, with end effectors attached to the ends of the arms. In this embodiment, the deformable sensor 112 may be provided on the end effector and may be independently movable relative to the robot 100 .

[0017] The deformable sensor 112 provided on the robot 100, as in the embodiments discussed herein, may be any suitable deformable sensor capable of identifying characteristics of a contacted object, such as its shape, pose, hardness, flexibility, and the like. The following description of the robot 100 generally refers to the deformable sensor 112 in terms of its use in identifying objects and helping the robot 100 determine their location.

[0018] 2 is a schematic diagram of example components of one non-limiting embodiment of the robot 100. In some embodiments, the robot 100 includes a communication path 228, a processor 230, a memory module 232, an inertial measurement unit 236, input devices 238, a camera 244, network interface hardware 246, a position sensor 250, a light emitter 252, a proximity sensor 254, a motorized wheel assembly 258, a battery 260, and a charging port 262. The components of the robot 100 may be contained within or attached to the casing 102. The various components of the robot 100 and their interactions are described in more detail below.

[0019] 2, the communication path 228 is formed from any medium capable of transmitting a signal, such as wires, conductive traces, optical waveguides, or the like. Furthermore, the communication path 228 can be formed from a combination of media capable of transmitting a signal. In one embodiment, the communication path 228 comprises a combination of conductive traces, wires, connectors, and buses that cooperate to allow transmission of electrical data signals to components such as processors, memories, sensors, input devices, output devices, and communication devices. Thus, the communication path 228 can comprise a bus. Furthermore, "signal" refers to a waveform (e.g., electrical, optical, magnetic, mechanical, or electromagnetic) that can travel through a medium, such as DC, AC, sine wave, triangular wave, square wave, vibration, and the like. The communication path 228 communicatively couples various components of the robot 100. As used herein, "communicatively coupled" means that the coupled components can exchange data signals with each other, such as electrical signals through a conductive medium, electromagnetic signals through air, optical signals through optical waveguides, and the like.

[0020] The processor 230 of the robot 100 may be any device capable of executing computer-readable instructions. Thus, the processor 230 may be a controller, an integrated circuit, a microchip, a computer, or other computing device. The processor 230 may be communicatively coupled to other components of the robot 100 by a communication path 228. Thus, in various embodiments, the processor 230 may receive data from one or more deformable sensors 112. In other embodiments, the processor 230 may receive data directly from one or more internal sensors that are part of the one or more deformable sensors 112 on the robot 100. Thus, the communication path 228 may communicatively couple any number of processors to each other, allowing the components coupled to the communication path 228 to operate in a distributed processing environment. In particular, each of the components may operate as a node that may transmit and / or receive data. Although the embodiment shown in FIG. 2 includes only one processor 230, other embodiments may include multiple processors.

[0021] 2, the memory module 232 of the robot 100 is coupled to the communication path 228 and communicatively coupled to the processor 230. The memory module 232 may contain computer readable instructions for, for example, detecting a shape of an object that has deformed the deformable sensor 112. In this embodiment, the instructions stored in the memory module 232, when executed by the processor 230, allow for a measurement of the shape of the object based on the observed deformation of the deformable sensor 112. The memory module 232 may include RAM, ROM, flash memory, hard drive, or any other non-transitory memory device capable of storing computer readable instructions such that the computer readable instructions may be accessed and executed by the processor 230. The computer readable instructions may include, for example, logic or algorithms written in any generation programming language (e.g., 1GL, 2GL, 3GL, 4GL, or 5GL), such as a machine language that may be directly executed by the processor, or an assembly language, object oriented programming (OOP), scripting language, microcode, etc. that may be compiled or assembled into computer readable instructions and stored in the memory module 232. Alternatively, the computer readable instructions may be written in a hardware description language (HDL), such as logic implemented via a programmable gate array (FPGA) configuration or an application specific integrated circuit (ASIC) or equivalent. Thus, the functionality described herein may be implemented in any conventional computer programming language, as pre-programmed hardware elements or as a combination of hardware and software components. Although the embodiment shown in FIG. 2 includes only one memory module 232, other embodiments may include multiple memory modules.

[0022] The inertial measurement unit 236 (if included) is coupled to the communication path 228 and communicatively coupled to the processor 230. The inertial measurement unit 236 may include one or more accelerometers and one or more gyroscopes. The inertial measurement unit 236 converts sensed physical movements of the robot 100 into signals representative of the orientation, rotation, velocity, or acceleration of the robot 100. The tasks of the robot 100 may depend on the orientation of the robot 100 (e.g., whether the robot 100 is parallel, tilted, or the like). Some embodiments of the robot 100 may not include the inertial measurement unit 236, such as embodiments that include an accelerometer but not a gyroscope, embodiments that include a gyroscope but not an accelerometer, or embodiments that include neither an accelerometer nor a gyroscope.

[0023] One or more input devices 238 are coupled to the communication path 228 and communicatively coupled to the processor 230. The input device 238 may be any device capable of converting a user contact into a data signal that can be transmitted over the communication path 228, such as a button, switch, knob, microphone, or the like. In various embodiments, the input device 238 may be a deformable sensor 112 as described herein. In some embodiments, the input device 238 includes a power button, a volume button, a start button, a scroll button, or the like. The one or more input devices 238 are provided to allow a user to interact with the robot 100, such as to navigate menus, make selections, set preferences, and other functionality described herein. In some embodiments, the input device 238 includes a pressure sensor, a tactile area, a pressure strip, or the like. It should be appreciated that some embodiments may not include an input device 238. As described in more detail below, embodiments of the robot 100 may include multiple input devices disposed on any surface of the casing 102. In some embodiments, one or more of the input devices 238 are configured as a fingerprint sensor for unlocking the robot 100. For example, only users with a registered fingerprint can unlock and use the robot 100.

[0024] Camera 244 is coupled to communication path 228 and communicatively coupled to processor 230. Camera 244 may be any device having an array of sensing elements (e.g., pixels) capable of detecting radiation in ultraviolet, visible, or infrared wavelength bands. Camera 244 may have any resolution. Camera 244 may be an omnidirectional or panoramic camera. In some embodiments, one or more optical components, such as mirrors, fisheye lenses, or other types of lenses, may be optically coupled to camera 244. As described in more detail below, camera 244 is a component of imaging assembly 222 that may be actuated to lift casing 102 to capture image data.

[0025] The network interface hardware 246 is coupled to the communication path 228 and communicatively coupled to the processor 230. The network interface hardware 246 may be any device capable of transmitting and / or receiving data over the network 270. Thus, the network interface hardware 246 may include a wireless communication module configured as a communication transceiver for transmitting and / or receiving any wired or wireless communication. For example, the network interface hardware 246 may include an antenna, a modem, a LAN port, a WiFi card, a WiMax card, mobile communication hardware, short range communication hardware, satellite communication hardware, and / or any wired or wireless hardware for communicating with other networks and / or devices. In one embodiment, the network interface hardware 246 includes hardware configured to operate according to the Bluetooth® wireless communication protocol. In another embodiment, the network interface hardware 246 may include a Bluetooth® transmit / receive module for sending and receiving Bluetooth® communications to / from the portable electronic device 280. The network internet hardware 246 may also include a radio frequency identification (RFID) reader configured to interrogate or read RFID tags.

[0026] In some embodiments, the robot 100 can be communicatively coupled to the portable electronic device 280 via the network 270. In some embodiments, the network 270 is a personal area network that utilizes Bluetooth technology to communicatively couple the robot 100 and the portable electronic device 280. In other embodiments, the network 270 can include one or more computer networks (e.g., personal area networks, local area networks, or wide area networks), cellular networks, satellite networks, and / or global positioning systems, and combinations thereof. Thus, the robot 100 can be communicatively coupled to the network 270 via wires, wide area networks, local area networks, personal area networks, cellular networks, satellite networks, or the like. Suitable local area networks can include wired Ethernet and / or wireless technologies, such as, for example, Wireless Fidelity (WiFi). Suitable personal area networks can include wireless technologies, such as IrDA, Bluetooth, Wireless USB, Z-Wave, ZigBee, and / or other short-range communication protocols. Suitable personal area networks may also include wired computer buses such as, for example, USB and FireWire. Suitable cellular networks include, but are not limited to, technologies such as LTE, WiMAX, UMITS, CDMA, and GSM.

[0027] As described above, the robot 100 can be communicatively coupled to a portable electronic device 280 using a network 270. The portable electronic device 280 can include a mobile phone, a smart phone, a personal digital assistant, a camera, a dedicated mobile media player, a mobile personal computer, a laptop computer, and / or any other portable electronic device that can be communicatively coupled to the robot 100. The portable electronic device 280 can include one or more processors and one or more memories. The one or more processors can execute logic to communicate with the robot 100. The portable electronic device 280 can be configured with wired and / or wireless communication capabilities to communicate with the robot 100. In some embodiments, the portable electronic device 280 can implement one or more elements of the functionality described herein, such as in embodiments in which the functionality described herein is distributed between the robot 100 and the portable electronic device 280.

[0028] The position sensor 250 is coupled to the communication path 228 and communicatively coupled to the processor 230. The position sensor 250 may be any device capable of providing an output representative of a position. In some embodiments, the position sensor 250 may include a Global Positioning System (GPS) sensor, but embodiments are not limited thereto. Some embodiments may not include the position sensor 250, such as embodiments in which the robot 100 does not measure the position of the robot 100 or the position is measured in other ways (e.g., based on information received from the camera 244, the network interface hardware 246, the proximity sensor 254, the inertial measurement unit 236, or the like). The position sensor 250 may also be configured as a wireless signal sensor capable of triangulating the position of the robot 100 and the user by wireless signals received from one or more wireless signal antennas.

[0029] The motorized wheel assembly 258 is coupled to the communication path 228 and communicatively coupled to the processor 230. As described in more detail below, the motorized wheel assembly 258 includes at least one wheel 110 driven by one or more motors (not shown). The processor 230 can provide one or more drive signals to the motorized wheel assembly 258 to actuate the wheel 110 such that the robot 100 travels to a desired location, such as a location where a user wants to obtain environmental information (e.g., the location of a particular object in, at, or near a desired location).

[0030] Lights 252 (if present) are coupled to communication path 228 and communicatively coupled to processor 230. Lights 252 may be any device capable of outputting light, such as, but not limited to, light emitting diodes, incandescent lights, fluorescent lights, or the like. Some embodiments include a power indicator light that illuminates when robot 100 is powered on. Some embodiments include an activity indicator light that illuminates when robot arm 100 is active or processing data. Some embodiments include illumination lights to illuminate the environment in which robot 100 is located. Some embodiments may not include lights 252.

[0031] Proximity sensor 254 (if included) is coupled to communication path 228 and communicatively coupled to processor 230. Proximity sensor 254 may be any device capable of outputting a proximity signal indicative of the proximity of robot 100 to another object. In some embodiments, proximity sensor 254 may include a laser scanner, a capacitive displacement sensor, a Doppler effect sensor, an eddy current sensor, an ultrasonic sensor, a magnetic sensor, an internal sensor, a radar sensor, a LiDAR sensor, a sonar sensor, or the like. Some embodiments may not include proximity sensor 254, such as embodiments in which the proximity of robot 100 to an object is determined from input provided by other sensors (e.g., camera 244, etc.) or embodiments that do not measure the proximity of robot 100 to an object.

[0032] The robot 100 may be powered by a battery 260. The battery is electrically coupled to various electrical components of the robot 100. The battery 260 may be any device capable of storing electrical energy for later use by the robot 100. In some embodiments, the battery 260 is a rechargeable battery, such as a lithium ion battery or a nickel cadmium battery. In embodiments in which the battery 260 is a rechargeable battery, the robot 100 may include a charging port 262 that may be used to charge the battery 260. Some embodiments, such as those in which the robot 100 is powered by a power grid, solar energy, or energy obtained from the environment, may not include the battery 260. Some embodiments, such as those in which the device utilizes disposable batteries for power, may not include the charging port 262.

[0033] 3 and 4 are schematic diagrams of one embodiment of the deformable sensor 112 of the robot 100. FIG. 3 is a cross-sectional view of the deformable sensor 112, and FIG. 4 is a top perspective view of the deformable sensor 112. The deformable sensor 112 generally comprises a housing 310 and a deformable membrane 320 coupled to the housing 310, such as at a top portion 311 of the housing 310. In some embodiments, the housing 310 is 3D printed. The housing 310 and the deformable membrane 320 define an enclosure 313 into which a medium is filled via one or more fluid conduits 312. The fluid conduits 312 can be valves or other suitable mechanisms. The fluid conduits 312 can be utilized to fill or empty the enclosure 313. In one example, the medium is a gas, such as air. Thus, air can be pumped into the enclosure 313 to a desired pressure so that the deformable membrane 320 forms a dome shape as shown in FIGS. 3 and 4, although any suitable shape can be used in other embodiments. In another example, the medium is a gel, such as silicone or other rubber-like material. In some embodiments, a material, such as solid silicone, can be molded into a given shape prior to assembly of the deformable sensor 112. In various embodiments, the medium can be anything that is transparent to the internal sensor 330, discussed in more detail below, such as to the wavelength of the time-of-flight sensor. The medium can include a transparent rubber in some embodiments. In other embodiments, the medium can be a liquid. In some examples, the deformable membrane 320 and the medium in the enclosure 313 can be made of the same material, such as, but not limited to, silicone. In some embodiments, the deformable sensor 112 can be attachable. For example, the enclosure 313 can include a bracket that can be attached to a suitable object, such as the robot 100 or a material. The deformable membrane 320 can be latex or other suitable material, such as a suitable thin non-porous rubber-like material, hi some embodiments, the deformable membrane 320 is laser cut from a 0.04 mm thick sheet of latex.

[0034] As used herein, "deformability" can refer to, for example, the ease with which a deformable sensor can deform. Deformability can refer to how easily a deformable membrane deforms when it contacts a target object. The deformability of the deformable sensor 112 can be adjusted / modified by changing the material of the deformable membrane 320 and / or the pressure in the surrounding body 313. By using a softer material (e.g., soft silicone), the deformable sensor 112 can deform more easily. Similarly, lowering the pressure in the surrounding body 313 can also allow the deformable membrane 320 to deform more easily, resulting in a more deformable sensor 112. In some embodiments, the deformable membrane 320 inflates to a height of 20 mm to 75 mm and a pressure of 0.20 psi to 0.30 psi.

[0035] As used herein, "spatial resolution" can refer to, for example, how many pixels a deformable sensor has. The number of pixels can be from one (e.g., a sensor that only detects contact with a target object) to thousands or millions of pixels (e.g., a high-density tactile sensor provided by a time-of-flight sensor with thousands of pixels), or any suitable number. The deformable sensor 112 can have high spatial resolution due to the high-density tactile sensor provided as an end effector of the robot 100, thereby providing the robot 100 with a fine sense of touch like a human finger. The deformable sensor 112 can also have depth resolution to measure movement toward and away from the sensor. In some embodiments, the deformable sensor 112 features changing tactile sensations due to changes in spatial resolution and / or depth resolution.

[0036] An internal sensor 330 capable of sensing depth can be disposed within the enclosing body 313. The enclosing body can be measured by the depth resolution of the internal sensor 330. The internal sensor 330 can have a field of view 332 through the medium to the bottom surface of the deformable membrane 320. In some embodiments, the field of view 332 of the internal sensor 330 is 62° x 45° + / - 10%. In some embodiments, the internal sensor 330 can be an optical sensor. As described in more detail below, the internal sensor 330 can detect the distortion of the deformable membrane 320 when the deformable membrane 320 contacts an object. In one example, the internal sensor 330 is a time-of-flight sensor capable of measuring depth. A time-of-flight sensor has individual detectors (i.e., "pixels") that emit an optical signal (e.g., an infrared signal) and detect the time it takes for the reflected signal to return to the sensor. The time-of-flight sensor can have any desired spatial resolution. The more pixels there are, the greater the spatial resolution. The spatial resolution of the sensors disposed within the internal sensor 330 can vary. In some cases, a lower spatial resolution (e.g., one pixel to detect a displacement of one point) may be preferred. In other cases, a high sensitivity time-of-flight sensor can be used, such as the high spatial resolution internal sensor 330 that provides high density tactile sensitivity. Thus, the internal sensor 330 can be modular, since the sensor can be changed depending on the application. A non-limiting example of a time-of-flight sensor is the Pico Flexx sold by PMD Technologies AG (Siegen, Germany). Other types of visual internal sensors include, by way of non-limiting example, stereo cameras, laser distance sensors, stereoscopic lighting sensors / 3D scanners, single cameras (with dots or other patterns within), or any other type of visual detector. For example, the internal sensor 330 can be configured as a stereo camera that can detect the distortion of the deformable membrane 320 by an object.

[0037] Any suitable number and / or type of internal sensors 330 may be utilized in a single deformable sensor 112 in some embodiments. In some examples, not all of the internal sensors 330 in a deformable sensor 112 need be of the same type. In various embodiments, one deformable sensor 112 may utilize a single internal sensor 330 having a high spatial resolution, while another deformable sensor 112 may use multiple internal sensors 330, each having a lower spatial resolution. In some embodiments, the spatial resolution of a deformable sensor 112 may be increased by increasing the number of internal sensors 330. In some examples, a reduction in the number of internal sensors 330 in a deformable sensor 112 may be compensated for by correspondingly increasing the spatial resolution of at least some of the remaining internal sensors 330. As discussed in more detail below, the aggregate deformation resolution may be measured as a function of the deformation resolution or depth resolution between the deformable sensors 112 in a portion of the robot 100. In some embodiments, the aggregate deformation resolution can be based on the number of deformable sensors 112 in a portion of the robot 100 and the deformation resolution obtained from each deformable sensor 112 in the portion.

[0038] 3, in the enclosure 313, a power conduit 314 may be utilized to provide power and / or data / signals to the internal sensor 330, etc., via a cable, such as for a USB (Universal Serial Bus) or other suitable type of power and / or signal / data connection. As used herein, an airtight conduit may include any type of passageway through which air or other fluids (such as liquids) cannot pass. In this example, the power conduit 314 is airtight, provides a passageway for solids (such as wires / cables) to pass through, and has an airtight seal (such as an O-ring) formed around the wires / cables at both ends of the power conduit 314. Other embodiments may utilize a wireless internal sensor 330 to transmit and / or receive data and / or power. In various embodiments where the medium is not a gas, such as silicon, the enclosure 313 and / or the power conduit 314 may not necessarily be airtight.

[0039] In some embodiments, the internal sensor 330 can include one or more internal pressure sensors (barometers, pressure sensors, etc., or any combination thereof) utilized to detect the overall deformation of the deformable membrane 320 through a medium. In some embodiments, the deformable sensor 112 and / or the internal sensor 330 can transmit and receive various data, such as via the power conduit 314 described above, wireless data transmission (WiFi, Bluetooth, etc.), or other suitable data communication protocols. For example, the pressure in the deformable sensor 112 can be specified by a pressurization parameter, which can be inversely proportional to the deformability of the deformable sensor 112. In some embodiments, the deformability of the deformable sensor 112 can be modified by changing the pressure in the enclosure 313 or the material of the deformable membrane 320. In some embodiments, the receipt of updated parameter values ​​can result in real-time or delayed updates (such as pressurization).

[0040] Next, FIGS. 5-7 are schematic diagrams of another deformable sensor 112'. FIG. 5 is a cross-sectional view of the deformable sensor 112'. The deformable sensor 112' is similar to the deformable sensor 112 shown in FIGS. 3 and 4, and generally includes a housing 410 and a bubble module 411 coupled to the housing 410. The bubble module 411 includes a deformable membrane 420 similar to the deformable membrane 320 shown in FIGS. 3 and 4. The deformable membrane 420 can therefore include any of the features disclosed herein in relation to the deformable membrane 320. The bubble module 411 of the deformable sensor 112' is similar to the upper portion 311 of the deformable sensor 112 shown in FIGS. 3 and 4. However, the bubble module 411 can be removed from the housing 410 and therefore can be replaced when necessary. The bubble module 411 defines an enclosure 413 into which a medium is filled through one or more fluid conduits 412. The conduit may be a valve or other suitable mechanism that passes through the housing 410 and terminates in the bubble module 411. As shown, the fluid conduit 412 includes a tube 412A and a tube fitting 412B. The fluid conduit 412 may be utilized to fill or empty the enclosure 413. When the enclosure 413 is filled with media, the deformable membrane 420 forms a dome shape as shown in Figures 5 and 6.

[0041] An internal sensor 430, capable of sensing depth similar to the internal sensor 330, can be disposed within the housing 410. The housing can be scaled with the depth resolution of the internal sensor 430. The internal sensor 430 has a field of view 432 with an angle A1 through the medium toward the bottom surface of the deformable membrane 420. As a non-limiting example, the angle A1 of the field of view 432 of the internal sensor 430 is 62° x 45° + / - 10%. In some embodiments, the internal sensor 430 can be an optical sensor. As described in more detail below, the internal sensor 430 can detect distortion of the deformable membrane 420 when the deformable membrane 420 contacts an object. In one implementation, the internal sensor 430 is a time-of-flight sensor capable of measuring depth. A time-of-flight sensor has individual detectors (i.e., pixels) that emit a light signal (e.g., an infrared signal) and detect the time it takes for the reflected signal to return to the sensor.

[0042] 5, the internal sensor 430 is disposed within the housing 410 and is oriented at an angle A2 relative to the bubble module 411 and the deformable membrane 420. Specifically, the internal sensor 430 extends along an axis that forms an angle A2 between the axis of the internal sensor 430 and the back plate 422 of the bubble module 411, which will be discussed in more detail below. As a non-limiting example, the angle A2 between the internal sensor 430 and the bubble module 411 may be 35° + / - 10%. The angled internal sensor 430 maximizes the field of view 432 and the accuracy of the depth measurement at the center and distal edges of the deformable membrane 420 opposite the internal sensor 430, while minimizing the overall width dimension of the deformable sensor 112'.

[0043] 6 and 7, the bubble module 411 of the deformable sensor 112' is shown separate from the housing 410. FIG. 6 shows the bubble module 411 in its assembled form, whereas FIG. 7 shows an exploded view of the bubble module 411. The bubble module 411 includes a deformable membrane 420, a back plate 422, and a ring 424 for securing the deformable membrane 420 to the back plate 422. The bubble module 411 can be removably coupled to the housing 410 using any suitable means, such as a threaded insert 425 that extends through a hole 427 in the back plate 422 to secure the back plate 422 to the housing 410. Alternatively or additionally, the threaded insert 425 can be used to further secure the outer edge 421 of the deformable membrane 420 to the back plate 422.

[0044] In particular, the back plate 422 includes a housing surface 422A, a membrane surface 422B, and an edge surface 422C extending between the housing surface 422A and the membrane surface 422B. The back plate 422 is formed from a transparent material, such as acrylic, so that the field of view 432 of the internal sensor 430 is not obstructed by the bubble module 411. When assembling the bubble module 411, an adhesive can be applied to the edge surface 422C of the back plate 422. The outer edge 421 of the deformable membrane 420 can then be positioned around the back plate 422 so as to contact and adhere to the edge surface 422c. Additionally, a ring 424 can be positioned around the edge surface 422C of the back plate 422 to sandwich the deformable membrane 420 between the back plate 422 and the ring 424. As described above, the threaded insert 425 can be used to further secure the deformable membrane 420 to the backplate 422 by positioning the outer edge 421 of the deformable membrane 420 along the housing surface 422A of the backplate 422 and inserting the threaded insert 425 through the outer edge 421 of the deformable membrane 420 and the backplate 422. The illustrated tube fitting 412B is attached to the backplate 422 at the orifice 423, and the tube 412A extends from the tube fitting 412B for delivering media into the bubble module 411.

[0045] Thus, if the deformable sensor 112' is damaged, for example if a hole is created in the deformable membrane 420 causing media to leak out of the bubble module 411, the deformable sensor 112' can be repaired without disturbing the housing 410 and the electrical components therein, such as the internal sensor 430. In this case, the bubble module 411 can be removed from the housing 410 using the threaded insert 425 or any other suitable means, and a bubble module replacement 411 can be coupled to the housing 410. Alternatively, it may be preferable to repair the existing bubble module 411 by replacing only the deformable membrane 420, or by repairing the deformable membrane 420 itself by patching to close any holes or other damaged areas. It should be appreciated that by providing the deformable sensor 112' with an easily replaceable bubble module 411, a larger portion of the deformable sensor 112' can be accommodated within the robot 100, with only the bubble module 411 exposed and accessible from the exterior of the robot 100. This reduces the size of the robot 100 and reduces the possibility of damage to the deformable sensor 112' during operation.

[0046] Turning now to FIG. 8 , in some embodiments, the deformable sensor 112, 112′ can include an optional filter layer 323. In a non-limiting example, the filter layer 323 shown is provided on the deformable sensor 112. The filter layer 323 can be disposed on the bottom surface 321 of the deformable membrane 320. As described in more detail herein, the bottom surface 321 of the deformable membrane 320 can be patterned (e.g., a dot pattern, a grid pattern, or any other suitable pattern). As a non-limiting example, a stereo camera can be utilized to detect the displacement of the deformable membrane 320 based on the identified deformation of the patterned bottom surface 321. The filter layer 323 can be configured to assist the internal sensor 330 in detecting the deformation of the deformable membrane 320. In some embodiments, the filter layer 323 reduces glare or inappropriate reflection of one or more optical signals emitted by the internal sensor 330. In some embodiments, the filter layer 323 can scatter one or more optical signals emitted by the internal sensor 330. The filter layer 323 may be an additional layer secured to the bottom surface 321 of the deformable membrane 320 or may be a coating and / or pattern applied to the bottom surface 321 of the deformable membrane 320 .

[0047] 9, in some embodiments, the deformable sensor 112, 112' can include an internal sensor filter 335. In a non-limiting example, the internal sensor filter 335 shown is provided on the internal sensor 330 of the deformable sensor 112. The internal sensor filter 335 can be positioned within the field of view 332 of the internal sensor 330. The internal sensor filter 335 can optimize the optical signal emitted by the internal sensor 330 for reflection on the bottom surface 321 of the deformable membrane 320. Similar to the filter layer 323, the internal sensor filter 335 can be positioned within the field of view 332 of the internal sensor 330 and can reduce glare or inappropriate reflection of the optical signal emitted by the internal sensor 330. In some embodiments, the internal sensor filter 335 can scatter one or more optical signals emitted by the internal sensor 330. In some embodiments, both the filter layer 323 and the internal sensor filter 335 can be utilized.

[0048] The pattern can be applied to the bottom surface of the deformable membrane 320 of the deformable sensor 112 or the bottom surface of the deformable membrane 420 of the deformable sensor 112'. As shown in FIG. 10, as a non-limiting example, a dot pattern 325 including a plurality of arranged dots can be applied to the bottom surface 321 of the deformable membrane 320 on the optional filter layer 323 or to the deformable membrane 320 itself to help detect the deformation of the deformable membrane 320. For example, the dot pattern 325 helps detect the deformation if the internal sensor 330 is a stereo camera. Alternatively, a stereo camera can be provided in addition to the internal sensor 330 to complement the deformation detection of the internal sensor 330. The change in the degree of distortion of the dot pattern 325 can be used to recognize how much deformation has occurred in the deformable membrane 320. The pattern on the bottom surface 321 can be random and does not need to be arranged in the dot pattern 325 or the array shown in FIG. 10.

[0049] In some embodiments where a dot pattern 325 is provided, an initial or pre-deformation image of the dot pattern 325 on the bottom surface 321 of the deformable membrane 320 can be captured before deformation of the deformable membrane 320. The internal sensor 330 or a separate stereo camera (if provided) then captures at least one post-deformation image of the dot pattern 325 during or after deformation of the deformable membrane 320. The pre-deformation image can be compared to the post-deformation image, and the position of each dot in the pre-deformation image can be compared to the corresponding dot in the post-deformation image to measure the displacement of the dot and therefore the displacement of the deformable membrane 320. The displacement of each dot can be used to measure the deformation of an individual quadrant or section of the dot pattern 325. The displacement of each dot is then converted to a distance measurement to measure the intrinsic deformation of the deformable membrane 320 or a section thereof in order to recognize the shape and / or pose of the object that deforms the deformable membrane 320.

[0050] In some embodiments, measurements between each dot or at least some of the dots of the dot pattern 325 can be stored in a memory module, such as memory module 232 (FIG. 2) of the deformable sensor 112 or an associated processor, such as processor 230 (FIG. 2). In this way, instead of simply measuring the shape and / or pose of the target object, dimensions of various sections of the target object can be measured by calculating the inherent deformation between adjacent dots of the dot pattern 325. When the dot pattern 325 includes more dots, the dot pattern 325 allows for detection of deformations within a smaller area of ​​the deformable membrane 320 compared to when the dot pattern 325 includes a smaller number of dots. In embodiments in which the dots of the dot pattern 325 are arranged in an array, the dots can be equally spaced from one another or arranged in any other suitable manner. However, in some embodiments, when the distances between the dots are not equally spaced from one another, the distances between the dots are stored in the memory module to identify the arrangement of the dots. It should further be appreciated that the same technique of comparing pre-deformation images with post-deformation images discussed above can be repeated for multiple post-deformation images taken during deformation of the deformable membrane 320 to provide real-time data regarding the shape, dimensions and / or pose of the target object. By comparing the post-deformation images to one another, the displacement of the deformable membrane 320 that occurs within a smaller time increment can be measured compared to the entire deformation of the deformable membrane 320 from an initial pre-deformation state.

[0051] 11, the pattern can be a grid pattern 322 applied to the bottom surface 321 of the deformable membrane 320 to aid in detecting deformation of the deformable membrane 320. For example, the grid pattern 322 can aid in detecting deformation when the internal sensor 330 is a stereo camera. For example, the change in the degree of distortion of the grid pattern 322 can be used to recognize how much deformation has occurred. In this example, the measurement of the distance between parallel lines of the grid pattern 322 and / or the curvature of the lines can be used to measure the amount of deformation at each point in the grid pattern 322. The pattern of the bottom surface 321 can be random and not necessarily arranged in the grid pattern 322 or in the array shown in FIG. 11. It should be understood that the embodiments are not limited to grid patterns and dot patterns as discussed herein, but other types of patterns are possible, such as shapes and the like.

[0052] FIG. 12 next illustrates an embodiment of a composite internal sensor 330' that can be used in place of the internal sensor 330 of the deformable sensor 112 or the internal sensor 430 of the deformable sensor 112'. In this embodiment, multiple internal sensors 502 are shown that are time-of-flight cameras. Other embodiments can utilize any combination of different types of internal sensors. In this embodiment, a cable 504 is used to provide data communication and power to the internal sensors, but other embodiments can use a different number of cables or wireless connections for data and / or power. In this embodiment, one support structure 506 is shown, but other embodiments can utilize multiple support structures 506 or no support structure. In this embodiment, the support structure 506 is rigid, but one or more of the support structures 506 can be flexible in some embodiments to change the orientation of the internal sensor 502. In this embodiment, the cable 504 can be connected to a base portion 508 for data communication and / or power.

[0053] FIG. 13 shows an image of an example object 615 displacing the deformable membrane 320 of the example deformable sensor 112. It should be understood that the deformable sensor 112′ can also be used in the same manner as discussed herein. In the illustrated embodiment, the display device 640 displays the output of the deformable sensor 112 on the device in real time as the object 615 contacts and / or deforms the deformable membrane 320. It should be understood that the display device 640 is provided for illustrative purposes only and that this embodiment can be used without the display device. When the object 615 is pushed into the deformable membrane 320, the object 615 imparts a shape to the deformable membrane 320 such that the deformable membrane 320 adapts to the shape of the object 615. The spatial resolution of the internal sensor 330 can be such that the internal sensor 330 detects the shape and / or pose of the displaced deformable membrane 320. For example, if the internal sensor 330 is a time-of-flight sensor, the optical signal reflected from the bottom surface 321 of the deformable membrane 320 being distorted by an object will have a shorter time of flight than the optical signal reflected by the deformable membrane 320 in areas outside the distorted area. Thus, a contact area 642 (or displacement area, as used interchangeably herein) having a shape and / or pose matching the shape of the object 615 can be output to and displayed on the display device 640.

[0054] Thus, the deformable sensor 112 can detect the presence of contact with the object 615 as well as the shape of the object 615. In this manner, a robot equipped with the deformable sensor 112 or 112' can measure the shape of the object based on contact with the object. Furthermore, the shape and / or pose of the object 615 can also be measured based on the shape information sensed by the deformable sensor 112, 112'. For example, a vector 644 perpendicular to the plane of the contact area 642 can be displayed, such as when measuring the pose of the object 615. The vector 644 can be used by the robot or another device, for example, to measure in which direction a particular object 615 is facing.

[0055] 14 is a flow chart of an example method 700 for measuring a pose and a force associated with an object in contact with the deformable sensor 112, 112'. However, as discussed herein, without limiting the scope of the disclosure, reference is made to FIGS. 3 and 4, which show the deformable sensor 112. In block 702, a medium (gas, liquid, silicon, etc.) is placed in an enclosure 313, in which the deformable membrane 320 is coupled to the upper portion 311 of the housing 310. In block 704, the deformation of the deformable membrane 320 can be measured based on contact with the object 615 by an internal sensor 330 in the enclosure 313 having a line of sight 332 through the medium to the bottom surface 321 of the deformable membrane 320. In block 706, the pose of the object 615 can be measured based on measuring the deformation of the deformable membrane 320, such as the contact area 642. As discussed above in connection with FIG. 13, the pose of the object 615 can be measured by forcing the object 615 into the deformable membrane 320, which conforms to the shape of the object 615. The internal sensor 330 then detects the shape and / or pose of the displaced deformable membrane 320. In block 708, the amount of force between the deformable membrane 320 and the object 615 is measured based on the measurement of the deformation of the deformable membrane 320. Blocks 706 and 708 can be performed simultaneously, but are not necessarily simultaneous. In block 710, it is determined whether further deformation and / or contact is detected. If further deformation or contact is detected, the flow chart returns to block 704. If not, the flow chart can end.

[0056] FIG. 16 now illustrates a flow chart of an example method 900 for determining the location of the robot 100 in a space, with reference to the robot 100 shown in FIGS. 1 and 2 and the example space 800 shown in FIG. 15 in which the robot 100 performs operations. Reference is made to the robot 100 utilizing a deformable sensor 112 as discussed herein. However, the method 900 is equally applicable to a robot 100 including a deformable sensor 112′. First, in block 902, a database is created. The database includes information of at least one space and objects present in the space, such that the robot 100 can identify its location in the space. The database can include information about a number of different spaces, such as different rooms in a house or facility. Thus, the spaces stored in the database can include unique identifiers to distinguish the spaces, such as “bedroom”, “kitchen”, “living room”, etc. For each space, a floor plan can be provided initially before the robot 100 operates, or the floor plan can be surveyed in real time by the robot 100, such that the robot 100 can determine its location in the space. In some embodiments, the floor plan stored in the database includes dimensions of the space, locations on the floor surface of the space of objects that the robot 100 may come into contact with during its operations, dimensions of the objects and / or clearance distances between objects in the space. Additionally, the database may include inherent information such as the material of construction of the object to associate certain characteristics with the object such as texture, hardness, flexibility, and / or the like. For example, if an object, such as a kitchen table, is identified as being made of wood, the database may assign the object the characteristic that it has an inherent hardness associated with wood or whatever material the object is made of. Or, if an object is identified as being made of fabric or leather, such as a chaise lounge, the database may assign the object the characteristic that it has a hardness associated with fabric or leather, which may be less than the hardness of a wooden object. Other characteristics that may be assigned to an object may include that the object has rounded edges, corners, protrusions, surface features, and the like. The database need not include each of the information and / or characteristics discussed herein.However, it will be appreciated that the more information and characteristics relating to the space and the objects placed therein that are provided in the database, the faster and more efficiently the robot 100 can determine its position in the space.

[0057] The space and / or object information discussed herein can be entered into the database manually, such as by activating the input device 238 or the portable electronic device 280. Thus, the space and / or object information can be modified and / or erased as necessary when the space and / or object are changed, such as when furniture is moved or replaced. The database can be stored on the network 270 or in the memory module 232 of the robot 100 itself. However, if the database is stored on the network 270, the robot 100 uses a connection to the network 270 during a task to retrieve and utilize the information stored in the database. Alternatively, the robot 100 can be instructed to download the database, or one or more portions of the database, from the network 270 to the memory module 232 of the robot 100 prior to performing a task. After the robot 100 has completed a task, the database can be erased to make storage space available for the next download of the database or portions thereof.

[0058] The deformable sensor 112 can also be used to survey a space when it contacts an object or a wall of the space during exploration. This allows the robot 100 to automatically survey a space during exploration of the robot 100, reducing or possibly eliminating the need for a user to manually input information about the space and objects contained therein. However, when surveying a space in real time during an operation, the robot 100 may not be able to measure its position until it has surveyed a sufficient amount of the space. In some embodiments, the robot 100 can include additional sensing devices, such as optical sensors, acoustic sensors, time-of-flight cameras, laser scanners, ultrasonic sensors, or the like, in addition to the deformable sensor 112, to automatically survey a space and measure information about objects contained therein.

[0059] 1 and 2 located in an example space 800, such as a family room or living room of a house shown in FIG. 15, the robot 100 is activated to perform a task at block 904. If the robot 100 is an autonomous vacuum cleaner, the task is a cleaning task. If the robot is an object retrieval robot or an object delivery robot, the task may be navigating to a destination to retrieve or deliver an object. The example space 800 includes a number of objects, such as an entertainment system 802, a couch 804, a table 806, a number of chairs 808 arranged around the table 806, and a cabinet 810. As mentioned above, in some embodiments, information about the space 800 and the objects included in the space 800 is manually entered into a database before the robot 100 begins its task. Thus, the database may include information such as the location and / or dimensions of the space 800, the entertainment system 802, the couch 804, the table 806, the number of chairs 808, and the cabinet 810. As mentioned above, the database may also include the materials of construction of each of the objects and / or the distances between each of the objects.

[0060] In the working example shown in FIG. 15, the robot 100 navigates through the space 800, and in block 906, one of the multiple deformable sensors 112 extending from the robot 100 contacts a first object, such as a cabinet 810 (possibly made of wood). Upon contact with the cabinet 810, the portion of the deformable sensor 112 in contact with the cabinet 810 deforms. In this working example, the first object is the cabinet 810. However, the first object may be any other object in the space 800 based on the initial starting point and travel direction of the robot 100. As will be discussed in more detail below with respect to the deformable sensor 112, the deformable sensor 112 provides data including the shape, pose, hardness, and / or flexibility of the cabinet 810 in block 908. Thus, based on the inherent deformation of the deformable sensor 112 in the illustrated embodiment shown in FIG. 15, the robot 100 identifies that the cabinet 810 has an inherent hardness. Furthermore, the robot 100 can recognize when the deformable sensor 112 is in contact with an edge or corner of the cabinet 810 based on the inherent deformation of the deformable sensor 112 .

[0061] In some embodiments, the robot 100, such as the edge surface 108 of the robot 100, may contact an object at a point between adjacent deformable sensors 112 such that none of the deformable sensors 112 deform upon hitting the object. In this case, the robot 100 can turn or rotate to reposition itself so that one of the deformable sensors 112 contacts the object and deforms. In some embodiments, the robot 100 includes additional smaller deformable sensors provided along the edge surface 108 of the robot 100 between adjacent deformable sensors 112 to determine where the robot 100 touches the object and how to reposition the robot 100 so that one of the deformable sensors 112 contacts the object.

[0062] In an embodiment where the database is populated with information about the space 800 and objects contained therein prior to the operation, the robot 100 compares the acquired information or data (measured by the deformable sensor 112) regarding the first object, e.g., 810, with the information associated with each of the objects in the space 800 stored in the database, in block 910. In doing so, the robot 100 can narrow at least possible options for identifying the first object that the robot 100 contacted in the space 800, in block 912. For example, the robot 100 can rule out that the contacted object is a fabric or leather couch 804, because the hardness of the couch 804 identified in the database is less than the hardness associated with the wooden cabinet 810. This allows the robot 100 to determine (or at least narrow) the location of the robot 100 in the space 800 based on possible objects in the space 800 that the robot 100 may have contacted, in block 914.

[0063] If the database contains multiple spaces, the robot 100 can compare the contacted first object to each object in each of the spaces in the database, although in some embodiments the robot 100 can compare the first object only to objects in a specific space or a subset of spaces based on instructions from a user indicating which space the robot 100 should begin its task in. Alternatively, the position sensor 250 of the robot 100 can be utilized to measure the overall position of the robot 100. This can help narrow the number of spaces the robot 100 may operate in.

[0064] To identify the first object as one of the multiple objects in the data and therefore the location of the robot 100, the robot 100 can also take into account the sensing distance around the object and the direction the robot 100 traveled before contacting the first object. As shown in Fig. 15, the robot 100 moves in a straight line from a starting point 803 along a distance 801 before contacting a cabinet 810. Based on this information and the information associated with the space 800 presented in the database, the robot 100 can eliminate possible objects such as a table 806 and a chair 808 that do not provide sufficient clearance to travel the distance 801.

[0065] Once the robot 100 identifies the first object that it contacted as the cabinet 810, the robot 100 can determine its location within the space 800 in block 914 based on the known location of the cabinet 810 within the space 800 stored in the database. However, if the robot 100 cannot accurately identify the first object based on the limited amount of information acquired by the deformable sensor 112, the robot 100 continues its task until it contacts a next object to confirm the identity of the first object and the location of the robot 100 from many possible locations within the space 800. For example, after contacting the first object, such as the cabinet 810, the robot 100 moves away from the first object and travels in another direction to continue its task. The robot 100 then contacts a next object, such as the couch 804, in block 916, and receives a next signal based on contacting the next object. In block 918, while contacting the next object, e.g., the couch 804, the deformable sensor 112 identifies the shape and hardness of the second object in a manner similar to that described herein in relation to the first block in blocks 908-912. In block 920, this information is compared to information presented in the database, and in block 922, the next object is identified as a next identified object of one of the objects in the database, or at least the possible identities of the first and next objects are narrowed. As described herein, the robot 100 can take into account the distance traveled from the point where the robot 100 contacted the first object to the point where the robot 100 contacted the next object to help accurately identify the first and next objects.

[0066] Once the identity of the first object, and in some cases the identity of the next object, has been confirmed in block 924, the robot 100 can determine its position within the space 800. In this way, the robot 100 can continue its task and avoid further contact with other objects within the space 800 based on the location information of other objects within the space 800 presented in the database. This is beneficial when the robot 100 needs to navigate a specific location within the space 800, such as to perform a cleaning task or an object retrieval / delivery task. The location of the robot 100 can be stored in the memory module 232 so that the robot 100 can continue to track the location of the robot within the space 800 in subsequent tasks. Thus, no contact with an additional or next object is necessary. Instead, the robot 100 can repeat blocks 916-924 for further objects if the location of the robot 100 is not accurately determined.

[0067] It should be understood that the embodiments of the present disclosure relate to a deformable sensor that can detect contact with an object as well as the shape and pose of the object. One or more deformable sensors can be provided on a robot, for example. The information provided by the deformable sensor can be used to control the interaction of the robot with a target object. The depth resolution and spatial resolution of the deformable sensor can vary depending on the location of the deformable sensor on the robot.

[0068] Describing components of the present disclosure as being "configured" or "programmed" in a particular manner to achieve particular properties or to function in a particular manner is a structural description, not an indication of intended use. More specifically, references to the manner in which a component is "configured" or "programmed" refer to the existing physical state of the component, and should therefore be considered as definitive references to the structural characteristics of the component.

[0069] The order of execution or performance of operations in the disclosed embodiments illustrated and described herein is not essential unless specifically specified. That is, operations can be performed in any order unless specifically specified, and embodiments of the present disclosure can include more or fewer operations than those disclosed herein. For example, it is contemplated within the scope of the present disclosure for an operation to be performed or performed before, contemporaneously with, or after another operation.

[0070] The terms "substantially," "approximately," and "about" may be used herein to express the inherent degree of uncertainty associated with a quantitative comparison, value, measurement, or other representation. These terms are also used to express the degree to which a quantitative representation may vary from the stated basis without changing the basic function of the object.

[0071] While specific embodiments have been illustrated and described herein, it should be understood that various changes and modifications can be made without departing from the spirit and scope of the claimed subject matter. Moreover, although various forms of the claimed subject matter have been described herein, these forms do not necessarily have to be utilized in combination. It is therefore intended that the claims cover all such changes and modifications that are within the scope of the claimed subject matter.

[0072] [Example 1] 1. A method for measuring a position of a robot including a deformable sensor, comprising: receiving, by a processor, a signal from the deformable sensor, the signal including data related to an area of ​​deformation of a deformable membrane of the deformable sensor resulting from contact with a first object; comparing, by the processor, the data associated with contact with the first object to information associated with a plurality of objects stored in a database; identifying, by the processor, the first object as a first identified object among a plurality of objects stored in the database, the first identified object being the object among the plurality of objects stored in the database that is most similar to the first object; determining, by the processor, a position of the robot based on a position of the first identified object; A method comprising: [Example 2] determining, by the processor, a distance traveled to the first object; identifying clearance distances within information about the plurality of objects stored in the database; comparing the distance traveled by the robot to the first object to the clearance distances of the plurality of objects; The method of Example 1, further comprising: [Example 3] 2. The method of example 1, further comprising receiving, by the processor, a next signal from the deformable sensor including data regarding the deformation area of ​​the deformable membrane resulting from contact with a next object. [Example 4] 4. The method of example 3, further comprising comparing, by the processor, data associated with contact with the next object to information associated with the plurality of objects stored in the database, the information including at least one of a shape, a pose, a stiffness, a flexibility, and a position of the plurality of objects. [Example 5] identifying, by the processor, the next object as a next identified object from the plurality of objects stored in the database; Further comprising: the position of the robot is based on the position of the first identified object and the position of the next identified object; The method described in Example 4. [Example 6] 2. The method of claim 1, further comprising, when the robot contacts the first object and the deformable sensor does not contact the first object, repositioning the robot, by the processor, so that the deformable sensor contacts the first object. [Example 7] 2. The method of example 1, further comprising modifying the deformability of the deformable sensor by varying a pressure within an enclosure defined in part by the deformable membrane. [Example 8] utilizing an internal sensor disposed within the deformable sensor and having a line of sight through a medium stored within an enclosure of the deformable sensor to a bottom surface of the deformable membrane; Scattering an optical signal emitted by the internal sensor by a filter layer disposed on a bottom surface of the deformable membrane; The method of Example 1, further comprising: [Example 9] 9. The method of example 8, further comprising: analyzing, by the processor, the deformation region by measuring a change in a coating or pattern on the bottom surface of the deformable membrane. [Example 10] 9. The method of example 8, wherein the internal sensor comprises a time-of-flight sensor. [Example 11] A robot for measuring a position in a space, comprising: a casing including an upper surface, an opposing lower surface, and an edge surface extending between the upper surface and the lower surface; at least one deformable sensor provided on the casing, the deformable sensor comprising an internal sensor and a deformable membrane, the internal sensor configured to output a deformation area in the deformable membrane resulting from contact with a first object; one or more processors; One or more memory modules comprising a computer readable medium storing computer readable instructions that, when executed by the one or more processors, cause the one or more processors to: receiving data from the internal sensor representative of a deformation area of ​​the deformable membrane when in contact with the first object; comparing data associated with the contact of the first object with information associated with a plurality of objects stored in a database; identifying the first object as a first identified object among the plurality of objects stored in the database, the first identified object being an object most similar to the first object among the plurality of objects stored in the database; determining a position of the robot based on a position of the first identified object; one or more memory modules; A robot equipped with the above. [Example 12] The one or more memory modules include a computer readable medium that stores computer readable instructions that, when executed by the one or more processors, cause the one or more processors to: causing a processor to measure a distance traveled to the first object; identifying clearance distances within information about the plurality of objects stored in the database; comparing a distance traveled by the robot to the first object with the clearance distances of the plurality of objects; The robot described in Example 11. [Example 13] The one or more memory modules include a computer readable medium that stores computer readable instructions that, when executed by the one or more processors, cause the one or more processors to: receiving, by the processor, a next signal from the deformable sensor, the next signal including data regarding the deformation area of ​​the deformable membrane resulting from contact with a next object; The robot according to Example 11. [Example 14] The one or more memory modules include a computer readable medium that stores computer readable instructions that, when executed by the one or more processors, cause the one or more processors to: causing the processor to compare data associated with the contact of the next object with information associated with the plurality of objects stored in the database, the information including at least one of a shape, a pose, a stiffness, a flexibility, and a position of the plurality of objects; The robot described in Example 13. [Example 15] The one or more memory modules include a computer readable medium that stores computer readable instructions that, when executed by the one or more processors, cause the one or more processors to: identifying, by the processor, the next object as a next identified object from the plurality of objects stored in the database; the position of the robot is based on the position of the first identified object and the position of the next identified object. The robot described in Example 14. [Example 16] 1. A system for measuring a position of a robot including a deformable sensor, comprising: a robot including an upper surface, an opposing lower surface, and an edge surface extending between the upper surface and the lower surface; at least one deformable sensor provided on the robot, the at least one deformable sensor comprising a housing, a deformable membrane coupled to an upper portion of the housing, an enclosure configured to be filled with a medium, and an internal sensor disposed within the housing and having a field of view configured to pass through the medium toward a bottom surface of the deformable membrane, the internal sensor configured to output a deformation area in the deformable membrane resulting from contact with a first object; one or more processors; One or more memory modules comprising a computer readable medium storing computer readable instructions that, when executed by the one or more processors, cause the one or more processors to: receiving data from the internal sensor representative of the deformation area upon contact with the first object; comparing data associated with contact with the first object with information associated with a plurality of objects stored in a database; identifying the first object as a first identified object among the plurality of objects stored in the database, the first identified object being the object most similar to the first object among the plurality of objects stored in the database; determining a position of the robot based on a position of the first identified object; one or more memory modules; A system comprising: [Example 17] The one or more memory modules include a computer readable medium that stores computer readable instructions that, when executed by the one or more processors, cause the one or more processors to: measuring a distance traveled to the first object, and the information of the plurality of objects stored in the database includes a clearance distance; comparing a distance traveled by the robot to the first object with the clearance distances of the plurality of objects; The system of Example 16. [Example 18] The one or more memory modules include a computer readable medium that stores computer readable instructions that, when executed by the one or more processors, cause the one or more processors to: receiving a signal from the deformable sensor including data regarding a deformation area of ​​the deformable membrane resulting from contact with a next object; The system of Example 16. [Example 19] The one or more memory modules include a computer readable medium that stores computer readable instructions that, when executed by the one or more processors, cause the one or more processors to: comparing data associated with the contact of the next object to information associated with the plurality of objects stored in the database, the information including at least one of a shape, a pose, a stiffness, a flexibility, and a position of the plurality of objects; The system of Example 18. [Example 20] The one or more memory modules include a computer readable medium that stores computer readable instructions that, when executed by the one or more processors, cause the one or more processors to: identifying the next object as a next identified object among the plurality of objects stored in the database, and determining the position of the robot based on the position of the first identified object and the position of the next identified object; The system of Example 19.

Claims

1. 1. A method for measuring a position of a robot including a deformable sensor, comprising: receiving, by a processor, a signal from the deformable sensor, the signal including data related to an area of ​​deformation of a deformable membrane of the deformable sensor resulting from contact with a first object; comparing, by the processor, the data associated with contact with the first object to information associated with a plurality of objects stored in a database; determining, by the processor, a distance traveled to the first object; identifying clearance distances within information about the plurality of objects stored in the database; identifying, by the processor, the first object as a first identified object among the plurality of objects stored in the database based on a comparison of the distance traveled by the robot to the first object with the clearance distances of the plurality of objects, the first identified object being the object among the plurality of objects stored in the database that is most similar to the first object; determining, by the processor, a position of the robot based on a position of the first identified object; A method comprising:

2. The method of claim 1 , further comprising receiving, by the processor, a next signal from the deformable sensor that includes data regarding the deformation area of ​​the deformable membrane resulting from contact with a next object.

3. 3. The method of claim 2, further comprising comparing, by the processor, data associated with contact with the next object to information associated with the plurality of objects stored in the database, the information including at least one of a shape, a pose, a stiffness, a flexibility, and a position of the plurality of objects.

4. identifying, by the processor, the next object as a next identified object from the plurality of objects stored in the database; Further comprising: a position of the robot based on a position of the first identified object and a position of the next identified object; The method according to claim 3.

5. 2. The method of claim 1, further comprising, when the robot contacts the first object and the deformable sensor does not contact the first object, repositioning, by the processor, the robot so that the deformable sensor contacts the first object.

6. The method of claim 1 , further comprising modifying the deformability of the deformable sensor by varying a pressure within an enclosure defined in part by the deformable membrane.

7. utilizing an internal sensor disposed within the deformable sensor and having a line of sight through a medium stored within an enclosure of the deformable sensor to a bottom surface of the deformable membrane; Scattering an optical signal emitted by the internal sensor by a filter layer disposed on a bottom surface of the deformable membrane; The method of claim 1 further comprising:

8. The method of claim 7 , further comprising: analyzing, by the processor, the deformation region by measuring changes in a coating or pattern on the bottom surface of the deformable membrane.

9. The method of claim 7 , wherein the internal sensor comprises a time-of-flight sensor.

10. A robot for measuring a position in a space, comprising: a casing including an upper surface, an opposing lower surface, and an edge surface extending between the upper surface and the lower surface; at least one deformable sensor provided on the casing, the deformable sensor comprising an internal sensor and a deformable membrane, the internal sensor configured to output a deformation area in the deformable membrane resulting from contact with a first object; one or more processors; One or more memory modules comprising a non-transitory computer readable medium storing computer readable instructions, the computer readable instructions, when executed by the one or more processors, cause the one or more processors to: receiving data from the internal sensor representative of a deformation area of ​​the deformable membrane when in contact with the first object; comparing data associated with the contact of the first object with information associated with a plurality of objects stored in a database; causing a processor to measure a distance traveled to the first object; identifying clearance distances within information about the plurality of objects stored in the database; identifying the first object as a first identified object among the plurality of objects stored in the database based on a result of comparing a distance traveled by the robot to the first object with the clearance distances of the plurality of objects, the first identified object being an object that is most similar to the first object among the plurality of objects stored in the database; determining a position of the robot based on a position of the first identified object; one or more memory modules; A robot equipped with the above.

11. The one or more memory modules include a non-transitory computer readable medium that stores computer readable instructions that, when executed by the one or more processors, cause the one or more processors to: receiving, by the processor, a next signal from the deformable sensor, the next signal including data regarding the deformation area of ​​the deformable membrane resulting from contact with a next object; The robot according to claim 10.

12. The one or more memory modules include a non-transitory computer readable medium that stores computer readable instructions that, when executed by the one or more processors, cause the one or more processors to: causing the processor to compare data associated with the contact of the next object with information associated with the plurality of objects stored in the database, the information including at least one of a shape, a pose, a stiffness, a flexibility, and a position of the plurality of objects; The robot according to claim 11.

13. The one or more memory modules include a non-transitory computer-readable medium that stores computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to: identifying, by the processor, the next object as a next identified object from the plurality of objects stored in the database; a position of the robot based on a position of the first identified object and a position of the next identified object; The robot of claim 12.

14. 1. A system for measuring a position of a robot including a deformable sensor, comprising: a robot including an upper surface, an opposing lower surface, and an edge surface extending between the upper surface and the lower surface; at least one deformable sensor provided on the robot, the at least one deformable sensor comprising: a housing; a deformable membrane coupled to an upper portion of the housing; an enclosure configured to be filled with a medium; and an internal sensor disposed within the housing and having a field of view configured to pass through the medium toward a bottom surface of the deformable membrane, the internal sensor configured to output a deformation area in the deformable membrane resulting from contact with a first object; one or more processors; One or more memory modules comprising a non-transitory computer readable medium storing computer readable instructions, the computer readable instructions, when executed by the one or more processors, causing the one or more processors to: receiving data from the internal sensor representative of the deformation area upon contact with the first object; comparing data associated with contact with the first object to information associated with a plurality of objects stored in a database; measuring a distance traveled to the first object, and the information of the plurality of objects stored in the database includes a clearance distance; identifying the first object as a first identified object among the plurality of objects stored in the database, the first identified object being the object most similar to the first object among the plurality of objects stored in the database; determining a position of the robot based on a position of the first identified object and a result of comparing a distance traveled by the robot to the first object with the clearance distances of the plurality of objects; one or more memory modules; A system comprising:

15. The one or more memory modules include a non-transitory computer readable medium that stores computer readable instructions that, when executed by the one or more processors, cause the one or more processors to: receiving a signal from the deformable sensor including data regarding a deformation area of ​​the deformable membrane resulting from contact with a next object; The system of claim 14.

16. The one or more memory modules include a non-transitory computer readable medium that stores computer readable instructions that, when executed by the one or more processors, cause the one or more processors to: comparing data associated with the contact of the next object to information associated with the plurality of objects stored in the database, the information including at least one of a shape, a pose, a stiffness, a flexibility, and a position of the plurality of objects; The system of claim 15.

17. The one or more memory modules include a non-transitory computer readable medium that stores computer readable instructions that, when executed by the one or more processors, cause the one or more processors to: identifying the next object as a next identified object among the plurality of objects stored in the database, and determining a position of the robot based on a position of the first identified object and a position of the next identified object; 17. The system of claim 16.

Images