Generating autonomous procedures for connecting connectors

Abstract

To provide a method of autonomous maneuver generation to mate connectors.SOLUTION: A method includes providing an image of feature data on the basis of a feature extraction model of connector mating. The image depicts a first portion of a first device and a second portion of a second device. The feature data includes coordinates representing key points of each of the first and second devices depicted in the image. The method also includes obtaining position data indicating a position in 3D space of a connector of the first device. The method further includes providing the feature data and the position data to a trained autonomous agent to generate a proposed maneuver to mate the connector with a connector of the second device.SELECTED DRAWING: Figure 1

Claims

1. Method (800, 900), Providing a first image (170) as input to a feature extraction model (130) in order to generate feature data (176), wherein the first image depicts a first portion (156) of a first device (102) and a second portion (162) of a second device (112), and the feature data includes a first coordinate (160) representing a key point (158) of the first device depicted in the first image, and a second coordinate (166) representing a key point (164) of the second device depicted in the first image, and providing the first image as input to the feature extraction model. Acquiring position data (178) indicating the position of the first connector (106) of the first device in three-dimensional space (108) from one or more sensors (124) mounted on the first device (804, 908), wherein the first connector is located on the first part of the first device, and A method comprising providing the feature data and the location data as input to a trained autonomous agent (132) (806, 914) in order to generate a proposed procedure (180) for connecting the first connector to a second connector of the second device.

2. The method according to claim 1, wherein the first device includes an aerial refueling tanker (102A), the first portion of the first device includes a refueling boom (104A), the first connector includes a refueling connector (106A) of the refueling boom, the second device includes an aircraft (112A), and the second connector includes a refueling port (116A) of the aircraft.

3. The method according to claim 1 or 2, wherein the first device includes a first spacecraft (102B), the first connector includes a first docking connector (106B), the second device includes a second spacecraft (112B), and the second connector includes a second docking connector (116B).

4. The method according to any one of claims 1 to 3, wherein the position data indicates a position in a reference frame associated with the first device and independent of the second device.

5. The method according to any one of claims 1 to 4, further comprising receiving a source image and downsampling the source image one or more times (904) in order to generate the first image before providing the first image as input to the feature extraction model.

6. To generate a segmentation map (174) representing the distinguishable regions (432, 732) of the first image, the first image is provided to a segmentation model (136), and The method according to any one of claims 1 to 5, further comprising providing the classification map as input to the feature extraction model (906) in order to generate the feature data.

7. Performing a comparison (910) between the second coordinates representing the key points of the second device and the known geometric dimensions (142) of the second device, and The method according to any one of claims 1 to 6, further comprising modifying the feature data based on the comparison (912).

8. The method according to any one of claims 1 to 7, further comprising (918) commanding the rearrangement of the first portion of the first device based on the proposed procedure.

9. Device (102), A movable coupling system (104) configured to move the first connector (106) relative to the second connector (116) of the second device (112), One or more sensors (124) configured to generate position data (178) indicating the position of the first connector in three-dimensional space (108), A camera (122) configured to generate image data (170) depicting a portion (156) of the movable coupling system and at least a portion (162) of the second device, A feature extraction model (130) configured to receive a first image based on the image data and generate feature data (176) based on the first image, wherein the feature data includes a first coordinate (160) representing a key point (158) of the movable coupling system depicted in the first image, and a second coordinate (166) representing a key point (164) of the second device depicted in the first image, and A device including a trained autonomous agent (132) configured to generate a proposed procedure (180) for connecting the first connector to the second connector of the second device, based on the feature data and the position data.

10. The device according to claim 9, wherein the movable coupling system includes a refueling boom (104A) of an aerial refueling tanker (102A), the first connector includes a refueling connector (106A) of the refueling boom, and the second device includes an aircraft (112A), the second connector includes a refueling port (116A) of the aircraft.

11. The device according to claim 9 or 10, wherein the movable coupling system includes a first docking connector (106B) of a first spacecraft (102B), and the second device includes a second spacecraft (112B), and the second connector includes a second docking connector (116B).

12. The device according to any one of claims 9 to 11, wherein the position data indicates a position in a reference frame associated with the device and independent of the second device, and the position data is generated based on the output of one or more position encoders (126) associated with the movable coupling system.

Images