Autonomous vehicle, program and control method

Abstract

To improve conveyance efficiency.SOLUTION: An autonomous vehicle according to an embodiment of the present disclosure includes: a docking mechanism for docking with a conveyance target; and a control device, wherein the control device detects the conveyance target and evaluates how easily the autonomous vehicle can dock with the detected conveyance target.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] This disclosure relates to an autonomous vehicle.

Background Art

[0002] Conventionally, it is known that an autonomous vehicle such as an automated guided vehicle performs transportation by docking with and towing an object to be transported.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] This disclosure provides a means for improving the efficiency of transportation.

Means for Solving the Problems

[0005] An autonomous vehicle according to an embodiment of this disclosure is An autonomous vehicle that enters the space beneath a transported object, docks with the transported object, and transports the transported object, comprising: a lighting device; a photographing device; and a control device that photographs a marker on the transported object with the photographing device during docking and controls the autonomous vehicle's movement based on the photographed marker, wherein the marker is provided in the space beneath the transported object, and the control device turns on the lighting device during docking so that the marker provided in the space is illuminated and photographed by the photographing device. . [[ID=ac]]

Brief Description of the Drawings

[0006] [Figure 1] A diagram showing an example of a usage scenario of an autonomous vehicle according to an embodiment of this disclosure. [Figure 2] A diagram showing an example of the external configuration of an autonomous vehicle according to an embodiment of this disclosure. [Figure 3] A diagram showing an example of the internal configuration and bottom surface configuration of an autonomous vehicle according to an embodiment of this disclosure. [Figure 4] A diagram showing a state where an autonomous vehicle according to an embodiment of this disclosure docks with a shelf that is an object to be transported. "" [Figure 5] A diagram showing the positional relationship between the casters of a shelf and the docking mechanism of an autonomous vehicle according to an embodiment of this disclosure. [Figure 6] This figure shows an example of the operation of the docking mechanism during docking according to one embodiment of the present disclosure. [Figure 7] This figure shows an example of the hardware configuration of a control device according to one embodiment of the present disclosure. [Figure 8] This figure shows an example of the functional configuration of a control device according to one embodiment of the present disclosure. [Figure 9] This is an example of a flowchart showing the flow of a cleanup process according to one embodiment of the present disclosure. [Figure 10] This is a diagram illustrating the transport sequence according to one embodiment of the present disclosure. [Figure 11] This figure illustrates the docking initiation range and the score calculation target area according to one embodiment of the present disclosure. [Figure 12] This figure illustrates the docking initiation range and the score calculation target area according to one embodiment of the present disclosure. [Figure 13] This is an example of a flowchart showing the flow of wall alignment processing according to one embodiment of the present disclosure. [Figure 14] This figure illustrates a wall surface detection according to one embodiment of the present disclosure. [Figure 15] This diagram illustrates how to align the position and orientation of a transported object with a wall surface according to one embodiment of this disclosure. [Figure 16] This is an example of a flowchart showing the process of turning on lighting according to one embodiment of the present disclosure. [Figure 17] This is an example flowchart showing the flow of the docking retry process according to one embodiment of the present disclosure. [Figure 18] This figure illustrates lateral displacement and angular displacement according to one embodiment of the present disclosure. [Figure 19] This figure illustrates lateral displacement and angular displacement according to one embodiment of the present disclosure. [Modes for carrying out the invention]

[0007] Hereinafter, embodiments of the present disclosure will be described based on the drawings.

[0008] [First Embodiment] [Usage Scenario of Autonomous Vehicle] First, the usage scenario of the autonomous vehicle according to the first embodiment will be described.

[0009] FIG. 1 is a diagram showing an example of a usage scenario of an autonomous vehicle 120 according to an embodiment of the present disclosure. As shown in FIG. 1, the autonomous vehicle 120 is used, for example, in a predetermined space 100 such as a living room of a house, in a scene where the user 110 relaxes on a sofa.

[0010] The usage scenario shown in FIG. 1 is, for example, when the user 110 issues, to the autonomous vehicle 120, · after uttering a wake word, · when uttering "tidy up everything" or "tidy up the shelves in the living room" (that is, when a voice-based conveyance instruction (hereinafter referred to as a voice instruction) is given). This is shown. In this case, the autonomous vehicle 120 detects conveyance targets (in the example of FIG. 1, shelves 130 to 150 with casters) that are outside a predetermined position (in the example of FIG. 1, anchors 170 to 190). Next, the autonomous vehicle 120 evaluates the ease of docking with its own conveyance targets (that is, the shelves 130 to 150), and based on this evaluation, determines the order in which the conveyance targets (that is, the shelves 130 to 150) are to be conveyed to the predetermined position (that is, the anchors 170 to 190). Next, the autonomous vehicle 120 docks with the conveyance targets (that is, the shelves 130 to 150) according to the order and conveys the conveyance targets (that is, the shelves 130 to 150) to the predetermined position (that is, the anchors 170 to 190). Note that the autonomous vehicle 120 may be configured to follow a voice instruction given without a wake word.

[0011] In this way, by using the autonomous vehicle 120, the user 110 can tidy up the conveyance targets (that is, the shelves 130 to 150 with casters) without moving from the sofa just by giving a voice instruction.

[0012] Note that the example in FIG. 1 shows the case where shelves 130 to 150 are located outside of anchors 170 to 190 within the predetermined space 100 when user 110 gives a voice instruction. Further, the example in FIG. 1 shows the case where a trash can 160 is placed as an obstacle on the shortest transport path when transporting shelves 130 to 150 to anchors 170 to 190. In such a case, the autonomous mobile vehicle 120 detects the trash can 160 during the transport of shelves 130 to 150, and avoids collision with the trash can 160 by transporting shelves 130 to 150 along the transport paths shown by the dotted arrows 171, 181, and 191.

[0013] <External Configuration of Autonomous Mobile Vehicle> FIG. 2 is a diagram showing an example of the external configuration of the autonomous mobile vehicle 120 according to an embodiment of the present disclosure.

[0014] As shown in FIG. 2(a), the autonomous mobile vehicle 120 has a rectangular parallelepiped shape as a whole, and the dimensions in the height direction (z-axis direction) and the width direction (x-axis direction) are defined so that it can enter below the lowermost stage of the shelf to be transported. Note that the shape of the autonomous mobile vehicle 120 is not limited to a rectangular parallelepiped.

[0015] On the upper surface 210 of the autonomous mobile vehicle 120, a lock pin 211, which is a member constituting a docking mechanism for docking with the shelf to be transported, is installed. Further, a LiDAR (Light Detection And Ranging) 212 is installed on the upper surface 210 of the autonomous mobile vehicle 120. The LiDAR 212 uses the front-rear direction (y-axis direction) and the width direction (x-axis direction) at the height position of the upper surface 210 of the autonomous mobile vehicle 120 as the measurement range, and by using the measurement result by the LiDAR 212, obstacles and the like within the measurement range can be detected.

[0016] The front 220 of the autonomous vehicle 120 is equipped with a front RGB camera 221 and a Time of Flight (ToF) camera (ToF camera 222). In this embodiment, the front RGB camera 221 is installed above the ToF camera 222, but the installation position of the front RGB camera 221 is not limited to this position.

[0017] The front RGB camera 221 is used when the autonomous vehicle 120 moves in the forward direction, for example, • The shelf to be transported (for example, shelf 130), • Users near the destination (e.g., User 110), installed objects near the destination, Obstacles in the transport path (e.g., trash can 160), It takes pictures of things like this and outputs color images.

[0018] The ToF camera 222 is an example of a sensor that acquires measurement data regarding the three-dimensional position of an object within the measurement range (i.e., the position of the autonomous vehicle 120 in the width, front-rear, and height directions). To avoid the multipath problem, the ToF camera 222 is installed facing upward on the front 220 of the autonomous vehicle 120 to such an extent that the driving surface on which the autonomous vehicle 120 travels (the floor surface 240 shown in Figure 2(b)) is not included in the measurement range. An example of the multipath problem is the deterioration of measurement accuracy due to light emitted from a light source being reflected by other objects after passing through the floor surface 240, and the ToF camera 222 receiving that reflected light. In this embodiment, the upward installation angle θ of the ToF camera 222 on the front 220 of the autonomous vehicle 120 is assumed to be approximately 50 degrees with respect to the floor surface 240.

[0019] Furthermore, when the autonomous vehicle 120 moves in the forward direction, the ToF camera 222 measures and photographs obstacles and other objects within a measurement range of at least the area through which the docked shelf passes (an area equal to the height of the docked shelf × the width of the docked shelf). The ToF camera 222 also outputs the captured distance image (depth image) as 3D position data. In this embodiment, the vertical field of view θv of the ToF camera 222 is assumed to be 70 degrees, and the horizontal field of view θh is assumed to be 90 degrees. In addition to the ToF camera 222, a stereo camera or a monocular camera may be used as a sensor device to acquire 3D position data of objects. In the case of a stereo camera, 3D position data within the measurement range can be calculated from two images captured at the same time. In the case of a monocular camera, 3D position data within the measurement range can be calculated from two images captured at different times, along with the direction and distance of movement of the autonomous vehicle 120.

[0020] A lighting mechanism 223 is installed on at least one of the front and rear of the autonomous vehicle 120. The lighting mechanism 223 lights up when docking with the transport object begins and turns off when docking with the transport object is completed.

[0021] The underside 230 of the autonomous vehicle 120 is fitted with drive wheels 231 and driven wheels 232, which support the autonomous vehicle 120.

[0022] The drive wheels 231 are installed one at a time in the width direction (x-axis direction) (a total of two are installed in the width direction), and each is independently motor-driven, allowing the autonomous vehicle 120 to move in the forward / reverse direction (y-axis direction). In addition, the drive wheels 231 can rotate the autonomous vehicle 120 around the z-axis.

[0023] The driven wheels 232 are installed one at a time in the width direction (x-axis direction) (a total of two in the width direction). Furthermore, each driven wheel 232 is installed on the autonomous vehicle 120 so as to be able to rotate around the z-axis. Note that the installation position and number of driven wheels 232 may differ from those described above.

[0024] <Details of the internal and underside configuration of the autonomous vehicle> Figure 3 shows an example of the internal and underside configuration of an autonomous vehicle 120 according to one embodiment of the present disclosure.

[0025] Figure 3(a) shows the autonomous vehicle 120 with its top cover removed, viewed from directly above. The following explanation will describe the various components that make up the interior of the autonomous vehicle 120, referring to Figure 3(a).

[0026] (a-1) First control board and second control board First, the first control board and the second control board will be described. As shown in Figure 3(a), the autonomous vehicle 120 has a first control board 311 and a second control board 312. In this embodiment, the first control board 311 controls, for example, electronic devices, and the second control board 312 controls, for example, drive devices. However, the division of roles between the first control board 311 and the second control board 312 is not limited to this.

[0027] In the example shown in Figure 3(a), the first control board 311 and the second control board 312 are shown to be installed separately, but the first control board 311 and the second control board 312 may be installed integrally as a single board. Regardless of whether the first control board 311 and the second control board 312 are installed separately or integrally, in this embodiment, a device that has both the functions of the first control board 311 and the functions of the second control board 312 is referred to as the control device 310.

[0028] (a-2) Docking mechanism Next, the docking mechanism will be described. As shown in Figure 3(a), the autonomous vehicle 120 has a solenoid-type lock pin 211 and a photoreflector 330 as a docking mechanism for docking with the shelf to be transported. In this embodiment, the docking mechanism uses a solenoid-type lock pin, but the raising and lowering of the lock pin may be performed by an electromagnetic actuator other than a solenoid, or by other actuators such as a rack and pinion mechanism, a trapezoidal screw mechanism, or a pneumatic drive mechanism.

[0029] In this embodiment, the solenoid-type lock pin 211 is positioned at the center of each drive wheel 231 in the width direction (x-axis direction), and is located on the rotation axis of the drive wheel 231 (see the dashed line in Figures 3(a) and (b)).

[0030] The solenoid-type lock pin 211 has a built-in compression coil spring. When the solenoid is turned ON, the lock pin 211 is attracted, and the compression coil spring compresses. Conversely, when the solenoid is turned OFF, the solenoid-type lock pin 211 protrudes upward (in the z-axis direction, towards the foreground of the paper in Figure 3(a)) due to the compressive force of the compression coil spring. The ON / OFF state of the solenoid is controlled by the control device 310.

[0031] The photoreflector 330 outputs a signal to determine whether or not the autonomous vehicle 120 can extend the lock pin 211 into the hole (details described later) of the lock guide attached to the shelf to be transported when the vehicle enters the area below the bottom shelf of the shelf to be transported.

[0032] The autonomous vehicle 120 turns off the solenoid when it determines, based on the signal output from the photoreflector 330, that it is possible to extend the lock pin 211. In this embodiment, a photoreflector is used to detect the facing state between the lock pin 211 and the hole in the lock guide, but this detection may be performed by a method other than a photoreflector. Examples of methods other than a photoreflector include methods using a camera, a physical switch, a magnetic sensor, an ultrasonic sensor, etc.

[0033] As a result, the lock pin 211 protrudes toward the hole in the lock guide, and the protruding lock pin 211 is inserted into the hole in the lock guide. This completes the docking between the autonomous vehicle 120 and the shelf to be transported.

[0034] As mentioned above, the solenoid-type lock pins 211 are positioned at the center of each drive wheel 231 in the width direction (x-axis direction) (they are symmetrical in the width direction). Therefore, when the autonomous vehicle 120 enters the area below the bottom shelf of the shelf to be transported, it can enter in either the forward or reverse direction.

[0035] On the other hand, when the autonomous vehicle 120 is docked with the shelf to be transported, turning on the solenoid causes the lock pin 211 to be attracted, which releases the docking between the autonomous vehicle 120 and the shelf to be transported.

[0036] (a-3) Various input / output devices Next, various input / output devices will be described. As shown in Figure 3(a), the autonomous vehicle 120 has, in addition to the LiDAR 212, front RGB camera 221, and ToF camera 222 mentioned above, a rear RGB camera 320, microphones 301-304, and speakers 305-306 as various input / output devices.

[0037] The installation location, orientation, measurement range, and measurement target of the LiDAR 212, front RGB camera 221, and ToF camera 222 have already been explained, so the explanation will be omitted here.

[0038] The rear RGB camera 320 is used when the autonomous vehicle 120 moves in the reverse direction, for example, • The shelf to be transported (for example, shelf 130), • Obstacles around the shelf to be transported, It takes pictures of things like this and outputs color images.

[0039] Microphones 301-304 are examples of sound input devices and are installed at four corners of the autonomous vehicle 120 (two on the front and two on the rear), respectively, to detect sound from each direction. By installing microphones 301-304 at the four corners of the autonomous vehicle 120 in this way, it is possible to determine which direction the user 110 who gave the voice command is in relative to the current position and orientation of the autonomous vehicle 120, and to estimate the position of the user 110.

[0040] Speakers 305-306 are an example of an audio output device and output audio toward the side of the autonomous vehicle 120. Speakers 305-306 output audio to confirm the content of the task recognized by the autonomous vehicle 120 in response to an audio command from the user 110.

[0041] Figure 3(b) shows the autonomous vehicle 120 viewed from below. The following explanation will describe the various parts that make up the underside of the autonomous vehicle 120, referring to Figure 3(b).

[0042] (b-1) Drive wheels First, let's explain the drive wheels 231. As shown in Figure 3(b), the autonomous vehicle 120 has drive wheels 231, one on each side in the width direction (x-axis direction). As described above, each drive wheel 231 is independently motor-driven, allowing the autonomous vehicle 120 to move forward / backward (y-axis direction) or to rotate around the z-axis.

[0043] Specifically, by rotating both drive wheels 231 forward, the autonomous vehicle 120 can be moved forward, and by rotating both drive wheels 231 backward, the autonomous vehicle 120 can be moved backward. In addition, by rotating one drive wheel 231 forward and the other backward, the autonomous vehicle 120 can be turned.

[0044] As mentioned above, the rotation axes of one drive wheel 231 and the other drive wheel 231 are formed coaxially, and the solenoid-type lock pin 211 is installed coaxially at the center between the one drive wheel 231 and the other drive wheel 231. Therefore, when one drive wheel 231 is rotated forward and the other drive wheel 231 is rotated backward, the autonomous vehicle 120 will rotate around the solenoid-type lock pin 211.

[0045] (b-2) Driven wheel Next, the driven wheels 232 will be described. As shown in Figure 3(b), the autonomous vehicle 120 has driven wheels 232, one on each side in the width direction (x-axis direction). As described above, each driven wheel 232 is installed so as to be able to pivot around the z-axis. Therefore, for example, if the autonomous vehicle 120 moves forward or backward and then turns, the driven wheels 232 can immediately adjust their orientation to follow the turning direction. Also, for example, if the autonomous vehicle 120 moves forward or backward after turning, the driven wheels 232 can immediately adjust their orientation to follow the forward or backward direction.

[0046] <Overview of docking> Figure 4 shows how an autonomous vehicle 120 according to one embodiment of this disclosure docks with a shelf 130 to be transported. The same procedure applies to shelves 140-150.

[0047] Figure 4(a) shows the autonomous vehicle 120 in the position just before it reverses toward the shelf 130 to be transported and docks with the shelf 130. As shown in Figure 4(a), the shelf 130 has three shelves, and frame guides 410 and 420 are mounted on the underside of the bottom shelf 400 at intervals corresponding to the width of the autonomous vehicle 120 and approximately parallel to each other. This defines the direction in which the autonomous vehicle 120 enters when it approaches the underside of the bottom shelf 400 of the shelf 130 to be transported. In addition, the frame guides 410 and 420 function as guides in the width direction when the autonomous vehicle 120 transports the shelf 130 to be transported, preventing the shelf 130 from shifting in the width direction relative to the autonomous vehicle 120. The autonomous vehicle 120 may also advance toward the shelf 130 to be transported and dock with it.

[0048] Furthermore, casters 431-434 are mounted at the base of the shelf 130 so as to be able to rotate. This allows the autonomous vehicle 120 to easily transport the docked shelf 130.

[0049] Figure 4(b) shows the autonomous vehicle 120 after docking with the shelf 130 to be transported. As shown in Figure 4(b), even when docked with the shelf 130, the front 220 of the autonomous vehicle 120 is not covered by any of the shelves 130 (the front 220 protrudes further forward than each of the shelves 130). Therefore, when the autonomous vehicle 120 transports the shelf 130, the measurement range of the front RGB camera 221 is not obstructed by any of the shelves 130.

[0050] Similarly, with respect to the ToF camera 222, when the autonomous vehicle 120 transports the shelf 130, the measurement range (vertical field of view θv, horizontal field of view θh) is not obstructed by any of the shelves 130.

[0051] On the other hand, when the autonomous vehicle 120 is docked to the shelf 130, the LiDAR212's measurement range in the front and rear at the height of the autonomous vehicle 120 is not obstructed. However, the measurement range in the width direction may be obstructed by the frame guides 410 and 420.

[0052] Therefore, the frame guides 410 and 420 of the shelf 130 are provided with openings 411 and 421 to reduce the percentage of the widthwise measurement range of the LiDAR 212 that is obstructed. As a result, when the autonomous vehicle 120 transports the shelf 130, the LiDAR 212 can measure the forward, backward, and widthwise measurement ranges at the height of the autonomous vehicle 120 without being obstructed by the shelf 130.

[0053] Although not shown in Figure 4(b), microphones 301 and 302 (microphones installed on the front) are also positioned to protrude forward from each shelf of the shelf 130 when the autonomous vehicle 120 is docked to the shelf 130. Therefore, when the autonomous vehicle 120 transports the shelf 130, the detection range of the front microphones 301 and 302 is not obstructed by any of the shelves of the shelf 130.

[0054] <Relationship between the position of the shelf's casters and the position of the autonomous vehicle's docking mechanism> Figure 5 shows the positional relationship between the casters of a shelf and the docking mechanism of an autonomous vehicle according to one embodiment of this disclosure. The positional relationship between the casters 431-434, which are rotatably mounted on the shelf 130, and the docking mechanism of the autonomous vehicle 120 will be explained.

[0055] Of these, Figure 5(a) shows the autonomous vehicle 120 docked to the shelf 130, viewed from directly above the bottom shelf 400 of the shelf 130. However, for the sake of explanation, only the outer frame of the bottom shelf 400 is shown. Figure 5(b) shows the autonomous vehicle 120 docked to the shelf 130, viewed from the direction of the front 220 of the autonomous vehicle 120.

[0056] As shown in Figure 5(a), the four casters 431-434 of the shelf 130 are swivelably mounted at the corners of the bottom shelf 400. The swivel ranges of the four casters 431-434 are as indicated by reference numerals 501-504, and the center position of the swivel range indicated by reference numerals 501-504 is the swivel center position of the casters 431-434.

[0057] Furthermore, as shown in Figure 5(a), a lock guide 510 is attached to the underside of the lowest shelf 400 of the shelf 130, and the lock guide 510 is provided with a hole 511 into which a solenoid-type lock pin 211 is inserted when it protrudes.

[0058] The lock guide 510 is assumed to have a surface made of, for example, white material. This is to make it easier to determine whether or not it is possible to insert the lock pin 211 into the hole 511 of the lock guide 510, based on the signal output from the photoreflector 330.

[0059] By inserting the lock pin 211 into the hole 511 of the lock guide 510, it is possible to prevent the shelf 130 from shifting in the forward or backward direction relative to the autonomous vehicle 120 when the autonomous vehicle 120 transports the shelf 130. In this embodiment, to clearly indicate whether the lock pin 211 is in a protruding state or not, the lock pin 211 in a protruding state is shown in black in the drawing.

[0060] Here, the hole 511 of the lock guide 510 is configured such that its center position coincides with the center position of each of the four casters 431 to 434 of the shelf 130 (see the dashed and dotted lines in Figures 5(a) and (b)). Therefore, when the autonomous vehicle 120 is docked to the shelf 130, the center position of the lock pin 211 is also the center position of each of the four casters 431 to 434 of the shelf 130.

[0061] As described above, since the autonomous vehicle 120 is configured to rotate around the lock pin 211, when the autonomous vehicle 120 rotates, the shelf 130 will rotate around the center position relative to each of the four casters 431 to 434's rotation centers. In other words, when the autonomous vehicle 120 rotates, the rotation range of the shelf 130 is the range indicated by reference numeral 520 (the autonomous vehicle 120 can rotate the shelf 130 within the minimum rotation range).

[0062] <Example of docking mechanism operation> Figure 6 shows an example of the operation of the docking mechanism during docking. An example of the operation of the docking mechanism when the autonomous vehicle 120 docks with the shelf 130 (here, an example of the operation when the autonomous vehicle 120 reverses and docks with the shelf 130) will be explained. Similar to Figure 5(a), Figure 6 shows a view from directly above the bottom shelf 400 of the shelf 130. However, for the sake of explanation, only the outer frame of the bottom shelf 400 is shown.

[0063] Figure 6(a) shows how the autonomous vehicle 120 searches for the shelf 130 after moving to a position near the shelf 130 to be transported, based on a color image captured by the front RGB camera 221. The method for searching for the shelf 130 is arbitrary; for example, the shelf 130 may be searched for by performing pattern matching based on pre-calculated shape features of the shelf 130 and shape features of the shelf 130 extracted from the color image. Alternatively, the shelf 130 may be searched for by extracting markers that have been pre-applied to the shelf 130 to identify it from the color image. In this case, the autonomous vehicle 120 may obtain information about the size of the shelf 130 (width, depth, and height information) contained in the markers by image analysis. This information can be used to control how far away the autonomous vehicle 120 should be from obstacles when transporting the shelf 130. Alternatively, the shelf 130 may be searched for by performing instance segmentation on the color image using a deep learning-based object recognition model.

[0064] Furthermore, Figure 6(a) shows that when the autonomous vehicle 120 is able to locate the shelf 130, it recognizes the position and orientation of the shelf 130 (the orientation of the frame guides 410 and 420), moves to the docking start position (the position where docking begins) which is set according to the position and orientation of the shelf 130, and rotates 180 degrees relative to the direction of entry when docking.

[0065] The autonomous vehicle 120, having rotated 180 degrees, begins docking based on the color image captured by the rear RGB camera 320.

[0066] Specifically, by turning on the solenoid, the lock pin 211 is attracted, and then it begins to move in the reversing direction, entering the space between the frame guide 410 and the frame guide 420, which is below the lowest stage 400.

[0067] Figure 6(b) shows the autonomous vehicle 120 moving backward and entering the space between the frame guide 410 and the frame guide 420. While entering, the autonomous vehicle 120 monitors the measurement results of the photoreflector 330 and determines whether it is possible to insert the lock pin 211 into the hole 511 of the lock guide 510.

[0068] Figure 6(c) shows the state in which the lock pin 211 can be inserted into the hole 511 of the lock guide 510. In the state shown in Figure 6(c), the autonomous vehicle 120 turns off the solenoid, causing the lock pin 211 to protrude and be inserted into the hole 511. This completes the docking of the autonomous vehicle 120 to the shelf 130.

[0069] The docking operation described above using Figure 6 is an example in which the autonomous vehicle 120 reverses and docks with the shelf 130 to be transported. However, the autonomous vehicle 120 may also operate to dock with the shelf 130 by moving forward. In other words, after the autonomous vehicle 120 recognizes the shelf 130 based on the color image captured by the front RGB camera 221, it may move to a docking start position set according to the position and orientation of the shelf 130, and then start docking by moving forward based on the color image captured by the front RGB camera 221.

[0070] <Control device hardware configuration> Figure 7 shows an example of the hardware configuration of the control device 310. The control device 310 has the following components: a processor 701, a main memory (memory) 702, an auxiliary storage device 703, a network interface 704, and a device interface 705. The control device 310 is implemented as a computer in which these components are connected via a bus 706. In the example in Figure 7, the control device 310 is shown as having one of each component, but the control device 310 may have multiple identical components.

[0071] The various calculations performed by the control device 310 may be executed in parallel using one or more processors. Alternatively, the various calculations may be distributed to multiple arithmetic cores within the processor 701 and executed in parallel. Furthermore, some or all of the processing and means of this disclosure may be executed by an external device 730 (at least one of a processor and a storage device) located on a cloud that can communicate with the control device 310 via the network interface 704. Thus, the control device 310 may take the form of parallel computing using one or more computers.

[0072] The processor 701 may be an electronic circuit (processing circuit, processing circuitry, CPU, GPU, FPGA, or ASIC, etc.). Alternatively, the processor 701 may be a semiconductor device including a dedicated processing circuit. Furthermore, the processor 701 is not limited to an electronic circuit using electronic logic elements, but may be implemented using an optical circuit with optical logic elements. The processor 701 may also include computational functions based on quantum computing.

[0073] The processor 701 performs various calculations based on various data and instructions input from the various devices and components of the control unit 310, and outputs the calculation results and control signals to the respective devices and components. The processor 701 controls the various components of the control unit 310 by executing the OS (Operating System) and applications.

[0074] Furthermore, the processor 701 may refer to one or more electronic circuits arranged on a single chip, or one or more electronic circuits arranged on two or more chips or devices. When multiple electronic circuits are used, each electronic circuit may communicate by wire or wireless means.

[0075] The main memory 702 is a storage device that stores instructions executed by the processor 701 and various data, and the various data stored in the main memory 702 is read by the processor 701. The auxiliary storage device 703 is a storage device other than the main memory 702. These storage devices refer to any electronic component capable of storing various data, and may be semiconductor memory. The semiconductor memory may be either volatile memory or non-volatile memory. The storage device for saving various data in the control device 310 may be implemented by the main memory 702 or the auxiliary storage device 703, or by the built-in memory of the processor 701.

[0076] Furthermore, multiple processors 701 may be connected to (combined with) one main memory 702, or a single processor 701 may be connected to it. Alternatively, multiple main memory 702 may be connected to (combined with) one processor 701. If the control device 310 consists of at least one main memory 702 and multiple processors 701 connected to (combined with) this at least one main memory 702, it may include a configuration in which at least one of the multiple processors 701 is connected to (combined with) at least one main memory 702. This configuration may also be realized by the main memory 702 and processors 701 included in multiple control devices 310. Furthermore, it may include a configuration in which the main memory 702 is integrated with the processor (for example, a cache memory including an L1 cache and an L2 cache).

[0077] The network interface 704 is an interface for connecting to the communication network 740 wirelessly or via a wired connection. The network interface 704 uses an appropriate interface, such as one conforming to existing communication standards. The network interface 704 may exchange various types of data with external devices 730 connected via the communication network 740. The communication network 740 may be a WAN (Wide Area Network), LAN (Local Area Network), PAN (Personal Area Network), or a combination thereof, as long as it allows for information exchange between a computer and other external devices 730. Examples of WANs include the Internet, examples of LANs include IEEE 802.11 and Ethernet, and examples of PANs include Bluetooth® and NFC (Near Field Communication).

[0078] The device interface 705 is an interface such as USB that connects directly to the external device 750.

[0079] The external device 750 is a device connected to the computer. The external device 750 may, for example, be an input device. In this embodiment, the input device is an electronic device such as a camera (front RGB camera 221, ToF camera 222, rear RGB camera 320), a microphone (microphones 301-304), or various sensors (photo reflector 330), and provides the acquired information to the computer.

[0080] Furthermore, the external device 750 may, for example, be an output device. In this embodiment, the output device may be a display device such as an LCD (Liquid Crystal Display), CRT (Cathode Ray Tube), PDP (Plasma Display Panel), or organic EL (Electro Luminescence) panel, or it may be a speaker (speakers 305-306) that outputs sound, etc. It may also be a drive device such as various drive devices (motors, solenoids).

[0081] Furthermore, the external device 750 may be a storage device (memory). For example, the external device 750 may be network storage, or it may be storage such as an HDD.

[0082] Furthermore, the external device 750 may be a device that has some of the functions of the components of the control device 310. In other words, the computer may transmit or receive some or all of the processing results of the external device 750.

[0083] <Function Block> Figure 8 shows an example of the functional configuration of a control device 310 according to one embodiment of the present disclosure. The control device 310 includes a voice instruction acquisition unit 810, a transport target identification unit 820, a transport destination identification unit 830, a transport order determination unit 840, a docking control unit 850, a transport control unit 860, a lighting control unit 870, and an environmental map storage unit 880. Furthermore, the control device 310 functions as the voice instruction acquisition unit 810, the transport target identification unit 820, the transport destination identification unit 830, the transport order determination unit 840, the docking control unit 850, the transport control unit 860, and the lighting control unit 870 by executing a program. Each of these will be described below.

[0084] The voice command acquisition unit 810 recognizes the wake word spoken by the user 110 from the sound data detected by microphones 301 to 304 and acquires the voice command that follows the wake word. The voice command acquisition unit 810 also notifies the transport target identification unit 820 and the transport destination identification unit 830 of the acquired voice command.

[0085] The transport target identification unit 820 analyzes the voice instruction notified by the voice instruction acquisition unit 810 and detects and identifies transport targets that are located outside of a predetermined location (for example, a predetermined location such as the position of the anchor 170). Specifically, the transport target identification unit 820 detects transport targets using color images captured by the RGB camera.

[0086] The destination identification unit 830 identifies the destination location of the transported object (for example, a predetermined location such as the location of anchor 170). The autonomous vehicle 120 may transport each transported object to a predetermined location (for example, shelf 130 to anchor 170, shelf 140 to anchor 180, shelf 150 to anchor 190), or to a predetermined location relative to the transported object (for example, any of anchors 170 to 190), or to a location along a wall.

[0087] The transport order determination unit 840 evaluates the ease of docking with the transport object and, based on that evaluation (i.e., ease of docking), determines the order in which the transport object is transported to a predetermined position.

[0088] The docking control unit 850 controls the autonomous vehicle 120 to either dock with or undock with the transport object. The docking control unit 850 can control docking with the transport object based on an evaluation of the ease of docking with that object. For example, the docking control unit 850 can set the starting position for docking with the transport object based on an evaluation of the ease of docking with that object. For example, if the evaluation of the ease of docking with the transport object is higher than a threshold (docking is not difficult), the docking control unit 850 decides to perform docking with that transport object, and if the evaluation is lower than the threshold (docking is difficult), it can choose not to perform docking with that transport object, cancel it, or postpone it. In addition, if the docking control unit 850 detects a displacement (lateral displacement and angular displacement) of the autonomous vehicle 120 relative to the transport object during a docking attempt, it can perform a docking retry process.

[0089] The transport control unit 860 controls the autonomous vehicle 120 to move. While the autonomous vehicle 120 is moving, the transport control unit 860 refers to the measurement results from the LiDAR 212, the color image from the front RGB camera 221, and the distance image from the ToF camera 222. The transport control unit 860 then calculates the current position of the autonomous vehicle 120 and, if it detects an obstacle on the transport path, controls the vehicle to avoid a collision.

[0090] The lighting control unit 870 controls the lighting mechanism 223 to turn it on or off.

[0091] The environmental map storage unit 880 stores an environmental map that describes information about the surrounding environment, such as the arrangement of objects within a predetermined space 100, which is the area in which the autonomous vehicle 120 travels. The environmental map stores the probability that there are no obstacles at each location on the environmental map (or a label corresponding to the probability that there are no obstacles (for example, "high" or "low" for the probability that there are no obstacles)). If the environmental map is a grid map, the environmental map stores the probability that there are no obstacles at each grid (or a label corresponding to the probability that there are no obstacles) for each grid.

[0092] <Processing method> Figure 9 is an example of a flowchart showing the flow of a cleanup process according to one embodiment of the present disclosure.

[0093] In step 1 (S1), the transport target identification unit 820 detects a transport target that is located outside of a predetermined position (for example, a predetermined position such as the position of the anchor 170).

[0094] For example, the autonomous vehicle 120 moves to a position within the predetermined space 100 (for example, a pre-set position, a position determined by the autonomous vehicle 120, a position instructed via the external device 730, etc.) in accordance with the voice instructions of the user 110, or without voice instructions from the user 110 (for example, when pre-set, when there are no other tasks, or when instructions are received via the external device 730, etc.). Next, the autonomous vehicle 120 uses the color image captured by the RGB camera to detect a transport object that can be docked with the autonomous vehicle 120. The autonomous vehicle 120 may also choose to clear away transport objects that have already been recognized (for example, transport objects recognized while performing another task).

[0095] The autonomous vehicle 120 may move to a predetermined location (for example, a predetermined location such as the location of the anchor 170) to determine whether or not there is a transport object, and then search for transport objects that are not in the predetermined location.

[0096] In step 2 (S2), the transport order determination unit 840 determines the order in which the transport objects are transported to predetermined positions based on the ease of docking with the transport objects.

[0097] In step 3 (S3), the transport order determination unit 840 controls the transport order determination unit to dock with the transport object and transport the transport object to a predetermined position according to the order determined in S2.

[0098] For example, the autonomous vehicle 120 may transport each item to a predetermined location (for example, shelf 130 to anchor 170, shelf 140 to anchor 180, shelf 150 to anchor 190), or to a predetermined location relative to the item (for example, any of anchors 170 to 190), or to a location along a wall.

[0099] <Transportation sequence based on ease of docking> The following describes the process of determining the order of transport based on the ease of docking in step 2 (S2) described above. The transport order determination unit 840 refers to the environmental map stored in the environmental map storage unit 880 (specifically, for each position on the environmental map, the probability that there is no obstacle at that position (or a label corresponding to the probability that there is no obstacle)), measurement results from the LiDAR 212, color images from the front RGB camera 221, distance images from the ToF camera 222, etc., to set a score calculation target area for each surface that the autonomous vehicle 120 enters to transport object (for example, the front of the transport object, the rear of the transport object, the side of the transport object), and calculates a score for the ease of docking of each transport object. In other words, the transport order determination unit 840 evaluates the ease of docking of each transport object. First, an overview of the transport order based on the ease of docking will be described with reference to Figure 10, and then the docking start range and the score calculation target area will be described with reference to Figures 11 and 12.

[0100] Figure 10 is a diagram illustrating a transport sequence according to one embodiment of the present disclosure. The autonomous vehicle 120 transports the transport targets S1, S2, S3, and S4 to predetermined positions P1, P2, P3, and P4. The autonomous vehicle 120 can start docking from the position marked with a circle, but it cannot start docking from the position marked with an X. The autonomous vehicle 120 transports the transport targets S1, S2, S3, and S4 to predetermined positions P1, P2, P3, and P4, respectively, in an order determined based on the ease of docking with each of the transport targets S1, S2, S3, and S4.

[0101] Figure 11 is a diagram illustrating the docking initiation range and the score calculation area according to one embodiment of the present disclosure. The shaded area in Figure 11 indicates the range in which the autonomous vehicle 120 can initiate docking with the transported object (shelf 130 in the example of Figure 11) (docking initiation range). The set of grids in Figure 11 indicates the area in which a score for the ease of docking with the transported object (shelf 130 in the example of Figure 11) is calculated (score calculation area). This score calculation area is set around the transported object (within a predetermined distance range from the transported object), and one grid corresponds to one position relative to the transported object, where the score for the ease of docking is calculated. The docking initiation range corresponds to the area of ​​grids with high docking ease scores. Since Figure 11 shows an example in which there are no obstacles around the transported object, the scores of each grid in the score calculation area are high, and the docking initiation range substantially coincides with the score calculation area.

[0102] Figure 12 is a diagram illustrating the docking initiation range and the score calculation area according to one embodiment of the present disclosure. Similar to Figure 11, the shaded area in Figure 12 indicates the range in which the autonomous vehicle 120 can initiate docking with the transported object (shelf 130 in the example of Figure 12) (docking initiation range). The set of grids in Figure 12 indicates the area in which a score for the ease of docking with the transported object (shelf 130 in the example of Figure 12) is calculated (score calculation area). Since Figure 12 shows an example in which obstacles to docking (walls, sofas, side tables) exist around the transported object, the scores of some grids in the score calculation area are high, and the docking initiation range roughly coincides with that part of the score calculation area.

[0103] Thus, the score calculation target area is set by the transport order determination unit 840 based on the position and orientation of the transport target, and corresponds to the maximum range of the docking initiation range (i.e., the range of docking in an area without obstacles).

[0104] [Score for each grid] First, the transport order determination unit 840 calculates a score for the ease of docking for each grid (each individual square) in the score calculation area set around the transported object, assuming docking is started from that grid position (in other words, if that grid position is set as the docking start position). The docking ease score for a given grid is calculated based on the probability of obstacle absence (the probability that no obstacles exist) or the corresponding label at each position along the path (straight line, curve, or a combination thereof) connecting that grid position to the transported object's position. The probability of obstacle absence or the label at each position is obtained from the environment map. For example, the transport order determination unit 840 refers to the environment map and calculates the average or minimum probability of obstacle absence at each position along the path connecting that grid position to the transported object's position as the docking ease score for each grid. In other words, the ease of docking according to one embodiment of this disclosure can be evaluated based on the grid score, which is an index that can be calculated based on an environmental map and represents the likelihood that there are no obstacles on the path (e.g., the shortest path) that the autonomous vehicle 120 travels from that grid to the transported object. In this way, the transport order determination unit 840 evaluates the ease of docking for each grid. The transport order determination unit 840 may also adjust the score according to the deviation of the autonomous vehicle 120 from the transported object (e.g., angular deviation, lateral deviation, etc.). [Score for each side] Next, the transport order determination unit 840 determines representative values ​​such as the highest score, average score, etc., of the score calculation area in each grid for each surface that enters the transport object (for example, the front, rear, and side surfaces of the transport object) as the score for the score calculation area of ​​that surface. In other words, the transport order determination unit 840 evaluates the ease of docking for each surface. [Score of each transported item] Next, the transport order determination unit 840 determines representative values ​​such as the highest score and average score of each surface of the transport target as the score of the transport target. In other words, the transport order determination unit 840 evaluates the ease of docking with the transport target. It can also be said that evaluating the ease of docking with the transport target is itself the evaluation of the ease of docking with the transport target by evaluating the ease of docking with the transport target for at least one grid of the score calculation target area set for the transport target. Furthermore, in one embodiment of this disclosure, the ease of docking with the transport target can be said to be the likelihood that there are no obstacles on the path (e.g., the shortest path) that the autonomous vehicle 120 travels to the transport target, and evaluating this likelihood is the evaluation of the ease of docking. In addition, the ease of docking described above is evaluated based on the probability that there are no obstacles on the path, but it may also be evaluated based on the probability that there are obstacles on the path, or based on labels corresponding to those probabilities. That is, in one embodiment of this disclosure, the ease of docking is evaluated by an index calculated based on the path to the transport target and information (existence information) regarding the presence of obstacles around the transport target. This presence information may include information about the location of obstacles. Furthermore, the ease of docking may be evaluated by an index calculated based on the presence information of obstacles around the transport target, without considering the path to the transport target. In this case, since the ease of docking is evaluated from the perspective of whether the autonomous vehicle 120 can approach the transport target, the ease of docking conceptually includes the likelihood that there are no obstacles around the transport target. [Transportation Order] Next, the transport order determination unit 840 compares the scores of each transport target and determines the order in which they are transported, starting with the transport targets that are easiest to dock. In other words, the transport order determination unit 840 determines the transport order of each transport target based on an evaluation of the ease of docking of each transport target. To put it another way, the transport order determination unit 840 determines the order in which each transport target will be docked. Furthermore, the transport order determination unit 840 can determine the order in which to transport targets to a predetermined position based not only on the ease of docking with the transport target at the present time, but also on the ease of docking with the transport target after other transport targets have been transported (for example, after transport targets S1 and S2 have been transported in Figure 10). In addition, if there is an obstacle while moving towards a transport target for docking, the transport order determination unit 840 can switch to transporting the next transport target in the order.

[0105] Subsequently, the autonomous vehicle 120 (docking control unit 850) sets the docking start position for each transported object to the grid position with the highest score within the score calculation area. In other words, the autonomous vehicle 120 (docking control unit 850) sets the docking start position for each transported object based on an evaluation of the ease of docking for each grid in the score calculation area. Then, for each transported object, the autonomous vehicle 120 (docking control unit 850) can move to the set docking start position and begin docking to that object from that position. It should be noted that evaluating the ease of docking conceptually includes evaluating the difficulty of docking.

[0106] Furthermore, the following method may be used as an alternative method for evaluating the ease of docking for each transported object. For example, the transport order determination unit 840 may evaluate the likelihood that there are no obstacles on the path from the location of the transported object (in other words, the location where it will be docked) to the destination location (in other words, the location where it will be undocked) for each transported object to be docked. In this case, the transport order determination unit 840 may set each grid on the path as a score calculation area and evaluate the ease of docking based on the score of each grid. That is, the ease of docking may conceptually include the likelihood that there are no obstacles on the path that the autonomous vehicle 120 travels from the location of the transported object to the destination location. Based on this evaluation for each transported object, the transport order determination unit 840 may determine the transport order of each transported object in order of decreasing likelihood of obstacle absence. As described above, the transport order determination unit 840 may evaluate each transport object based at least on the position of each transport object and the environmental map (information on the presence of obstacles in the area where the autonomous vehicle 120 travels), and determine the transport order of each transport object. The transport order determination unit 840 may also evaluate each transport object based on the position of the destination for each transport object.

[0107] <Wall alignment> Figure 13 is an example of a flowchart showing the flow of wall alignment processing according to one embodiment of the present disclosure. The transport control unit 860 may perform wall alignment processing when clearing away the transported objects, or when processing other than clearing away the transported objects. Furthermore, the transport control unit 860 may set whether or not to perform wall alignment processing according to voice instructions from the user 110 or instructions from the user 110 via an external device 730.

[0108] In step 11 (S11), the transport control unit 860 detects the wall surface.

[0109] In step 12 (S12), the transport control unit 860 aligns at least one of the position and orientation of the transport object with the wall surface detected in S11.

[0110] In step 13 (S13), the transport control unit 860 instructs the docking control unit 850 to undock.

[0111] In this way, the transport control unit 860 can detect a wall at a predetermined position and release the docking by aligning the position or orientation of the transported object with the wall.

[0112] Figure 14 is a diagram illustrating wall detection (step 11(S11)) according to one embodiment of the present disclosure. The autonomous vehicle 120 transports the object to be transported to the wall closest to the front of the autonomous vehicle 120 (i.e., the surface perpendicular to the orientation of the autonomous vehicle 120).

[0113] First, the autonomous vehicle 120 can extract areas assumed to be walls from the point cloud acquired by LiDAR using the Split & Merge method (ST Pfister, SI Roumeliotis, and JW Burdick, "Weighted line fitting algorithms for mobile robot map building and efficient data representation" in Proc. IEEE Intl. Conf. on Robotics and Automation (ICRA), Taipei, Taiwan, 14-19 Sept., 2003). However, the method for extracting areas assumed to be walls is not limited to this method, and other known methods may be used.

[0114] In step 101 (S101), the autonomous vehicle 120 (transport control unit 860) extracts the point where a perpendicular line from the autonomous vehicle 120 to the wall intersects with the wall (the square mark in Figure 14). In other words, the autonomous vehicle 120 identifies the foot of the perpendicular line drawn from the autonomous vehicle 120 to the area assumed to be the wall surface, or to the area extended laterally from that area, as the point of intersection.

[0115] In step 102 (S102), the autonomous vehicle 120 (transport control unit 860) designates the intersection points of each perpendicular line in S101 with each wall (or the endpoints if the intersection point is not on a wall (as in wall 2 in Figure 14)) as candidate points.

[0116] In step 103 (S103), the autonomous vehicle 120 (transport control unit 860) determines a candidate point (in Figure 14, a candidate point on wall 1) from among candidate points (in Figure 14, candidate points on wall 1 and wall 2 located within a circle centered on the autonomous vehicle 120) whose distance from the autonomous vehicle 120 is less than or equal to a threshold, such that the distance between the "candidate point" and the "point obtained by projecting the candidate point onto the x-axis (the axis indicating the direction of the autonomous vehicle 120)" (i.e., the length of the perpendicular line drawn from the candidate point to the x-axis) is minimized. Note that the method of determining the candidate point is not limited to this; for example, the autonomous vehicle 120 may determine a candidate point that minimizes the distance between the "candidate point" and the autonomous vehicle 120. The autonomous vehicle 120 moves to the determined candidate point as the target position.

[0117] Figure 15 illustrates the method of aligning the position and orientation of the transported object with the wall surface (step 12(S12)) and undocking (step 13(S13)) according to one embodiment of the present disclosure, which is performed with respect to the determined candidate point, i.e., the wall closest to the front of the autonomous vehicle 120. In this case, the autonomous vehicle 120 is in the process of transporting the transported object.

[0118] In step 1001 (S1001), the autonomous vehicle 120 measures the angle between the autonomous vehicle 120 and the wall, and the distance between the autonomous vehicle 120 and the wall.

[0119] In step 1002 (S1002), the autonomous vehicle 120 uses LiDAR to rotate so that it is perpendicular to the wall.

[0120] In step 1003 (S1003), the autonomous vehicle 120 moves backward towards the wall (wall surface) while measuring the distance to the wall until the distance to the wall falls below a threshold. The autonomous vehicle 120 also detects a marker attached to the transported object (for example, a barcode attached to the underside of the bottom shelf 400 of the shelf 130) when docking with the transported object, analyzes the information contained in the detected marker to determine the size of the transported object to which the marker is attached, and moves a distance corresponding to the determined size of the transported object. For example, based on the distance to the wall and the size of the transported object in the front-to-back direction of the autonomous vehicle (i.e., the depth of the transported object), the autonomous vehicle 120 can transport the transported object to a position that is a distance away from the wall corresponding to its size. In this way, by transporting the transported object to the wall while considering the depth of the transported object, which may vary depending on the type of transported object, the autonomous vehicle 120 can position the transported object precisely against the wall without touching it.

[0121] In step 1004 (S1004), the autonomous vehicle 120 fine-tunes the angle between the autonomous vehicle 120 and the wall (i.e., the orientation of the transported object relative to the wall surface) using the dead reckoning method.

[0122] In step 1005 (S1005), the autonomous vehicle 120 undocking and terminates the process.

[0123] <Lights illuminate when docking> Figure 16 is an example of a flowchart showing the flow of the lighting process according to one embodiment of this disclosure. The lighting process may be performed during the cleanup process of the transported objects, or during processes other than the cleanup process of the transported objects.

[0124] In step 21 (S21), the lighting control unit 870 detects that the autonomous vehicle 120 has begun docking with the transported object.

[0125] For example, the lighting control unit 870 can recognize the transported object in the color image captured by the RGB camera of the autonomous vehicle 120, and when it detects that the vehicle has moved to the front or rear of the transported object, it can determine that the autonomous vehicle 120 should begin docking with the transported object.

[0126] In step 22 (S22), the lighting control unit 870 turns on the lights in response to the detection of the start of docking in S21. Specifically, the lighting control unit 870 controls the lighting mechanism 223 of the autonomous vehicle 120 to turn on the lights.

[0127] Subsequently, the autonomous vehicle 120 illuminates the markers attached to the transported object (for example, two barcodes attached to the underside of the bottom shelf 400 of the shelf 130) with its lights, and based on these markers, it can determine the positional relationship between the autonomous vehicle 120 and the transported object (for example, by performing triangulation using the two barcodes).

[0128] Here, the direction, range, and brightness of the lighting will be described. The direction and range of the lighting will be adjusted to be the same as the direction and range of the RGB camera of the autonomous vehicle 120. The brightness of the lighting will be adjusted to be bright enough for the RGB camera of the autonomous vehicle 120 to recognize markers in the dark, while minimizing glare from the floor surface. A diffuser may be provided in the lighting mechanism 223.

[0129] In step 23 (S23), the lighting control unit 870 detects that the autonomous vehicle 120 has completed docking with the transported object (i.e., the autonomous vehicle 120 has docked with the transported object).

[0130] For example, if the lighting control unit 870 detects, using sensors or other means provided by the autonomous vehicle 120, that the lock pin 211 of the autonomous vehicle 120 has been inserted into the hole of the lock guide of the shelf 130, or if it determines that the lock pin 211 has been inserted into the hole, it can determine that the autonomous vehicle 120 has completed docking with the transported object.

[0131] In step 24 (S24), the lighting control unit 870 turns off the lights in response to detecting the completion of docking in S23. Specifically, the lighting control unit 870 controls the lighting mechanism 223 of the autonomous vehicle 120 to turn it off.

[0132] <Retrying docking> Figure 17 is an example of a flowchart showing the flow of the docking retry process according to one embodiment of the present disclosure. The docking retry process may be performed during the packing up process of the transported object, or during a process other than the packing up process of the transported object.

[0133] In step 31 (S31), the docking control unit 850 detects at least one of the lateral displacement of the autonomous vehicle 120 relative to the transported object and the angular displacement of the autonomous vehicle 120 relative to the transported object while the autonomous vehicle 120 is moving toward the transported object for docking (during the docking trial). For example, while the autonomous vehicle 120 is moving toward the transported object, the docking control unit 850 continuously measures at least one of the amount of lateral displacement and the amount of angular displacement of the autonomous vehicle 120 relative to the transported object, and detects the occurrence of lateral displacement or angular displacement when at least one of the amount of lateral displacement and the amount of angular displacement exceeds a predetermined threshold.

[0134] In step 32 (S32), the docking control unit 850 performs a return operation in response to the detection in S31. Specifically, the docking control unit 850 instructs the transport control unit 860 to have the autonomous vehicle 120 move away from the transport target. At this time, the docking control unit 850 may instruct the transport control unit 860 to have the autonomous vehicle 120 return to the docking start position. This is because there is a high probability that there are no obstacles between the transport target and the docking start position, so it is expected that the return operation can be performed reliably. Alternatively, the docking control unit 850 may instruct the transport control unit 860 to have the autonomous vehicle 120 move to a position facing the transport target, which is a predetermined distance away from the transport target. This is expected to increase the probability of successfully retrying docking.

[0135] In step 33 (S33), the docking control unit 850 controls the system to restart docking.

[0136] Thus, the docking control unit 850 can control the autonomous vehicle 120 to retry docking with the transported object depending on its position or orientation during the docking attempt. Lateral displacement and angular displacement will be described below with reference to Figures 18 and 19.

[0137] Figure 18 is a diagram illustrating lateral displacement and angular displacement according to one embodiment of the present disclosure. Assume that the autonomous vehicle 120 is attempting to dock with the object to be transported (shelf 130 in the example of Figure 18). Markers (for example, two barcodes in a left-right pair on the front of shelf 130 and two barcodes in a left-right pair on the rear of shelf 130 (the barcodes on the front and rear are different)) are attached to the underside of the bottom shelf 400 of shelf 130 so as to intersect perpendicularly with frame guides 410 and 420) (shaded area in Figure 18).

[0138] The autonomous vehicle 120 can perform triangulation using two barcodes, a left-right pair, in the color image captured by the RGB camera, and recognize the relative position (x,y) and relative angle θ from the RGB camera to the center point of the shelf 130.

[0139] Figure 19 is a diagram illustrating lateral displacement and angular displacement according to one embodiment of the present disclosure.

[0140] <<Lateral displacement>> Figure 19 This indicates the position of the autonomous vehicle 120 during the docking trial to the shelf 130. The autonomous vehicle 120 (docking control unit 850) detects that the distance from the center point of the shelf 130 to the autonomous vehicle 120 is below a threshold, and that the lateral displacement of the autonomous vehicle 120 from the center line of the shelf 130 (d in Figure 19 (distance between the center line of the shelf 130 and the autonomous vehicle 120)) is above a threshold, and cancels the docking trial. The threshold for the distance from the center point of the shelf 130 to the autonomous vehicle 120 is set to a predetermined value greater than the distance from the center point of the shelf 130 to the outer circumference of the shelf 130, so that the autonomous vehicle 120 can start the return operation before it enters below the bottom shelf 400 of the shelf 130. These thresholds are stored internally by the autonomous vehicle 120. After that, the autonomous vehicle 120 returns to the predetermined position and starts docking again. The designated position may be the docking start position, a position on the center line of shelf 130 (opposite the shelf 130), or a position at a predetermined distance from shelf 130 (the star in Figure 19).

[0141] <<Angle deviation>> Figure 19 This indicates the orientation of the autonomous vehicle 120 during the docking trial to the shelf 130. The autonomous vehicle 120 (docking control unit 850) detects that the distance from the center point of the shelf 130 to the autonomous vehicle 120 is below a threshold, and that the angular deviation of the autonomous vehicle 120 from the center line of the shelf 130 (dotted line in Figure 19) (θ in Figure 19 (angle between the center line of the shelf 130 and the direction of travel of the autonomous vehicle 120)) is above a threshold, and then cancels the docking trial. The threshold for the distance from the center point of the shelf 130 to the autonomous vehicle 120 is set to a predetermined value greater than the distance from the center point of the shelf 130 to the outer circumference of the shelf 130, so that the autonomous vehicle 120 can start the return operation before it enters below the bottom shelf 400 of the shelf 130. These thresholds are stored internally by the autonomous vehicle 120. After that, the autonomous vehicle 120 returns to the predetermined position and starts docking again. The designated position may be the docking start position, a position on the center line of shelf 130 (opposite the shelf 130), or a position at a predetermined distance from shelf 130 (the star in Figure 19).

[0142] [Second Embodiment] In the first embodiment described above, a docking mechanism having a solenoid-type locking pin 211 and a photoreflector 330 was illustrated, but the docking mechanism is not limited to this, and any conventional mechanism can be applied. Also, in the first embodiment described above, the case in which docking occurs after entering the area below the lowest shelf to be transported was explained, but the mechanism may be configured to dock without entering the area below the lowest shelf to be transported. For example, docking may be performed by gripping the legs of the shelf to be transported with a gripper.

[0143] Furthermore, although a shelf was used as an example of the object to be transported in the first embodiment described above, the object to be transported is not limited to a shelf; it may be any other piece of furniture to which casters are attached so as to be rotatable.

[0144] Furthermore, in the first embodiment described above, when docking with a shelf at the anchor position, the autonomous vehicle 120 was described as moving in the reverse direction when entering the area below the lowest shelf. However, it may also enter the area below the lowest shelf by moving in the forward direction.

[0145] Furthermore, although a detailed explanation of the anchor's position was omitted in the first embodiment described above, the anchor's position is, for example, the position where a two-dimensional identifier such as a QR code (registered trademark) is installed within a predetermined space 100.

[0146] Furthermore, in the first embodiment described above, when a task is recognized by a voice command from the user, the case where there is no further voice command until the task is completed was explained. However, the next voice command may be entered before the currently running task is completed.

[0147] Furthermore, in the first embodiment described above, a case was described in which the autonomous vehicle 120 immediately executes a task corresponding to a voice instruction given by the user 110. However, if the voice instruction from the user 110 is a voice instruction to reserve the execution of a task at a predetermined time, the autonomous vehicle 120 will execute the task after the predetermined time has arrived.

[0148] In other words, if the user's (user's) voice command includes the timing for executing a task, the autonomous vehicle 120 detects that the timing for executing the task has arrived and executes the task at the timing determined based on the voice command. Note that setting (also called reserving) the timing for executing a task is not limited to voice commands; it may also be done electronically from an external device 730 that can communicate with the autonomous vehicle 120 (control device 310). Examples of such external devices 730 include a mobile device such as a smartphone owned by the user.

[0149] In the embodiments described above, docking with the transported object by the autonomous vehicle 120 and transporting the transported object were described as being controlled based on the user's voice acquired via a microphone, which is a sound input device. However, docking with the transported object and transporting the transported object may also be controlled based on a specific sound acquired via a microphone, which is a sound input device. Examples of specific sounds include a series of sounds such as clapping hands N times at intervals of approximately M seconds, or whistling. In this case, at least one of the transported object and the transport destination may be pre-set for each specific sound. In other words, voice instructions include not only instructions by the user's voice but also instructions by specific sounds.

[0150] [Other embodiments] In this specification (including the claims), when the expression "at least one of a, b, and c" or "at least one of a, b, or c" (including similar expressions) is used, it includes any of a, b, c, ab, ac, bc, or abc. Furthermore, any element may have multiple instances, such as aa, abb, aabbcc, etc. In addition, it is also possible to add other elements other than the enumerated elements (a, b, and c), such as abcd which has d.

[0151] In this specification (including the claims), when expressions such as "using data as input / based on data / according to / in accordance with data" (including similar expressions) are used, unless otherwise specified, this includes using the data itself or using data that has been processed in some way (e.g., data with added noise, normalized data, features extracted from the data, intermediate representations of the data, etc.). Furthermore, when it is stated that some result is obtained "using data as input / based on data / according to / in accordance with data" (including similar expressions), unless otherwise specified, this includes cases where the result is obtained based solely on the data in question or where the result is influenced by other data, factors, conditions, and / or states other than the data in question. Furthermore, when it is stated that "data is output" (including similar expressions), unless otherwise specified, this includes cases where the data itself is used as output or where data that has been processed in some way (e.g., data with added noise, normalized data, features extracted from the data, intermediate representations of various types of data, etc.) is used as output.

[0152] In this specification (including the claims), the terms “connected” and “coupled” are intended to be non-restrictive terms that include any direct connection / coupling, indirect connection / coupling, electrical connection / coupling, communicative connection / coupling, operational connection / coupling, physical connection / coupling, etc. The terms should be interpreted as appropriate in the context in which they are used, but any form of connection / coupling that is not intentionally or naturally excluded should be interpreted non-restrictively as being included in the terms.

[0153] In this specification (including the claims), when the expression "A configured to B" is used, it may include that the physical structure of element A has a configuration capable of performing operation B, and that the permanent or temporary setting / configuration of element A is configured to actually perform operation B. For example, if element A is a general-purpose processor, it is sufficient that the processor has a hardware configuration capable of performing operation B, and that it is configured to actually perform operation B by the setting of a permanent or temporary program (instruction). Furthermore, if element A is a dedicated processor, dedicated arithmetic circuit, etc., it is sufficient that the circuit structure of the processor is implemented to actually perform operation B, regardless of whether control instructions and data are actually attached.

[0154] Wherever terms meaning "comprising" or "possessing" (e.g., "comprising / including," "having," etc.) are used herein, they are intended to be open-ended terms, including cases where the subject matter of such terms is not the object of the term. Where the object of such terms meaning "comprising" or "possessing" is an expression that does not specify a quantity or suggests a singular number (an expression with the article "a" or "an"), such expression should be interpreted as not being limited to a specific number.

[0155] In this specification (including the claims), even if expressions such as "one or more" or "at least one" are used in some places, and expressions that do not specify a quantity or suggest a singularity (expressions using the articles a or an) are used in other places, the latter expressions are not intended to mean "one." In general, expressions that do not specify a quantity or suggest a singularity (expressions using the articles a or an) should not necessarily be interpreted as not being limited to a specific number.

[0156] In this specification, if a particular configuration of an embodiment is described as having a specific advantage or result, it should be understood, unless otherwise stated, that the same advantage or result can also be obtained from one or more other embodiments having that configuration. However, it should be understood that the presence or absence of such advantage or result generally depends on various factors, conditions, and / or states, and that such advantage or result cannot necessarily be obtained from that configuration. The advantage or result can only be obtained from the configuration described in the embodiment when various factors, conditions, and / or states are met, and it cannot necessarily be obtained from the invention claimed to define that configuration or a similar configuration.

[0157] In this specification (including the claims), when multiple hardware components perform a predetermined process, each component may cooperate to perform the predetermined process, or some components may perform all of the predetermined process. Alternatively, some components may perform part of the predetermined process, while other components perform the remainder. In this specification (including the claims), when expressions such as "one or more hardware components perform a first process, and the one or more hardware components perform a second process" (including similar expressions) are used, the hardware component performing the first process and the hardware component performing the second process may be the same or different. In other words, it is sufficient that the hardware component performing the first process and the hardware component performing the second process are included in the one or more hardware components. Hardware may include electronic circuits, devices containing electronic circuits, etc.

[0158] In this specification (including the claims), when multiple memory devices store data, each of the multiple memory devices may store only a portion of the data or the entire data. Furthermore, a configuration in which some of the multiple memory devices store data is also included.

[0159] While embodiments of this disclosure have been described in detail above, this disclosure is not limited to the individual embodiments described above. Various additions, modifications, substitutions, and partial deletions are possible, provided that they do not depart from the conceptual idea and spirit of the present invention derived from the claims and their equivalents. For example, where numerical values ​​or mathematical formulas are used in the description of the embodiments described above, these are provided for illustrative purposes only and do not limit the scope of this disclosure. Similarly, the sequence of operations shown in the embodiments is also illustrative and does not limit the scope of this disclosure.

Claims

1. An autonomous vehicle that enters the space beneath an object to be transported, docks with the object to be transported, and transports the object, Lighting equipment, A camera and A control device that photographs a marker on the transported object with the photographing device during docking and controls the movement of the autonomous vehicle based on the photographed marker, It has, The marker is provided in the space of the object to be transported. The control device, during docking, turns on the illumination device so that the markers provided in the space are illuminated and photographed by the photographing device. Autonomous vehicle.

2. When the control device detects that the autonomous vehicle has moved to the transport object to be docked, it turns on the lighting device to project light into the space equipped with the marker. The autonomous vehicle according to claim 1.

3. The control device is Based on the image captured by the aforementioned imaging device, the transport object to be docked is searched for, When the camera detects that the autonomous vehicle has moved to the object being transported, as recognized in the image captured by the camera, the lighting device is turned on. The autonomous vehicle according to claim 2.

4. The control device is During the docking process, the lighting device is turned on before the autonomous vehicle enters the space to illuminate the marker. Until the docking is complete, the vehicle's movement during docking is controlled based on the illuminated and photographed marker. The autonomous vehicle according to claim 1.

5. The control device determines the positional relationship between the autonomous vehicle and the transported object based on the illuminated and photographed marker, and controls the movement of the autonomous vehicle during docking according to the determined positional relationship. The autonomous vehicle according to claim 4.

6. The control device determines the positional relationship between the autonomous vehicle and the transported object based on the illuminated and photographed marker, and controls the autonomous vehicle's movement to re-dock with the transported object according to the determined positional relationship. The autonomous vehicle according to claim 4.

7. The control device, in the docking rework, causes the autonomous vehicle to move away from the transported object and then to dock with the transported object. The autonomous vehicle according to claim 6.

8. When the control device moves the autonomous vehicle away from the transported object, it moves the autonomous vehicle back to the position where docking was initiated. The autonomous vehicle according to claim 7.

9. When the control device moves the autonomous vehicle away from the object to be transported, it moves the autonomous vehicle to a position a predetermined distance away from the object to be transported. The autonomous vehicle according to claim 7.

10. The control device determines the positional relationship between the autonomous vehicle and the transported object by triangulation using the marker that has been illuminated and photographed. The autonomous vehicle according to claim 5.

11. The marker is a plurality of barcodes, The autonomous vehicle according to claim 10.

12. The control device controls the movement of the autonomous vehicle when it is docked and transporting the transported object, based on the information contained in the marker. The autonomous vehicle according to claim 1.

13. The control device controls the autonomous vehicle's movement to perform a movement appropriate to the size of the object being transported, based on the information contained in the marker. The autonomous vehicle according to claim 1.

14. The control device determines the size of the object to be transported based on the information contained in the marker. The autonomous vehicle according to claim 13.

15. The control device turns off the lighting device when docking with the transported object is complete. An autonomous vehicle according to any one of claims 1 to 14.

16. The control device recognizes the marker from the captured image of the marker and controls the driving of the autonomous vehicle based on the recognized marker. An autonomous vehicle according to any one of claims 1 to 14.

17. The lighting device lights up to a brightness sufficient for the control device to recognize the marker from the image captured by the imaging device. The autonomous vehicle according to claim 16.

18. The control device is The autonomous vehicle is controlled to transport the docked object to a predetermined location. The wall surface at the predetermined location is detected, The autonomous vehicle is controlled to align the position or orientation of the transported object with the wall surface and to disdock it. An autonomous vehicle according to any one of claims 1 to 14.

19. When the control device docks with a plurality of transport targets in sequence and transports them to their respective destinations, it controls the illumination of the lighting device and the movement of the autonomous vehicle for each of the transport targets. An autonomous vehicle according to any one of claims 1 to 14.

20. The control device determines the docking and transport order for the plurality of transport objects, and controls the autonomous vehicle to dock with and transport each of the transport objects according to the determined order. The autonomous vehicle according to claim 19.

21. The control device evaluates the ease of docking with each of the plurality of transport objects and determines the order based on the evaluation. The autonomous vehicle according to claim 20.

22. A program for causing a computer to function as the control device according to any one of claims 1 to 14.

23. A control method for an autonomous vehicle that enters the space below a transport object, docks with the transport object, and transports the transport object, When the autonomous vehicle enters the space and docks with the transported object, the marker provided in the space of the transported object is photographed with a camera. The autonomous vehicle's movement is controlled based on the captured markers. In the control method, When the autonomous vehicle enters the space and docks with the transported object, the marker provided in the space is illuminated with a lighting device, and the image is taken of the illuminated marker. Control method.

Images