Autonomous movement device, autonomous movement improvement system, and movement improvement method for autonomous movement device

Abstract

Provided is a movement improvement method for an autonomous mobile device that autonomously moves on the basis of a received signal and an autonomous movement algorithm, the method comprising: acquiring data of a signal received by the autonomous mobile device and an image captured by a camera mounted on the autonomous mobile device; data and an image of a signal in a predetermined movement state of the autonomous movement device are identified, and an autonomous movement algorithm of the autonomous movement device is changed on the basis of the identified data and image of the signal.

Description

Technical Field

[0001] The present disclosure relates to an autonomous mobile device, an autonomous mobile improvement system, and a mobile improvement method for an autonomous mobile device. Background Art

[0002] An autonomous mobile device is known that receives radio waves from a beacon output from a target object and sound waves reflected by obstacles, and autonomously moves to the target object while avoiding the obstacles based on the radio waves and sound waves (see Patent Document 1).

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: International Publication No. 2022 / 181488 Summary of the Invention

[0006] [summary]

[0007] Since radio waves and sound waves are both waves as sensing data, the shape of obstacles cannot be clearly understood based on radio waves and sound waves alone. Therefore, it is not easy to improve the autonomous movement of the autonomous movement device based only on the radio waves and sound waves received by the autonomous movement device.

[0008] An object of the present disclosure is to provide an autonomous mobile device, an autonomous mobile improvement system, and a method for improving the movement of an autonomous mobile device, which can accurately improve the autonomous movement of the autonomous mobile device.

[0009] To address the aforementioned issues, one aspect of the present disclosure is a method for improving the movement of an autonomous mobile device, which autonomously moves based on received signals and an autonomous movement algorithm. In this method, data on signals received by the autonomous mobile device and images captured by a camera mounted on the autonomous mobile device are obtained, the data on the signals and images under a predetermined movement state of the autonomous mobile device are determined, and the autonomous movement algorithm of the autonomous mobile device is modified based on the determined data on the signals and images.

[0010] Another embodiment of the present invention is an autonomous mobile device comprising: a receiving device for receiving a signal; a control unit for moving the autonomous mobile device based on the signal received by the receiving device and an autonomous mobile algorithm; a camera for photographing the surroundings of the autonomous mobile device; and a movement improvement unit for changing the autonomous mobile algorithm based on data of the signal received by the receiving device and an image captured by the camera.

[0011] Another embodiment of the present disclosure is an autonomous mobility improvement system comprising: one or more autonomous mobile devices; and a server communicably connected to the one or more autonomous mobile devices via a network. Each autonomous mobile device comprises: a receiving device for receiving signals; a control unit for moving the autonomous mobile device based on the signals received by the receiving device and an autonomous mobility algorithm; and a camera for capturing images of the surroundings of the autonomous mobile device. The server receives data of the signals received by the receiving device and images captured by the camera from the one or more autonomous mobile devices, changes the autonomous mobility algorithm based on the data of the signals received by the receiving device and the images captured by the camera, and transmits the changed autonomous mobility algorithm to the one or more autonomous mobile devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is a plan view schematically illustrating a digital pheromone as a structure for the autonomous mobile device 100 to reach a target object (transmitting device) 200 while avoiding obstacles J1 and J2 .

[0013] Figure 2 This is a plan view explaining an outline of echolocation as a configuration in which the autonomous mobile device 100 reaches a target object (destination) P1 while avoiding obstacles p1 to p4 .

[0014] Figure 3 It is a block diagram showing a detailed configuration of the autonomous mobile device 100 according to the embodiment.

[0015] Figure 4 is shown for implementing Figure 3 Detailed block diagram of the structural elements of the echolocation function of the autonomous mobile device 100.

[0016] Figure 5 This is a block diagram showing an example of the flow of data in the autonomous mobility improvement system according to the second embodiment.

[0017] Figure 6 This is a block diagram showing an example of the flow of data in the autonomous mobile device 100 according to the first embodiment.

[0018] Figure 7 This is a flowchart showing an example of a method for improving mobility of the autonomous mobility device 100 . DETAILED DESCRIPTION

[0019] [Detailed description]

[0020] (Autonomous mobility device and autonomous mobility system)

[0021] Hereinafter, an overview of an autonomous mobile device and an autonomous mobile system according to an embodiment of the present disclosure, which autonomously move to a target object while avoiding obstacles, will be described.

[0022] In addition, the embodiments described below show general or specific examples. The numerical values, shapes, materials, structural elements, the locations of the structural elements, and the connection forms in the embodiments described below are examples and are not intended to limit the present disclosure. In addition, among the structural elements in the following embodiments, the structural elements that are not recorded in the independent claims representing the highest concepts are described as arbitrary structural elements. Furthermore, for the sake of convenience of explanation, the dimensional ratios of the drawings are sometimes exaggerated and different from the actual ratios. In addition, in the following embodiments and their modified examples, the same structural elements are sometimes included, and the same structural elements are given common symbols and repeated descriptions are omitted.

[0023] Autonomous mobility devices, for example, have a structure capable of autonomously reaching a target object within the interior spaces of structures such as homes and offices, factories, or, as appropriate, exterior spaces. Furthermore, by using propellers capable of moving in mid-air as a mobility mechanism, so-called drones and other aerial vehicles can also be configured to autonomously reach a target object. Furthermore, the system can also be applied to vehicles such as cars and buses, aircraft, spacecraft, ships, submarines, and other mobile objects. In the embodiments, the description continues using wheeled vehicles such as cars and buses as an example.

[0024] The autonomous mobile device does not use cameras or other imaging devices, LiDAR (Light Detection and Ranging), or radar. Instead, it uses signals emitted by the target object to reach the target while avoiding obstacles. The signals emitted by the target object are not particularly limited and can be, for example, sound waves, radio waves, or high-frequency electromagnetic waves. The following explanation uses radio waves as an example. The autonomous mobile device receives radio waves from beacons and other sources using multiple antennas. Using a technique for estimating the direction of arrival of the radio waves, it estimates the direction of the target object emitting the radio waves and moves in that estimated direction. If there is an obstacle outside the line of sight between the target object and the autonomous mobile device, the autonomous mobile device may move in the direction of the radio waves reflected from the obstacle. However, during movement, the autonomous mobile device may also receive radio waves directly from the target object. In this case, the autonomous mobile device can change its direction toward the target object while moving toward the obstacle. As a result, it can avoid the obstacle and move toward the target object. Furthermore, if an obstacle exists in the line of sight between the target object and the autonomous mobile device, the intensity of the radio wave reception fluctuates as the autonomous mobile device moves toward the obstacle, allowing the autonomous mobile device to detect the presence of the obstacle. In this way, the autonomous mobile device estimates the direction of the incoming radio wave and continues moving in the direction of stronger radio wave reception, thus avoiding the obstacle and reaching the target object.

[0025] Reference Figure 1 as well as Figure 2 , an overview of an autonomous mobile system 1000 including an autonomous mobile device 100 and a target object 200 is given.

[0026] (Overview of Digital Pheromones)

[0027] Reference Figure 1 This section describes a digital pheromone system in which the autonomous mobile device 100 estimates the direction of arrival of radio waves and continues moving in the direction of strong radio wave reception, thereby reaching the target object (transmitter 200) while avoiding obstacles J1 and J2. The autonomous mobile device 100 receives radio waves transmitted from the transmitter of the target object 200. The line of sight between the autonomous mobile device 100 and the target object 200 is blocked by obstacle J2. Therefore, the autonomous mobile device 100 receives radio waves via path K3, path K2, and then path K1. Depending on the size of obstacle J2 and the frequency of the beacon, the autonomous mobile device 100 may also receive radio waves from the line of sight, but it is assumed that the intensity of the radio waves received via path K1 is the highest. The autonomous mobile device 100 estimates the direction of arrival of the strongest radio waves using the multiple antennas mounted on the autonomous mobile device 100 and moves based on this estimated direction of arrival.

[0028] As the autonomous mobile device 100 approaches obstacle J1 on path K1, the intensity of radio wave reception increases, causing it to continue moving toward obstacle J1 along path K1. However, upon reaching position X1, the transmitting device 200 appears in front of the autonomous mobile device 100's line of sight, allowing the autonomous mobile device 100 to directly receive radio wave TS3. Consequently, at position X1, the intensity of radio wave TS3 received becomes greater than that of radio wave TS2, and the autonomous mobile device 100 changes its direction of movement to the direction of arrival of radio wave TS3. The autonomous mobile device 100 could also move along the line of arrival of radio wave TS3, but in this case, there is a possibility that the autonomous mobile device 100 will collide with obstacle J2. Therefore, based on the fact that radio wave TS3 cannot be received on path K1 until position X1, and the estimated direction of arrival of radio wave TS3 based on the strong reception intensity at position X1, the autonomous mobile device 100 recognizes the presence of obstacle J2 and moves toward path K2. The autonomous mobile device 100, moving in the direction of path K2, recognizes the presence of obstacle J1 based on the fact that the direction of arrival of the radio waves output from the transmitter 200 is gradually expanding relative to the direction of travel of the autonomous mobile device 100 and the fact that the direction of travel has changed at position X1, and estimates path K3. Therefore, the autonomous mobile device 100 can change its direction of travel toward the transmitter 200 at position X2 and reach the transmitter 200. In addition, a detailed structural example of the digital pheromone structure of the autonomous mobile device 100 is also disclosed in International Publication No. 2022 / 181488, which will be referred to in the following sections of this specification. Figure 3 To be described later.

[0029] (Overview of echolocation)

[0030] Reference Figure 2 The following describes an overview of echolocation as a configuration in which the autonomous mobile device 100 reaches the destination P1 as a target object while selecting a preferred driving route that is less affected by obstacles p1 to p4. Figure 2 1 is an explanatory diagram showing a situation in which the autonomous mobile device 100 travels on a planar travel path having a plurality of obstacles p1 to p4 and heads toward a destination P1 .

[0031] When autonomous mobile device 100 autonomously moves from position p0 to destination P1, obstacle p2 appears on the shortest travel path x0. In this situation, autonomous mobile device 100 outputs sound waves in the direction of travel and receives the sound waves reflected from the surface of obstacle p2. Based on the received sound waves, obstacle p2 is detected and the direction of movement is changed at position P2 in front of obstacle p2, avoiding collision with obstacle p2. The sound waves received by autonomous mobile device 100 are an example of signals received by autonomous mobile device 100. Signals received by autonomous mobile device 100 include radio waves from beacons and other sources, as well as sound waves reflected from obstacles.

[0032] At this point, it's desirable for the autonomous mobile device 100 to change its direction of movement to the left to avoid obstacle p2. Specifically, the driving path x1 to the left of obstacle p2, as viewed from the autonomous mobile device 100, is an open space, allowing the autonomous mobile device 100 to travel unrestricted by the obstacle. However, the driving path x2 to the right of obstacle p2 is complex and subject to significant obstacles during travel. Therefore, it's preferable to change the direction of movement of the autonomous mobile device 100 to the left.

[0033] On the other hand, the driving path x2 to the right of obstacle p2 is complex and subject to significant obstacles during driving. More specifically, a) if an obstacle is present in the immediate vicinity of the antenna, a phase shift occurs, significantly reducing the accuracy of direction detection. Furthermore, b) when entering a complex space with numerous obstacles, the autonomous mobile device 100's radio waves are reflected in a complex manner, making it difficult to escape.

[0034] Autonomous mobility device 100 uses a pair of left and right microphones to receive sound reflected from surrounding objects (obstacles p1-p4). By comparing the left and right sound signals, it can avoid a complex space with many obstacles p1-p4, allowing it to navigate open spaces without being restricted by surrounding objects and reach destination P1. A more detailed method will be described later.

[0035] (Details of autonomous mobility devices)

[0036] Reference Figure 3 , the detailed structure of the autonomous mobile device 100 will be described. The autonomous mobile device 100 includes a receiving unit 110, a switch unit 120 for selecting a receiving element of the receiving unit 110, a control unit 130, a storage unit 140, an information acquisition unit 150, a driving unit 160, and a moving unit 170. Figure 3 The driving information output by the driving unit 160 shown in FIG. 1 drives the moving unit 170 such as wheels, conveyor belts, crawlers, and propellers, and the autonomous moving device 100 moves. In addition, the receiving unit 110 has a plurality of receiving elements. Figure 3 The camera 180 and the movement improvement unit 190 shown will be described later in the section “Movement improvement of an autonomous mobile device using camera images”.

[0037] Receiver 110 is an antenna that receives radio waves including high-frequency electromagnetic waves output from transmitter 200. For example, receiver 110 is an array antenna composed of multiple antenna elements as an example of receiving elements. The antenna elements constituting the array antenna can be arranged in any arrangement.

[0038] The switch unit 120 is configured to select any receiving element of the receiving unit 110 and output the radio waves received by the receiving element. Therefore, the number of switches comprising the switch unit 120 is equal to the number of receiving elements provided by the receiving unit 110, with one switch corresponding to one receiving element. For example, if the receiving unit 110 is an array antenna, multiple antenna elements are selected, and information such as the intensity and phase of the radio waves received by the multiple antenna elements is output to the phase difference determination unit 131 and the reception strength determination unit 132, described later. The switch unit 120 may also be a semiconductor switch, but is not limited thereto; any switch having any structure capable of opening and closing an electrical connection may be employed.

[0039] Control unit 130 can be implemented using a microcomputer equipped with a CPU (Central Processing Unit). A computer program (autonomous motion program) for enabling the microcomputer to function as control unit 130 is installed in the microcomputer and executed. This allows the microcomputer to function as the multiple information processing units included in control unit 130.

[0040] The control unit 130 includes a phase difference determination unit 131, a reception intensity determination unit 132, a reception element selection unit 133, an angle estimation unit 134, an operation control unit 135, and an obstacle avoidance unit 136 as multiple information processing units. The control unit 130 controls the autonomous mobility device 100 to move based on signals (including radio and acoustic signals) received by the receiving unit 110 and the information acquisition unit 150 and the autonomous mobility algorithm described below.

[0041] Phase difference determination unit 131 analyzes the received signals from the multiple receiving elements of receiving unit 110 selected by receiving element selection unit 133 and determines the phase difference between the received signals based on the difference in arrival time between the received signals. The determined phase difference is output to angle estimation unit 134. Furthermore, even when autonomous mobile device 100 is stationary or moving, phase difference determination unit 131 can determine a single angle based on multiple phase differences between the multiple received signals.

[0042] The reception strength determination unit 132 determines the reception strength of the multiple receiving elements of the receiving unit 110 selected by the receiving element selection unit 133. The estimated reception strength is output to the operation control unit 135. Alternatively, the estimated reception strength may be output to the receiving element selection unit 133. Furthermore, the reception strength may be expressed in any unit related to the reception strength or as relative information. The reception strength may be output to the operation control unit 135 and the receiving element selection unit 133 as reception strength information in any format.

[0043] The receiving element selection unit 133 simultaneously selects one or more receiving elements from the multiple receiving elements included in the receiving unit 110 that receive radio waves. The receiving element selection unit 133 simultaneously selects multiple receiving elements to determine the phase difference in the phase difference determination unit 131. Alternatively, the receiving elements may be selected by switching them sequentially, and the reception strength determination unit 132 determines which of the one or more receiving elements has the highest reception strength. The angle estimation unit 134 estimates the direction of arrival of radio waves, etc., based on the signals received by the one or more receiving elements determined to have the highest reception strength, via the phase difference determination unit 131.

[0044] The angle estimation unit 134 can employ any method for estimating the direction of arrival using an array antenna. For example, the angle estimation unit 134 can employ a method that uses a pair of antenna elements, pre-calculates the complex reception response to the incoming wave based on the phase difference between the two paired antenna elements, introduces an evaluation function, and uses the angle with the maximum evaluation function value as the method for estimating the direction of arrival of the radio wave. The angle estimation unit 134 can also estimate the direction of arrival of the radio wave based on the phase differences of multiple antenna elements. For example, the MUSIC (Multiple Signal Classification) method or the Root-MUSIC method, which uses the eigenvalues ​​and eigenvectors of the correlation matrix, can also be employed. Furthermore, the ESPRIT (Estimation of Signal Parameters via Rotational Invariance Techniques) method can also be employed. The angle thus estimated is stored as angle information relative to an arbitrary reference axis in the angle information storage unit 141 of the storage unit 140. Furthermore, the estimated angle information may be associated with the reception intensity determined by the reception intensity determination unit 132 and stored in the angle information storage unit 141. The estimated angle information may also be associated with the determined reception intensity and time information and stored in the angle information storage unit 141. The time information may be received from outside the autonomous mobile device 100 by the reception unit 110 or may be timed by the autonomous mobile device 100 using a timer (not shown).

[0045] There are sometimes multiple angles estimated by the angle estimation unit 134. When there are multiple estimated angles, the angle estimation unit 134 receives the reception intensity at each angle from the reception intensity determination unit 132, associates each angle with the reception intensity, and stores the result in the angle information storage unit 141. For example, as shown in FIG. Figure 1 As described, the autonomous mobile device 100 at position X1 receives radio waves reflected by obstacle J1 and radio waves propagating in the line of sight at different angles. Radio waves reflected by obstacle J1 may also be reflected by other obstacles J2 and received by the autonomous mobile device 100 at different angles. In this way, reflected waves from obstacles may be reflected multiple times before reaching the autonomous mobile device 100. Basically, the autonomous mobile device 100 moves in the direction of high reception intensity, but due to obstacles, it may be impossible to move in the direction of high reception intensity or it may take the wrong path. In this way, the autonomous mobile device 100 may be forced to move in the direction of other reflected waves. Therefore, when multiple angles are estimated, they may be associated with the reception intensity, and the autonomous mobile device 100 may store this information in the angle information storage unit 141.

[0046] The motion control unit 135 generates movement direction information including the movement direction for the autonomous mobile device 100 based on the magnitude or change in the reception intensity of the radio wave signal determined by the reception intensity determination unit 132 and the arrival direction of the radio wave estimated by the angle estimation unit 134. In an embodiment, the obstacle avoidance unit 136 determines the presence of an obstacle or a complex space around the autonomous mobile device 100 based on the acoustic signal received by the information acquisition unit 150, described later. In this case, the motion control unit 135 generates the movement direction information when the reliability (I) of either or both of the estimation result of the angle estimation unit 134 and the determination result of the reception intensity determination unit 132 is below a predetermined reference value. In other words, if the reliability (I) is below the predetermined reference value, the motion control unit 135 controls the autonomous mobile device 100 to move around the obstacle or complex space surrounding the autonomous mobile device 100. Reliability (I) is a coefficient that decreases when there are obstacles or complex spaces around autonomous mobile device 100 and decreases as the distance from autonomous mobile device 100 to the obstacle or complex space is shorter. Reliability (I) is a coefficient assigned to the estimation results of angle estimation unit 134 and the determination results of reception intensity determination unit 132.

[0047] For example, the motion control unit 135 may generate movement direction information by weighting the estimated radio wave arrival directions according to the reliability (I). More specifically, the motion control unit 135 may multiply the reception strength (R) of multiple estimated radio wave arrival directions by the reliability (I) and control the autonomous mobile device 100 to move toward the arrival direction with the larger product (R×I). This is not limited to estimating multiple radio wave arrival directions simultaneously; multiple radio wave arrival directions may also be compared based on past historical records. In other words, the motion control unit 135 may weight the radio wave arrival directions stored in the storage unit 140 with an index corresponding to the reliability (I) to generate movement direction information. Furthermore, if the reliability (I) decreases, the motion control unit 135 may control the autonomous mobile device 100 to move to a space or direction with higher reliability (I).

[0048] Various methods exist for determining that the reliability (I) is low due to the presence of an obstacle or a complex space around the autonomous mobile device 100 by the motion control unit 135. For example, the motion control unit 135 may determine the reliability (I) based on at least one of the magnitude, variability, number of receptions, left-right comparison, comparison with past records, distance to an estimated obstacle, shape of the space or path, reception intensity, noise level, and stability of the direction of arrival angle of the received signal. Furthermore, the "received signal" includes both radio signals received by the receiving unit 110 and acoustic signals received by the information acquisition unit 150. For example, if the reception intensity determined by the reception intensity determination unit 132 periodically fluctuates in the estimated direction of arrival of the radio waves, it may be determined that an obstacle exists in the estimated direction of arrival, thus reducing the reliability (I). This is because if the reception intensity periodically fluctuates, there may be an obstacle around the autonomous mobile device 100 or between the autonomous mobile device 100 and the target object, potentially receiving diffracted waves.

[0049] To allow the motion control unit 135 to detect surrounding obstacles or complex spaces, the autonomous mobile device 100 may also include an obstacle avoidance unit 136 that measures the direction and distance to obstacles. Alternatively, the information acquisition unit 150 may be an infrared sensor, an ultrasonic sensor, or a depth sensor. Furthermore, upon receiving contact prediction information or contact information from the obstacle avoidance unit 136, the motion control unit 135 may determine that the reliability (I) is low and change the direction of movement to avoid obstacles or complex spaces. In this case, the changed direction may be maintained temporarily or for a predetermined period of time.

[0050] In this way, the motion control unit 135 can associate the index corresponding to the reliability (I), the direction of arrival and reception strength of the radio wave, and the history of the control details of the autonomous mobile device 100, and store them in the storage unit 140. Taking into account the passage of time in the history, the motion control unit 135 can also associate the movement direction, the movement time or distance in that direction, and the reliability (I), and store them in the movement direction information storage unit 142. As described above, based on the information stored in the movement direction information storage unit 142, the motion control unit 135 can also calculate the past movement history and generate mapping information, allowing movement to avoid obstacles with low reliability (I) and complex spaces.

[0051] Furthermore, even if the radio wave intensity is too weak to estimate the direction of arrival of the radio wave using the angle estimation unit 134, the motion control unit 135 may maintain the current direction of movement. For example, if interference between the radiated and reflected radio waves creates a null point, the autonomous mobile device 100 may be able to re-estimate the direction of arrival of the radio wave by moving to another point.

[0052] Furthermore, the motion control unit 135 can also perform machine learning or deep learning using the movement history, angle information, estimated radio wave direction, and reliability (I), and store the machine learning and deep learning results in the storage unit 140. Therefore, the motion control unit 135 stores the reliability (I) and movement direction history as training data in the storage unit 140. Furthermore, the machine learning and deep learning results can also be associated with the movement direction, angle information, estimated radio wave direction, and reliability (I) and stored in the storage unit 140.

[0053] The obstacle avoidance unit 136 determines whether there is a possibility that the autonomous mobile device 100 has come into contact with an obstacle based on the sound wave signal received by the information acquisition unit 150. The information acquisition unit 150 transmits the sound wave signal to the obstacle avoidance unit 136. If the autonomous mobile device 100 is predicted to have come into contact with an obstacle based on the movement direction and size of the autonomous mobile device 100 and the obtained information about the obstacle, the obstacle avoidance unit 136 transmits the contact prediction information to the motion control unit 135. Furthermore, if the autonomous mobile device 100 is determined to have come into contact with an obstacle, the obstacle avoidance unit 136 transmits the contact information to the motion control unit 135. The motion control unit 135 determines the reliability (I) based on the contact prediction information or the contact information.

[0054] Storage unit 140 is a computer-readable storage medium. For example, storage unit 140 may be ROM (Read Only Memory) or EPROM (Erasable Programmable ROM). Alternatively, storage unit 140 may be EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), or a hard disk.

[0055] The storage unit 140 includes an angle information storage unit 141, a movement direction information storage unit 142, and a reception intensity information storage unit 143. The following describes the information stored in the storage unit 140 and the information processed by the control unit 130. The storage unit 140 also stores data on radio wave signals received by the receiving unit 110.

[0056] The angle information storage unit 141 stores the angle information of the radio wave in the direction of arrival estimated by the angle estimation unit 134. The angle information may be information relative to a predetermined reference axis determined based on the physical outline of the autonomous mobile device 100. For example, the outline may be represented by two-dimensional relative coordinates different from the space in which the autonomous mobile device 100 moves, with the line represented by these relative coordinates serving as the reference axis. The angle information may also be associated with the estimated radio wave reception strength and the time when the angle information was estimated.

[0057] The movement direction information storage unit 142 can store the actual movement direction of the autonomous mobile device 100, determined by the motion control unit 135, in association with information about the time when movement in that direction began and the time when movement in that direction ended. Furthermore, the time when movement in that direction began or ended, as well as the time information for movement in that direction, can also be associated with the movement direction information and stored in the movement direction information storage unit 142. The motion control unit 135 can also reconstruct the past movement paths of the autonomous mobile device 100 based on this information. To reach a target object, the motion control unit 135 can also select a path that avoids following the same movement path, using past movement paths as a reference. The obstacle avoidance unit 136 can also estimate the position of obstacles using past movement paths as a reference. The results of machine learning or deep learning, i.e., machine learning results and deep learning results, can also be stored in the storage unit 140, which includes the movement direction information storage unit 142. The machine learning results and deep learning results may be stored in association with the movement direction, angle information, radio wave estimation direction, and reliability (I) information.

[0058] The reception strength information storage unit 143 stores the reception strength of the radio waves received by the plurality of receiving elements determined by the reception strength determination unit 132. The reception strength of the radio waves in the direction of arrival of the radio waves estimated by the plurality of receiving elements is stored in the reception strength information storage unit 143. The reception strength is stored in the reception strength information storage unit 143 in association with the time information of the reception strength determination.

[0059] The drive unit 160 includes a mechanism for driving the moving unit 170 to move the autonomous mobile device 100 in the movement direction determined by the motion control unit 135. For example, if the moving unit 170 is a wheel, the drive unit 160 includes a mechanism for rotating the wheel; if the moving unit 170 is a track, the drive unit 160 includes a mechanism for rotating the track; and if the moving unit 170 is a propeller, the drive unit 160 includes a mechanism for rotating the propeller. Furthermore, the drive unit 160 is not limited to the above-described configuration and may include any other drive mechanism that drives the moving unit 170.

[0060] The moving unit 170 is a component that provides the means for moving the autonomous mobile device 100. If the autonomous mobile device 100 is a vehicle, the moving unit 170 may be wheels including tires or tracks. Alternatively, if the autonomous mobile device 100 is an aerial vehicle such as a drone or helicopter, the moving unit 170 may be a propeller. Furthermore, the moving unit 170 is not limited to the aforementioned configuration and may include any other moving mechanism that enables the autonomous mobile device 100 to move.

[0061] (Details of echolocation)

[0062] The information acquisition unit 150 may also be a device having one or more speakers and two or more microphones. Figure 4 , a device with one speaker and two microphones is described. Figure 4 This is a block diagram showing details of each component in the echolocation structure.

[0063] like Figure 4 As shown in FIG. 1 , as one embodiment, the information acquisition unit 150 includes one sound transmission unit 150C and two sound reception units 150L and 150R.

[0064] The sound transmitter 150C is mounted on the vehicle body of the autonomous mobile device 100 and transmits sound waves to an area including the front of the vehicle body (positive direction of the X axis). The sound transmitter 150C includes a speaker 41, an amplifier 42, and a D / A converter 43. Figure 4 In the description, a single sound transmitting unit 150C is used as an example, but a plurality of sound transmitting units 150C may be provided. For example, if the sound emitted from the sound transmitting unit 150C does not reach the entire periphery of the autonomous mobile device 100, two sound transmitting units 150C may be provided on the left and right sides of the autonomous mobile device 100.

[0065] Sound transmitter 150C outputs ultrasonic waves or sound waves with frequencies in the human audible band. Sound transmitter 150C can also be configured to output sound waves with frequencies other than ultrasonic waves or audible frequencies. "Sound waves" are a general term for elastic waves that propagate through gases, liquids, and solids.

[0066] The sound transmitter 150C outputs sound waves at a predetermined period or irregularly. It also has the function of changing the frequency of the sound waves it transmits. Specifically, when the frequency of the sound waves generated by the sound signal generator 26 (described later) is changed, the sound waves of the changed frequency are transmitted. If multiple sound transmitters 150C are provided, the timing of their sound wave output can be synchronized.

[0067] The D / A converter 43 converts a digital sound signal generated by the control unit 130 described later into an analog signal. Sound includes frequencies audible to humans, ultrasonic waves above audible frequencies, and ultra-low frequency sounds below audible frequencies. Sound is an example of a sound wave.

[0068] Amplifier 42 amplifies the analog sound signal. Furthermore, when an ultrasonic speaker is used as speaker 41, a digital rectangular wave can be directly output. In other words, a logic output rather than an analog output is possible, and a buffer circuit can be provided in place of D / A converter 43 and amplifier 42.

[0069] The speaker 41 outputs the analog sound signal amplified by the amplifier 42 as a sound wave. For example, the speaker 41 is positioned so as to face the straight-ahead direction of the autonomous mobile device 100 and output the sound wave in that direction. Specifically, the sound transmitter 150C outputs the sound wave in a single direction (e.g., the straight-ahead direction) serving as a reference for the autonomous mobile device 100. Furthermore, as long as the range in which the speaker 41 outputs the sound wave includes the straight-ahead direction of the autonomous mobile device 100, the central axis of the speaker 41 may not be aligned with the straight-ahead direction (forward) of the autonomous mobile device 100. Hereinafter, the "straight-ahead direction" may be referred to as the forward direction.

[0070] The sound receiving unit 150L and the sound receiving unit 150R are mounted on the body of the autonomous mobile device 100. They receive sound waves reflected by objects around the autonomous mobile device 100 and convert them into electrical signals. The sound receiving unit 150L (sound wave receiving unit) is positioned to the left of the front of the autonomous mobile device 100. The sound receiving unit 150R (sound wave receiving unit) is positioned to the right of the front of the autonomous mobile device 100. In other words, two sound receiving units are arranged symmetrically with respect to a single reference direction (e.g., the front) of the autonomous mobile device 100.

[0071] One sound receiving unit 150L receives sound waves from the left side of the front of the autonomous mobile device 100. The other sound receiving unit 150R receives sound waves from the right side of the front of the autonomous mobile device 100. The two sound receiving units 150L and 150R are multiple sound wave receiving units with different sound wave input directions. Each sound receiving unit 150L and 150R includes a microphone 51L or 51R and an A / D converter 53L or 53R.

[0072] Microphones 51L and 51R receive sound waves reflected by objects and convert them into sound signals as electrical signals. The left microphone 51L is positioned so as to face leftward, for example, 30 degrees relative to the front of the autonomous mobile device 100. The right microphone 51R is positioned so as to face rightward, for example, 30 degrees relative to the front of the autonomous mobile device 100.

[0073] In addition, each microphone 51L, 51R is configured to clamp the speaker 41 in the left and right directions perpendicular to the front of the autonomous mobile device 100. In other words, the speaker 41 can be located between the microphones 51L, 51R in the vehicle width direction. The direction of the microphones 51L, 51R is not limited to an angle as long as it is between the front and the left and right directions of the autonomous mobile device 100. The two microphones 51L, 51R can also be set in different directions. In the case where the autonomous mobile device 100 is a drone or the like that moves in a three-dimensional space, for example, it is also possible to set the sound receiving unit to four parts of the autonomous mobile device 100, namely, the left and right, and the top and bottom. In this case, it is preferable that the microphones are set at four parts, namely, the top and bottom, and the left and right, on the front side of the autonomous mobile device 100.

[0074] The A / D converters 53L and 53R digitize the analog audio signals output from the microphones 51L and 51R, respectively, and output the digitized signals to the control unit 130 .

[0075] The storage unit 140 includes an echo signal storage unit 31 and a control result storage unit 33 .

[0076] The echo signal storage unit 31 stores echo signals measured by the echo signal measurement unit 21 (described later). Here, "echo" (echo signal) refers to the phenomenon in which sound is reflected off a surface and heard again. This concept also includes "reverberation," which occurs when sound continues to be heard after the sound source stops vibrating due to repeated reflections from the ceiling, walls, and other sources.

[0077] The obstacle avoidance unit 136 calculates the movement direction and reliability (I) for avoiding obstacles and complex spaces based on the echo signals received by the sound receiving units 150R and 150L and the various movement control algorithms described below, and transmits these as control signals to the motion control unit 135. The obstacle avoidance unit 136 includes an echo signal measurement unit 21, a movement direction setting unit 24, a reliability determination unit 25, and a sound signal generation unit 26.

[0078] The audio signal generating unit 26 generates an audio signal of a predetermined frequency, and outputs the generated audio signal to the audio transmitting unit 150C at predetermined time intervals (for example, every 1 second).

[0079] The sound signal generator 26 changes the frequency of the sound signal as needed. For example, if a mobile device other than the autonomous mobile device 100 transmits a sound signal, and the frequency of the sound signal is similar to or identical to the frequency of the sound signal transmitted by the sound transmitter 150C of the autonomous mobile device 100, the sound signal generator 26 changes the frequency so that the frequency of the sound signal transmitted from the sound transmitter 150C differs from that of the sound signal transmitted from the other mobile device.

[0080] The echo signal measurement unit 21 receives the sound signals output from the A / D converters 53L and 53R, and transfers the sound signals to the echo signal storage unit 31 and the movement direction setting unit 24 .

[0081] If an obstacle exists in the direction of movement of the autonomous mobile device 100, the movement direction setting unit 24 analyzes the sound signal forwarded from the echo signal measurement unit 21 and the data stored in the echo signal storage unit 31 and the control result storage unit 33 to set the movement direction of the autonomous mobile device 100. The movement direction setting unit 24 calculates movement information such as the rotation direction, rotation angle, and movement speed of the autonomous mobile device 100. Furthermore, if there are multiple arrival directions of the echo signal, the movement direction setting unit 24 does not set a movement direction but instead provides various information such as the multiple arrival directions to the reliability determination unit 25.

[0082] Reliability determination unit 25 determines reliability (I) based on various information received from movement direction setting unit 24 and outputs various control signals, including reliability, to motion control unit 135. The control signals include not only reliability (I) but also information related to driving, such as movement direction, rotation direction, rotation angle, and travel speed.

[0083] The reliability determination unit 25 outputs the control signal output to the drive unit 160 to the control result storage unit 33. The control result storage unit 33 stores the control signal output from the reliability determination unit 25.

[0084] Next, a description will be given of a method for setting the moving direction of the autonomous mobile device 100 by the moving direction setting unit 24 when avoiding an obstacle.

[0085] (1st setting method)

[0086] The moving direction setting unit 24 sets the moving direction of the autonomous mobile device 100 based on the received echo signal. The moving direction setting unit 24 outputs information on the set moving direction to the reliability determination unit 25.

[0087] For example, when the autonomous mobile device 100 Figure 2 When autonomously traveling from position p0 to destination P1, if there is an obstacle p2 on the shortest path x0, the autonomous mobile device 100 changes its direction of travel at position P2 in front of the obstacle p2 to avoid the obstacle p2. In this case, the path x2 to the right of the obstacle p2 is a complex space, and the obstacle will significantly restrict the autonomous mobile device 100 during travel. Therefore, it is better to change the direction of travel of the autonomous mobile device 100 to the left. Figure 4The obstacle avoidance unit 136 determines surrounding obstacles and complex spaces, calculates reliability (I), and provides a control signal including the reliability (I) to the motion control unit 135 .

[0088] (Second setting method)

[0089] The moving direction setting unit 24 acquires the echo signals received in the past and stored in the echo signal storage unit 31 , and the control signals in the past and stored in the control result storage unit 33 .

[0090] The movement direction setting unit 24 performs machine learning on each of the left and right sound receiving units 150L and 150R based on past control signals output by the reliability determination unit 25. By performing machine learning, the movement direction setting unit 24 obtains a correlation between the echo signal and the movement direction of the autonomous mobile device 100 when avoiding an obstacle. Machine learning is a well-known technique, and therefore a detailed description thereof will be omitted.

[0091] The movement direction setting unit 24 sets the optimal movement direction of the autonomous mobile device 100 based on the acquired correlation, and the reliability determination unit 25 determines the reliability (I) based on the machine learning result.

[0092] The reliability determination unit 25 uses machine learning results based on past control performance to determine movement information such as the autonomous mobile device 100's movement direction, rotation direction, rotation angle, and movement speed, and outputs a control signal along with the reliability (I) to the motion control unit 135. As a result, the autonomous mobile device 100 can select a more open travel path to avoid obstacles.

[0093] According to the above structure, SLAM (Simultaneous Localization and Mapping) in the autonomous mobile device 100 does not require expensive equipment such as LiDAR (Light Detection and Ranging), and a simple structure can be used to reduce manufacturing costs. In addition, when the autonomous mobile device 100 is introduced to a new location, or whenever the layout of an existing location is changed, there is no need to create a map of the location or layout, which can also reduce introduction costs. There is also no need to predetermine a travel route plan. There is no need to lay guidance tapes, magnetic rods, or QR codes on the ground as is required for guided automated guided vehicles (Automatic Guided Vehicles (AGVs)). There is no need for the massive data processing and expensive computers that accompany it, as is required for autonomous mobile robots (Autonomous Mobile Robots (AMRs)), and power consumption can be suppressed.

[0094] The autonomous mobile device 100 may also be equipped with a proximity sensor to prevent contact with obstacles in the immediate vicinity (e.g., within 50 cm), or a bumper sensor or contact sensor to detect collision with an obstacle. Furthermore, when the autonomous mobile device 100 is used in an automated guided vehicle (AGV), it is essential that it include functions that meet the requirements of ISO 3691-4 / JIS D 6802, "Automated Guided Vehicles and AGV Systems - Safety Requirements and Verification," regarding AGV safety.

[0095] (Improvement of autonomous mobile device movement using camera images)

[0096] Next, the improvement of the movement of the autonomous mobile device 100 using the camera image will be described. Figures 1 to 4 As described, the autonomous mobile device 100 autonomously moves toward the target object 200 while avoiding obstacles based on the radio wave signals received by the receiving unit 110, the echo signals (an example of acoustic signals) received by the information acquisition unit 150, and the motion control algorithm of the control unit 130. Because radio wave signals and acoustic wave signals are wave signals, it is difficult to clearly understand the shape of obstacles based solely on these signals. Therefore, it is not easy to analyze the movement results of the autonomous mobile device 100 based solely on the radio wave signals and acoustic wave signals recorded in the storage unit 140 and to feed the analysis results back to the motion control algorithm executed by the control unit 130.

[0097] Therefore, in the embodiments, an autonomous mobile device 100, an autonomous mobile improvement system, and a method for improving the movement of the autonomous mobile device 100 are described, which accurately improve the movement of the autonomous mobile device 100 by combining images captured by the camera 180 mounted on the autonomous mobile device 100 with data of at least one of radio wave signals and acoustic wave signals. The autonomous mobile device 100 can use the camera 180 to obtain images of the surroundings of the autonomous mobile device 100. The camera 180 and the images from the camera 180 are used to improve the movement of the autonomous mobile device 100, but are not used by the control unit 130 for controlling the movement of the autonomous mobile device 100, nor do they constitute part of the movement control algorithm executed by the control unit 130.

[0098] (Example 1)

[0099] In the first embodiment, the following embodiment is described: in an environment where the autonomous mobile device 100 operates independently without being connected to the network 10 or other devices, an improvement in the mobile control algorithm can be performed. Figure 3As shown, the autonomous mobile device 100 according to Example 1 includes: a receiving device for receiving signals; a control unit 130 for moving the autonomous mobile device 100; a camera 180 for capturing images of the surroundings of the autonomous mobile device 100; and a movement improvement unit 190. The receiving device for receiving signals includes a receiving unit 110 for receiving radio signals and an information acquisition unit 150 for receiving acoustic signals. The signals include radio and acoustic signals. The control unit 130 moves the autonomous mobile device 100 based on the radio and acoustic signals received by the receiving device and an autonomous movement algorithm. The movement improvement unit 190 modifies the autonomous movement algorithm based on data from at least one of the radio and acoustic signals in the signals received by the receiving device and the images captured by the camera 180.

[0100] It is difficult to determine where the autonomous mobile device 100 is having trouble moving smoothly, or what improvements should be made, simply by looking at the radio and acoustic signals that represent movement results. By combining the images captured by the camera 180 with the radio and acoustic signals, the shape of obstacles can be clearly determined from the images. This allows for easy and accurate analysis of movement results, enabling precise improvement of the movement control algorithm.

[0101] The mobility improvement unit 190 changes the autonomous mobility algorithm by machine learning using data of at least one of radio wave signals and sound wave signals and images as input data. Machine learning includes deep learning. The mobility improvement unit 190 can use a known machine learning method. According to Example 1, the autonomous mobility device 100 itself can improve the autonomous mobility algorithm internally. Different autonomous mobility algorithms can be improved for each environment in which the autonomous mobility device 100 is used or for each autonomous mobility device 100. Therefore, the autonomous mobility algorithm of the autonomous mobility device 100 can be customized for each user or for each usage environment. The autonomous mobility device 100 and its mobility improvement method of Example 1 can be implemented even in an independent operation environment that operates independently without being connected to a network or other machines. The mobility improvement unit 190 can use, for example, an AI chip specifically for machine learning or deep learning.

[0102] Reference Figure 6 , an example of the flow of data in the autonomous mobile device 100 involved in Example 1 is described. The autonomous mobile device 100 includes a camera 180, a storage unit 140, an obstacle avoidance unit 136, a motion control unit 135, a destination position detection unit 70, and a movement improvement unit 190. Figure 3As shown, the destination position detection unit 70 is a functional block that executes the movement control algorithm related to the digital pheromone in the control unit 130. Specifically, the destination position detection unit 70 includes a phase difference determination unit 131, a reception strength determination unit 132, a reception element selection unit 133, and an angle estimation unit 134. Figure 6 The movement control of the autonomous mobile device 100 is completed by the obstacle avoidance unit 136, the destination position detection unit 70, and the action control unit 135. The camera 180 is not used in the movement control of the autonomous mobile device 100.

[0103] The obstacle avoidance unit 136 stores at least the acoustic signal in the storage unit 140 as a result of movement control. The obstacle avoidance unit 136 may also store echolocation data in the storage unit 140, including reliability (I) and control signals obtained by processing the acoustic signal. As a result of movement control, the destination position detection unit 70 stores at least the radio signal in the storage unit 140. The destination position detection unit 70 may also store digital pheromone data in the storage unit 140, including the radio signal reception intensity and radio angle information obtained by processing the radio signal. Furthermore, the motion control unit 135 may also store sensing data obtained by processing the radio signal and acoustic signal in the storage unit 140. Here, "sensing data" includes movement direction information, reliability (I) determination results, received signal strength, signal temporal variation, number of signal receptions, left-right comparison of signals, comparison with past historical records, estimated distance to obstacles, estimated shape of the space or path, signal noise level, stability of the arrival direction angle, an index corresponding to reliability (I), and mapping information.

[0104] The image captured by camera 180 is linked to the data stored in storage unit 140 by obstacle avoidance unit 136, destination position detection unit 70, and motion control unit 135. Specifically, the data and image of signals (radio and acoustic signals) received or captured at the same time are linked. Echolocation data, digital pheromone data, and sensory data obtained by processing the radio and acoustic signals are linked to the image using the time of receipt of the original radio and acoustic signals. This allows for comparison between the signals received and the captured image at the same time. The shape of obstacles, the shape of the passage, and the space surrounding the autonomous mobile device 100 at the time the signal was received can be clearly understood, enabling precise movement improvements.

[0105] The autonomous mobile device 100 is equipped with a camera 180 for capturing images of the surroundings of the autonomous mobile device 100. There may be one or more cameras 180. The camera 180 is configured to capture a predetermined angle of view including the direction of travel (front) of the autonomous mobile device 100. For example, the camera 180 is disposed near the front end of the autonomous mobile device 100 when viewed from above. Specifically, the camera 180 is disposed on the inner side of the anti-collision bumper. The camera 180 may be disposed in the same direction as the speaker 41. If a plurality of speakers 41 are provided, the same number of cameras 180 may be prepared and the speakers 41 may be paired with the cameras 180. The center line of the angle of view of the camera 180 may be horizontal or tilted downward, for example, by about 60 degrees. In the case of tilting downward, the camera 180 is installed at a higher position on the autonomous mobile device 100. Therefore, compared with the case where the camera 180 is installed with the center line of the viewing angle horizontally, a single camera can capture a wider range from the area around the anti-collision bumper in the direction of travel of the autonomous mobile device 100 to the distance in the direction of travel, for example, from the end of the direction of travel to about 1 m.

[0106] Camera 180 is a digital camera that uses an imaging element that converts light into electrical signals. Besides capturing two-dimensional images, camera 180 can also be a depth camera or a stereo camera capable of acquiring depth information. The captured images are output as data and stored in storage unit 140. The images are associated with at least one of a radio signal and an acoustic signal and stored in storage unit 140. Time information such as a timestamp can be used for binding. Specifically, the images and radio or acoustic signals are bound so that the time of reception of the radio or acoustic signal coincides with the time the image was captured. Information representing the radio or acoustic signal can be displayed in the image instead of a timestamp. The sensing results from control unit 130, vehicle control information, and the movement status of autonomous mobile device 100 can also be added to this binding. The sensing results include the radio signal's reception strength, the radio signal's arrival angle, reliability (I), the relative direction and distance to obstacles, contact prediction information, and contact information. The vehicle control information includes information indicating the movement direction and speed of autonomous mobile device 100. As a method of binding, the above-mentioned information (radio signal, sound wave signal, sensing result, vehicle control information, movement status) may be displayed together with the time information on the image instead of the time stamp.

[0107] (Example 2)

[0108] In Example 2, referring to Figure 5, an autonomous mobility improvement system is described in which one or more autonomous mobility devices 100 are connected to a server 80 via a network 10, and the server 80 improves the mobility control algorithm of the one or more autonomous mobility devices 100. The autonomous mobility improvement system includes one or more autonomous mobility devices 100 and a server 80 that is communicably connected to the one or more autonomous mobility devices 100. The various components of the autonomous mobility device 100 are as follows, except that the autonomous mobility device 100 does not have Figure 6 In addition to the mobile improvement unit 190, Figure 6 The structure of the autonomous mobile device 100 is the same, so the description is omitted.

[0109] The server 80 includes a driving analysis unit 81 and a program update unit 82. The server 80 receives data on radio signals and acoustic signals received by each autonomous mobile device 100, as well as images captured by the camera 180. The data on at least one of the radio signals and acoustic signals, and the images stored in the storage unit 140 of each autonomous mobile device 100, are uploaded to the server 80 via the network 10 in a bounded state. The data on at least one of the radio signals and acoustic signals, and the images, transmitted from two or more autonomous mobile devices 100 to the server 80 constitutes big data. The manufacturer of the autonomous mobile device 100 or the provider of the autonomous mobile improvement service identifies, from this big data, data on at least one of the radio signals and acoustic signals, and the images of autonomous mobile devices 100 under predetermined movement conditions, which are the subject of mobility improvement. Specifically, the control history data and images of the autonomous mobile device 100 when it was unable to move smoothly are extracted as the control history data and images for improvement.

[0110] The driving analysis unit 81 analyzes the movement of the autonomous mobile device 100 based on the control history data and images of the improvement object. The specific analysis method will be described as a specific example of the movement improvement described later. The program update unit 82 updates the autonomous mobile program as an example of the autonomous mobile algorithm based on the analysis results. The updated autonomous mobile program is downloaded from the server 80 to the action control unit 135, the obstacle avoidance unit 136, and the destination position detection unit 70 respectively. In this way, the autonomous mobile improvement system of Example 2 can simultaneously perform movement improvements that are common to all autonomous mobile devices 100 based on big data (signal data and images) collected from an unspecified number of autonomous mobile devices 100. The autonomous mobile improvement system can centrally and efficiently perform movement improvements that are common to one or more autonomous mobile devices 100 in the server 80.

[0111] Furthermore, in Example 2, the manufacturer of the autonomous mobility device 100 or the provider of the autonomous mobility improvement service, i.e., a person, can also compare signal data and images to identify control history data and images under the specified motion state to be improved from the big data. Alternatively, the server 80 can use image recognition technology to identify control history data and images. Furthermore, similar to Example 1, the server 80 can also use an AI chip specialized for machine learning or deep learning to identify signal data and images under the specified motion state to be improved and perform mobility improvement.

[0112] (Specific example of mobility improvement)

[0113] Next, examples of predetermined operating states of the autonomous mobile device 100 to be improved and examples of improving the movement control algorithm in the predetermined operating states will be described.

[0114] <Collision with obstacles>

[0115] As an example of a predetermined operating state of the autonomous mobile device 100 that is the target of improvement, there is a collision with an obstacle. It is difficult to accurately grasp the shape of the obstacle using only radio waves and sound waves, so the autonomous mobile device 100 may collide with (including contact with) the obstacle. First, as Figure 7 As shown, in step S01, data and images of at least one of radio wave signals and sound wave signals are obtained from the storage unit 140. The data of at least one of radio wave and sound wave are sometimes referred to as "signal data". Proceeding to step S02, the data and images of the signals before and after the autonomous mobile device 100 collides with the obstacle are determined as the data and images of the signals in the specified moving state. The collision detection with the obstacle can use an anti-collision bumper equipped with a bumper sensor. Alternatively, the processing of step S02 can be performed by a person based on the image, or can be performed by the mobility improvement unit 190 or the server 80 using image processing technology. It is also possible to further determine the sensing data obtained as a result of processing the signal data, the sensing result of the control unit 130, and the vehicle control information that are bound to the determined signal data.

[0116] Proceeding to step S03, the autonomous mobility algorithm is modified using the determined signal data and image. As an example of modifying the autonomous mobility algorithm, at least one of the threshold and coefficient used in determining whether the autonomous mobility device 100 avoids a collision with an obstacle may be modified. For example, signal data and images are determined for 10 seconds before and after the collision. The collision with a thin pole (obstacle) can be determined from the image or the detection results of the bumper sensor. At least one of the threshold and coefficient used in echolocation or digital pheromone detection is modified to enable identification of the thin pole using sound or radio waves based on its shape, such as its width. For example, by lowering the threshold for obstacle detection, a thin pole with a low intensity of reflected sound waves can be detected. Alternatively, at least one of the threshold and coefficient may be modified to increase the echolocation sensing frequency. Specifically, the sensing cycle may be shortened without changing the movement speed, the sensing cycle may be shortened without changing the movement speed, or the sensing cycle may be shortened while the movement speed is reduced. This enables detection of thin poles using radio or sound waves.

[0117] <Crowd Status>

[0118] Another example of a predetermined operating state for the autonomous mobile device 100 that is the target of improvement is a crowd state. In step S02, the signal data and image in the crowd state are determined using images as the signal data and image in the predetermined movement state. A state in which a predetermined number of people or more are located around the autonomous mobile device 100 is referred to as a "crowd state." For example, the signal data and image for 10 seconds before and after the autonomous mobile device 100 enters a crowd are determined.

[0119] In step S03, the autonomous movement algorithm can be changed to avoid the crowd using the signal data and images in the crowd state. First, the signal data before entering the crowd is compared with the signal data after entering the crowd, that is, in the crowd state. In this way, the noise component caused by people contained in the signal data can be determined. Various radio wave signals generated by people or electronic devices carried by people, as well as sound wave signals such as conversations and footsteps, are equivalent to the noise component caused by people. In response to receiving a signal with a noise component caused by people, the movement control algorithm of the control unit 130 is changed to determine a direction different from the arrival direction of the signal with the noise component caused by people as the movement direction of the autonomous movement device 100. In this way, the autonomous movement algorithm can be changed to avoid the crowd.

[0120] Alternatively, in step S03, the autonomous movement algorithm can be modified using the signal data and images from the crowd situation to enable movement through the crowd. First, similar to the avoidance scenario, the noise component caused by humans contained in the signal data is determined. If the noise component caused by humans in the received acoustic signal exceeds a certain value, the control unit 130's movement control algorithm is modified so that, for a certain period thereafter, movement control is performed based solely on the radio signal, not the acoustic signal. In other words, the movement control algorithm is modified to disable the echolocation function and perform movement control solely using the digital pheromone function and collision detection function. If the noise component caused by humans exceeds a certain value, the autonomous movement algorithm can be modified to remove the human noise component from the signal data. If the noise component caused by humans exceeds a certain value, the autonomous movement algorithm can be modified to reduce the coefficient of contribution of the signal containing the human noise component to autonomous movement. Alternatively, if the noise component caused by humans exceeds a certain value, the autonomous movement algorithm can be modified to perform movement control using the signal data before the human noise component exceeded the certain value. In this way, the autonomous movement algorithm can be changed so that the autonomous movement device 100 can pass through the crowd.

[0121] <State where a passable route is not passed>

[0122] Another example of a predetermined operating state of the autonomous mobile device 100 that is a target for improvement is a state in which a traversable passage is not possible. For example, if a device emitting sound or radio waves is installed at the entrance of a traversable passage, the autonomous mobile device 100 may mistakenly believe that the passage is impassable and choose another route, resulting in a longer detour to the target object 200.

[0123] Therefore, in step S02, the image is used to determine the state in which the otherwise traversable passage is not passed. Specifically, the signal data and image are determined when the otherwise traversable passage is not passed. For example, a person can observe the image while determining the scene in which the otherwise traversable passage is not passed. In step S03, the autonomous movement algorithm is modified to allow the passage to be passed. For example, if a sound wave different from that emitted by the autonomous movement device 100 is received, or a radio wave with a frequency different from that emitted by the target object 200 is received, the autonomous movement algorithm can be modified to perform movement control without considering these unnatural signals. Furthermore, if a sound wave different from that emitted by the autonomous movement device 100 is received, the autonomous movement algorithm can be modified to disable the echolocation function and perform movement control using only the radio wave signal and collision avoidance function. This allows the passage to be passed. The autonomous movement device 100 will be able to move in a room or passage equipped with a device that outputs sound waves.

[0124] If the width of the passage is too narrow relative to the width of the autonomous mobile device 100, that is, if the autonomous mobile device 100 could barely pass through the passage, the passage would be deemed impassable, even though it should be considered traversable. However, since the shape of the obstacle cannot be clearly identified using only sound and radio waves, the passage is deemed impassable. In step S02, the image is used to determine that the passage was not traversable. Specifically, a person observes the image and estimates the width of the passage, identifying a scenario in which the autonomous mobile device 100 failed to pass through the barely traversable passage. In step S03, the autonomous mobile device 100's sensitivity (parameter) to obstacles is reduced to enable the passage. This modification of the autonomous mobile algorithm enables the autonomous mobile device 100 to autonomously pass through the barely traversable passage.

[0125] When the autonomous mobile device 100 passes through an automatic door, it may mistakenly believe it cannot pass through a location that it should have considered passable. The autonomous mobile device 100 detects the closed automatic door from a distance, mistaking it for an impassable obstacle. Subsequently, even if the autonomous mobile device 100 approaches the automatic door and it opens, it will mistake it for an obstacle and circumvent it. This scenario may occur if the autonomous mobile device 100's sensing frequency is low.

[0126] Therefore, by increasing the sensing frequency, the autonomous mobile device 100 can detect an automatic door in an open state and identify the automatic door as a passable passage. Therefore, the autonomous mobile device 100 can pass through the automatic door. In step S02, the image is used to determine the movement state that has not passed through the open automatic door. In step S03, the autonomous mobile algorithm is changed to pass through the open automatic door. For example, at least one of the judgment threshold or coefficient is changed to increase the frequency of sensing obstacles. Specifically, around the automatic door, the sensing cycle is shortened without changing the movement speed, the movement speed is reduced without changing the sensing cycle, or the movement speed is reduced while shortening the sensing cycle. As a result, the autonomous mobile device 100 will be able to detect and pass through the open automatic door.

[0127] If the autonomous mobile device 100 is facing a highly flat surface, the reflected wave received by the autonomous mobile device 100 from that surface becomes stronger. If the angle of the surface relative to the autonomous mobile device 100 changes slightly, the intensity of the reflected wave decreases dramatically. The autonomous mobile device 100 may sometimes temporarily react sensitively to strong signals, interpreting them as large obstacles. Therefore, the autonomous mobile algorithm is modified to increase the sensing frequency. This allows for identification of small obstacles, enabling robust movement. Alternatively, the autonomous mobile algorithm can be modified to use a moving average of time-series data, including radio and acoustic signals, for movement control.

[0128] <State Not Approaching Target Object 200>

[0129] As another example of a prescribed operational state of the autonomous mobile device 100 that is an object of improvement, there is a state in which the autonomous mobile device 100 is not close to the target object 200. In step S02, data and images of signals in a state in which the autonomous mobile device 100 is not close to the target object 200 are determined. Specifically, when the temporal variation in the intensity of the radio wave signal received from the target object 200 is less than a specific value, the temporal variation in the distance from the autonomous mobile device 100 to the target object 200 is also small. The state of not passing through a passage that could have been passed through is determined using an image, but the state of not approaching the target object 200 can be determined by methods other than images. Therefore, the autonomous mobile algorithm can be improved to have Figure 6 The autonomous mobility device 100 of the mobility improvement unit 190 is shown approaching a target object 200 .

[0130] In step S02, the state of wandering around the entrance of a specific passage (wandering state) is determined. The signal data and image obtained in step S01 are used to determine the wandering state. This determination can be made using an image or based on the temporal changes in the intensity of the radio signal received from target object 200.

[0131] In step S03, machine learning is performed using the data from the signal and image during the wandering state as input data to modify the movement control algorithm. For example, the speed may be reduced when traveling in front of a specific passage. This enables movement control using more detailed sensory data, thereby improving the autonomous movement algorithm to approach target object 200 via a specific passage.

[0132] The above embodiment is an example of the present invention. Therefore, the present invention is not limited to the above embodiment, and various changes can be made according to the design etc. even in forms other than the embodiment as long as they do not depart from the scope of the technical concept involved in the present invention.

[0133] (Note 1)

[0134] A method for improving the movement of an autonomous mobile device 100 that moves autonomously based on received signals and an autonomous movement algorithm obtains signal data received by the autonomous mobile device 100 and images captured by a camera 180 mounted on the autonomous mobile device 100, identifies the signal data and images in a predetermined movement state of the autonomous mobile device 100, and modifies the autonomous movement algorithm of the autonomous mobile device 100 based on the identified signal data and images. By combining the images captured by the camera 180 with the signal data, the shape of obstacles, etc., can be clearly determined from the images. This makes it possible to easily and accurately analyze movement results and accurately improve the movement control algorithm.

[0135] (Note 2)

[0136] The method for improving the movement of the autonomous mobile device 100 described in Supplementary Note 1 includes determining signal data and images before and after the autonomous mobile device 100 collides with obstacles J1, J2, and p1-p4 as signal data and images in a predetermined movement state, and using the determined signal data and images to change at least one of a threshold value and a coefficient used in determining whether the autonomous mobile device 100 avoids collisions with obstacles J1, J2, and p1-p4s. This method can accurately improve the autonomous movement algorithm so that collisions with obstacles J1, J2, and p1-p4s can be avoided.

[0137] (Note 3)

[0138] The method for improving the movement of the autonomous mobile device 100 described in Supplementary Note 1 includes using an image to determine signal data and images in a crowd state in which a predetermined number of people or more are located around the autonomous mobile device 100 as signal data and images in a predetermined movement state, and using the signal data and images in the crowd state to modify the autonomous movement algorithm to avoid the crowd. This method enables accurate improvement of the autonomous movement algorithm to avoid the crowd state.

[0139] (Note 4)

[0140] The method for improving the mobility of the autonomous mobility device 100 as described in Supplementary Note 1 includes using an image to determine signal data and images of a crowd state in which a predetermined number of people or more are located around the autonomous mobility device as signal data and images of a predetermined mobility state, and using the signal data and images of the crowd state to modify the autonomous mobility algorithm so as to remove noise components caused by people from the signal data or to reduce the coefficient of contribution of the signal containing noise components caused by people to the autonomous mobility. The autonomous mobility algorithm can be accurately improved to enable the autonomous mobility device 100 to navigate the crowd state.

[0141] (Note 5)

[0142] The method for improving the movement of the autonomous movement device 100 as described in Supplementary Note 1 includes determining the signal data and image in a state where the autonomous movement device 100 is not passing through a traversable path as the signal data and image in a predetermined movement state, and using the signal data and image in a state where the autonomous movement device 100 is not passing through a traversable path to modify the autonomous movement algorithm so that the autonomous movement device 100 can pass through the path. This method enables accurate improvement of the autonomous movement algorithm so that the autonomous movement device 100 can pass through a traversable path.

[0143] (Note 6)

[0144] The method for improving the movement of the autonomous mobile device 100 as described in Supplementary Note 1 includes determining the signal data and image when the autonomous mobile device 100 is not approaching the target object 200 as the signal data and image when the autonomous mobile device 100 is in a predetermined movement state, and using the signal data and image when the autonomous mobile device 100 is not approaching the target object 200, modifying the autonomous movement algorithm so as to approach the target object 200. This makes it possible to accurately improve the autonomous movement algorithm so as to approach the target object 200.

[0145] (Note 7)

[0146] As described in Supplementary Note 6, the method for improving the movement of the autonomous mobile device 100 uses machine learning using signal data and images when the autonomous mobile device 100 is not approaching the target object 200 as input data to modify the autonomous movement algorithm so as to approach the target object 200. The autonomous mobile device 100 itself can improve the autonomous movement algorithm internally so as to approach the target object 200.

[0147] (Note 8)

[0148] In the method for improving mobility of the autonomous mobile device 100 as described in any one of Supplementary Notes 1 to 7, the signal data and image to be determined are signal data and image received or captured at the same time. Since the image and signal data captured when the signal was received can be clearly understood, accurate mobility improvement can be performed.

[0149] (Note 9)

[0150] The method for improving the mobility of an autonomous mobile device 100 as described in any one of Supplementary Notes 1 to 8, wherein signal data and images are obtained from a plurality of autonomous mobile devices 100, can centrally and efficiently perform mobility improvement commonly applicable to two or more autonomous mobile devices 100 in the server 80.

[0151] (Note 10)

[0152] The autonomous mobility device 100 includes a receiving device that receives signals; a control unit 130 that moves the autonomous mobility device 100 based on the signals received by the receiving device and an autonomous mobility algorithm; a camera 180 that captures images of the surroundings of the autonomous mobility device 100; and a movement improvement unit 190 that modifies the autonomous mobility algorithm based on data from the signals received by the receiving device and images captured by the camera 180. By combining the images captured by the camera 180 with the signal data, the shape of obstacles, etc., can be clearly determined from the images. This allows for easy and accurate analysis of movement results and precise improvement of the movement control algorithm.

[0153] (Note 11)

[0154] In the autonomous movement device 100 described in Supplementary Note 10, the movement improvement unit 190 modifies the autonomous movement algorithm by machine learning using signal data and images as input data. The autonomous movement device 100 itself can improve the autonomous movement algorithm internally.

[0155] (Note 12)

[0156] The autonomous mobility improvement system includes one or more autonomous mobility devices 100 and a server 80 communicatively connected to the one or more autonomous mobility devices 100 via a network 10. Each autonomous mobility device 100 includes a receiving device for receiving signals, a control unit 130 for moving the autonomous mobility device 100 based on the signals received by the receiving device and an autonomous mobility algorithm, and a camera 180 for capturing images of the surroundings of the autonomous mobility device 100. The server receives data from the signals received by the receiving device and images captured by the camera 180 from the one or more autonomous mobility devices 100, modifies the autonomous mobility algorithm based on the data from the signals received by the receiving device and the images captured by the camera 180, and transmits the modified autonomous mobility algorithm to the one or more autonomous mobility devices 100. The autonomous mobility improvement system enables the server 80 to efficiently and centrally perform mobility improvements common to the two or more autonomous mobility devices 100.

[0157] (Note 13)

[0158] In the autonomous mobility improvement system described in Supplementary Note 12, the server 80 modifies the autonomous mobility algorithm through machine learning using signal data and images as input data. The server 80 can improve the autonomous mobility algorithm internally.

[0159] Description of Reference Numerals

[0160] 10 Network; 80 Server; 100 Autonomous mobile device; 110 Receiving unit (receiving device); 150 Information acquisition unit (receiving device); 180 Camera; 190 Mobility improvement unit; J1, J2, p1-p4 obstacles.

Claims

1. A method for improving the movement of an autonomous mobile device, wherein the method comprises: acquiring data of the signal received by the autonomous mobile device and an image captured by a camera mounted on the autonomous mobile device, determining the data of the signal and the image in a prescribed movement state of the autonomous mobile device, An autonomous movement algorithm of the autonomous movement device is modified based on the determined data of the signal and the image.

2. The method for improving mobility of an autonomous mobile device according to claim 1, wherein: determining the signal data and the image before and after the autonomous mobile device collides with an obstacle as the signal data and the image in the predetermined movement state; At least one of a threshold value and a coefficient used in determining whether to avoid a collision between the autonomous mobile device and an obstacle is changed using the determined signal data and image.

3. The method for improving the mobility of an autonomous mobile device according to claim 1, wherein: using the image, determining the signal data and the image in a crowd state in which a predetermined number of persons or more are located around the autonomous mobile device as the signal data and the image in the predetermined movement state, Using the data of the signal in the crowd state and the image, the autonomous movement algorithm is modified to avoid the crowd.

4. The method for improving the mobility of an autonomous mobile device according to claim 1, wherein: Using the image, the signal data and the image in a crowd state in which a predetermined number of people or more are located around the autonomous moving vehicle are determined as the signal data and the image in the predetermined moving state, Using the data of the signal in the crowd state and the image, the autonomous movement algorithm is changed to remove the noise component caused by humans from the signal data, or to reduce the coefficient of contribution of the signal with the noise component caused by humans to autonomous movement.

5. The method for improving the mobility of an autonomous mobile device according to claim 1, wherein: determining the signal data and the image in a state where the autonomous mobile device does not pass through a passable path as the signal data and the image in the predetermined movement state, The autonomous movement algorithm is modified so as to pass through the passable path using the data of the signal in the state where the passable path is not passed and the image.

6. The method for improving the mobility of an autonomous mobile device according to claim 1, wherein: determining the signal data and the image in a state where the autonomous mobile device is not approaching a target object as the signal data and the image in the predetermined movement state, The autonomous movement algorithm is modified so as to approach the target object using the signal data and the image in a state where the target object is not approached.

7. The method for improving the mobility of an autonomous mobile device according to claim 6, wherein: The autonomous movement algorithm is modified so as to approach the target object by machine learning using the data of the signal and the image when the autonomous movement device is not approaching the target object as input data.

8. The method for improving the mobility of an autonomous mobile device according to any one of claims 1 to 7, wherein: The determined signal data and image are the signal data and image received or captured at the same time.

9. The method for improving the mobility of an autonomous mobile device according to any one of claims 1 to 7, wherein: The signal data and the image are obtained from a plurality of autonomous mobile devices.

10. An autonomous mobile device comprising: a receiving device for receiving a signal; a control unit configured to move the autonomous movement device based on the signal received by the receiving device and an autonomous movement algorithm; a camera, for capturing images of the surroundings of the autonomous mobile device; as well as The movement improvement unit changes the autonomous movement algorithm based on the data of the signal received by the receiving device and the image captured by the camera.

11. The autonomous mobile device according to claim 10, wherein: The movement improvement unit changes the autonomous movement algorithm through machine learning using the signal data and the image as input data.

12. An autonomous mobility improvement system comprising one or more autonomous mobility devices and a server communicably connected to the one or more autonomous mobility devices via a network. Each of the autonomous mobile devices comprises: a receiving device for receiving a signal; a control unit configured to move the autonomous movement device based on the signal received by the receiving device and an autonomous movement algorithm; as well as a camera for capturing the surroundings of the autonomous mobile device, The server receiving the data of the signal received by the receiving device and the image captured by the camera from the one or more autonomous mobile devices, Based on the data of the signal received by the receiving device and the image captured by the camera, the autonomous movement algorithm is changed. The modified autonomous movement algorithm is sent to the one or more autonomous movement devices.

13. The autonomous mobility improvement system according to claim 12, wherein: The server changes the autonomous movement algorithm through machine learning using the signal data and the image as input data.

Images