Autonomous Mobile Device

Abstract

To provide an autonomous moving device which correctly recognizes a landmark that has a state changing, and which correctly estimates the own location.SOLUTION: An autonomous moving device includes: a ceiling camera that images a landmark; an environmental sensor that obtains a surrounding situation; an internal sensor that detects information on the own position; and an own position estimating device. The own position estimating device includes: a landmark detection processing unit that detects the information on the landmark; a storing unit that stores the information on the landmark and environmental map data; a landmark state determining unit which determines the state of the landmark on the basis of the detected information on the landmark and of the stored information on the landmark, and which calculates the positional information on the landmark having undergone weighting in accordance with the detection precision of the information on the landmark; and an own position estimating unit based on the positional information by the internal sensor, the surrounding situation in the travelling direction obtained by the environmental sensor, the environmental map data stored by the storing unit, and the weighted positional information on the landmark calculated by the landmark state determining unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an autonomous mobile device that detects landmarks using a ceiling camera facing the ceiling and estimates its own position based on the landmark information. [Background technology]

[0002] 2. Description of the Related Art Autonomous mobile devices, such as robots, have been developed that are equipped with external and internal sensors, use these external and internal sensors to estimate their own position, and autonomously move to a destination.

[0003] In particular, an autonomous mobile device has been proposed that estimates its own position using an external sensor such as a ceiling camera facing the ceiling so that it can stably detect landmarks even when the autonomous mobile device is surrounded by people or the like.

[0004] Background art in this technical field includes, for example, Japanese Patent Application Laid-Open No. 2005-242409 (Patent Document 1) and Japanese Patent Application Laid-Open No. 6-4127 (Patent Document 2).

[0005] Patent Document 1 describes an autonomous mobile robot system (autonomous mobile device) that includes a sign detection means for detecting signs (landmarks) installed in the mobile environment, a distance measurement means for acquiring distance information to surrounding objects, a map storage means for storing map information of the mobile environment, a self-position calculation means for estimating a self-position by comparing the acquired distance information with the stored map information, and a route generation means for generating a route to a destination (see Abstract). Patent Document 1 also describes that the signs detected by the sign detection means are installed on the ceiling of the mobile environment (see 0041).

[0006] Furthermore, Patent Document 2 describes an indoor moving body (autonomous moving device) that uses a TV camera mounted on a moving cart to capture images of the ceiling above the travel path (travel environment), including fluorescent lights (landmarks), and calculates (detects) the position of the fluorescent light in the captured image using an image processing device, and then calculates the position of the moving cart using a computing device based on the position of the fluorescent light in the captured image and the absolute position of the fluorescent light (see abstract). [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2005-242409 A [Patent Document 2] Japanese Patent Application Publication No. 6-4127 Summary of the Invention [Problem to be solved by the invention]

[0008] Patent Documents 1 and 2 describe an autonomous mobile device that detects landmarks (signs and fluorescent lights) installed on the ceiling of the mobile environment and estimates its own position.

[0009] However, Patent Documents 1 and 2 do not describe situations in which an installed sign cannot be detected due to deterioration of the sign over time or changes in the surrounding lighting environment, or situations in which an installed light cannot be detected due to reasons such as the light going out.

[0010] Therefore, the present invention provides an autonomous mobile device that correctly recognizes landmarks whose states change and correctly estimates its own position. [Means for solving the problem]

[0011] In order to solve the above problems, the present invention provides an autonomous mobile device that moves autonomously based on a self-position estimated by a self-position estimation device, the autonomous mobile device being installed on a ceiling. Multiple lights Landmark As the picture shownA ceiling camera to take pictures, a ceiling height measuring means for measuring a distance from the ceiling camera to a ceiling on which a landmark photographed by the ceiling camera is installed; It has an environmental sensor that acquires the surrounding conditions in the direction of travel, and an internal sensor that detects its own position information.

[0012] The self-location estimation device that estimates the self-location is a ceiling camera. and ceiling height measuring means a landmark detection processing unit that detects information about landmarks based on the information detected by the Pre-set A memory unit stores information about landmarks and environmental map data, and determines the state of the landmarks based on the information about the landmarks detected by the landmark detection processing unit and the information about the landmarks stored in the memory unit, and For each light of the ceiling camera Depending on the detection accuracy, Depending on the magnitude of the vector from the center of the image to the center of each light, a coefficient from 0.0 to 1.0 is multiplied for each light so that the smaller the vector from the center of the image, the greater the weighting. The landmark status determination unit calculates the position information of the weighted landmarks, and the internal sensor detects the landmarks. Self The vehicle has a self-position estimation unit that estimates its own position based on position information, surrounding conditions in the direction of travel acquired by an environmental sensor, environmental map data stored in a memory unit, and weighted landmark position information calculated by a landmark state determination unit. Effect of the Invention

[0013] According to the present invention, it is possible to provide an autonomous mobile device that correctly recognizes landmarks whose states change and correctly estimates its own position.

[0014] Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0015] [Figure 1] 1 is an explanatory diagram illustrating a configuration of an autonomous mobile device 1 and a processing flow in the autonomous mobile device 1 according to a first embodiment. [Diagram 2] FIG. 11 is an explanatory diagram illustrating a process flow of a self-position estimation unit 1005 described in the first embodiment. [Diagram 3] 3 is an explanatory diagram for explaining a method of estimating a self-position based on illumination information of the autonomous mobile device 1 described in the first embodiment. FIG. [Figure 4] 4 is an explanatory diagram for explaining a method for determining an illumination state of the autonomous mobile device 1 described in the first embodiment. FIG. [Diagram 5] 3 is an explanatory diagram for explaining a user interface related to control of the autonomous mobile device 1 described in the first embodiment. FIG. [Figure 6] 4 is an explanatory diagram for explaining a method of notifying the determination result of the illumination state of the autonomous mobile device 1 described in the first embodiment. FIG. [Figure 7] FIG. 2 is an explanatory diagram for explaining a user interface for editing illumination information when the autonomous mobile device 1 described in the first embodiment is offline. [Figure 8] FIG. 11 is an explanatory diagram illustrating a configuration of an autonomous mobile device 1 described in a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the same reference numerals are used in different drawings for parts having substantially the same or similar configurations or functions, and where explanations are repeated, the explanations may be omitted.

[0017] In addition, in the drawings, the position, size, shape, range, etc. of each configuration or part having each function may differ from the actual position, size, shape, range, etc. in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. shown in the drawings. EXAMPLES

[0018] First, the configuration of the autonomous mobile device 1 and the process flow in the autonomous mobile device 1 according to the first embodiment will be described.

[0019] FIG. 1 is an explanatory diagram illustrating a configuration of an autonomous mobile device 1 and a processing flow in the autonomous mobile device 1 according to the first embodiment.

[0020] The autonomous mobile device 1 described in the first embodiment has a ceiling camera 101 facing the ceiling, a ceiling height measuring means 102, an environmental sensor 103, wheels 104a (front wheels) and 104b (rear wheels), and rotary encoders 105a and 105b.

[0021] The ceiling camera 101 captures an image of the ceiling above (upper part of) the autonomous mobile device 1, and captures (detects) landmarks (in the first embodiment, lights, e.g., fluorescent lights) installed on the ceiling. Note that multiple ceiling cameras 101 may be installed to capture a wider range.

[0022] The ceiling height measuring means 102 measures (detects) the distance (ceiling height) from the ceiling camera 101 to the ceiling on which the landmark photographed by the ceiling camera 101 is installed (for example, the distance from the ceiling camera 101 to the ceiling (the distance from the autonomous mobile device 1 to the ceiling)). For example, the ceiling height measuring means 102 is a laser range finder.

[0023] In addition, the ceiling camera 101 may have the function of ceiling height measuring means 102 by using a stereo camera that processes the captured parallax information and detects the distance to the object, or a depth camera that has a depth sensor that detects depth information.

[0024] The information detected by the ceiling camera 101 and the ceiling height measurement means 102 is detected by the landmark detection processing unit 1001 described later as landmark information (e.g., lighting information), which is the landmark position (e.g., lighting position) and landmark shape (e.g., lighting shape).

[0025] That is, for example, the lighting position is detected based on an image of the ceiling captured by ceiling camera 101 and the distance to the ceiling detected by ceiling height measuring means 102.

[0026] The environmental sensor (external (outside) sensor) 103 is installed in front of the autonomous mobile device 1 and acquires (detects) the surrounding situation in the traveling direction (for example, a point cloud (point group) or an image in the traveling direction). Note that multiple environmental sensors 103 may be installed in order to detect a wider range.

[0027] The environmental sensor 103 is, for example, a LiDAR (Laser Imaging Detection and Ranging) that measures scattered light from a pulsed laser irradiation light and detects the distance to a distant object, a stereo camera, a depth camera, or the like.

[0028] The rotary encoder 105a measures the rotation angle (amount of rotation) of the wheel 104a, outputs (detects) position information, and is connected to the wheel 104a. In other words, it is an internal (internal) sensor that detects (acquires) the state (own position information) of the autonomous mobile device 1. The rotary encoder 105a may be an inertial measurement device.

[0029] The rotary encoder 105b measures the rotation angle (amount of rotation) of the wheel 104b, outputs (detects) position information, and is connected to the wheel 104b. In other words, it is an internal (internal) sensor that detects (acquires) the state (own position information) of the autonomous mobile device 1. The rotary encoder 105b may be an inertial measurement device.

[0030] The autonomous mobile device 1 also includes a self-position estimation device 1000 that estimates its own position, a control unit 1006, and communication means that communicates between the self-position estimation device 1000 and the outside (user).

[0031] The control unit 1006 determines a control amount based on the self-position estimated by the self-position estimation device 1000 (the self-position estimation unit 1005 described later), controls the wheels 104a and 104b, and controls the autonomous movement (driving) of the autonomous moving device 1.

[0032] That is, the self-position estimation device 1000 estimates its own position based on information detected by the ceiling camera 101, the ceiling height measurement means 102, the environmental sensor 103, the rotary encoder 105a, and the rotary encoder 105b.

[0033] In the first embodiment, a self-location estimation device 1000 is installed in the autonomous mobile device 1, and the self-location estimation device 1000 is directly connected to the ceiling camera 101, the ceiling height measurement means 102, the environmental sensor 103, the rotary encoder 105a, and the rotary encoder 105b by wiring such as a cable, and information is directly communicated between the self-location estimation device 1000 and the ceiling camera 101, the ceiling height measurement means 102, the environmental sensor 103, the rotary encoder 105a, and the rotary encoder 105b.

[0034] The self-location estimation device 1000 includes a landmark detection processing unit 1001, a storage unit 1002, a landmark state determination unit 1003, a landmark state notification unit 1004 (communication means), and a self-location estimation unit 1005.

[0035] The landmark detection processing unit 1001 detects (extracts) the position and shape of the landmark (landmark information: note that the landmark information may also include color information of the landmark as a feature in order to improve detection accuracy) based on the information detected by the ceiling camera 101 and the ceiling height measurement means 102 (the landmarks imaged by the ceiling camera 101 and the distance to the ceiling measured by the ceiling height measurement means 102).

[0036] The memory unit 1002 memorizes (records, stores) environmental map data (movement environment map: map coordinate system), which is map data of the moving environment in which the autonomous moving device 1 moves autonomously, and landmark information, which is the positions and shapes of landmarks that are set in advance (note that the landmark information may also include color information of the landmarks).

[0037] In addition, when the landmark information stored in the memory unit 1002 is a light, the information is lighting information, and may be, for example, a lighting ON state and / or a lighting OFF state, and it is particularly preferable that the lighting ON state and the lighting OFF state are provided.

[0038] In the first embodiment, landmark information and environmental map data are stored in advance in the storage unit 1002. However, the storage unit 1002 may be operated like SLAM (Simultaneous Localization and Mapping) in which landmark information and environmental map data are stored in the storage unit 1002 while the autonomous mobile device 1 is moving.

[0039] The landmark status determination unit 1003 determines the status of the landmark based on the position information output by the rotary encoder 105a and the rotary encoder 105b, the landmark information (landmark position and landmark shape) detected by the landmark detection processing unit 1001, and the landmark information (landmark position and landmark shape) stored in the memory unit 1002.

[0040] Note that "determining the status of the landmark" means detecting (detecting) the status of the landmark, for example, detecting whether the installed lighting is in a situation where it cannot be detected due to reasons such as a light going out.

[0041] In other words, the landmark status discrimination unit 1003 compares the landmark information detected by the landmark detection processing unit 1001 with the landmark information stored in the memory unit 1002 at a position identified based on the position information output by the rotary encoder 105a and the rotary encoder 105b, and detects whether the landmark is functioning properly.

[0042] When comparing the information of the landmark detected by the landmark detection processing unit 1001 with the information of the landmark stored in the memory unit 1002, the information of the landmark detected by the landmark detection processing unit 1001 and the information of the landmark stored in the memory unit 1002 are matched at positions identified based on the position information output by the rotary encoder 105a and the rotary encoder 105b, and are compared by reference.

[0043] In this case, the landmark information detected by the landmark detection processing unit 1001 and the landmark information stored in the memory unit 1002 are weighted (weighted landmark information is calculated) according to the landmark detection state (detection accuracy of the landmark information) based on the position information output by the rotary encoder 105a and the rotary encoder 105b.

[0044] In other words, the landmark status discrimination unit 1003 discriminates the status of the landmark based on the landmark information detected by the landmark detection processing unit 1001 and the landmark information stored in the memory unit 1002, and calculates weighted landmark position information according to the detection accuracy of the landmark information.

[0045] If an abnormality is detected in the status of a landmark based on the status of the landmark detected by the landmark status determination unit 1003, the landmark status notification unit 1004 notifies the user of information about the landmark in which the abnormality was detected.

[0046] In this way, by instantly notifying the user of information about the landmark where an abnormality has been detected, landmarks (e.g., lights) present in the movement environment (roadway: movement environment: movement route) of the autonomous moving device 1 can also be inspected at the same time.

[0047] The self-position estimation unit 1005 estimates its own position based on the position information output by the rotary encoder 105a and the rotary encoder 105b, the surrounding conditions in the direction of travel detected by the environmental sensor 103, the environmental map data stored in the memory unit 1002, and the position information of the weighted landmarks calculated by the landmark state discrimination unit 1003.

[0048] That is, the self-position estimation unit 1005 estimates a self-position in a first state by odometry based on the position information output by the rotary encoder 105a and the rotary encoder 105b, associates the environmental map data stored in the storage unit 1002 with the surrounding conditions in the traveling direction detected by the environmental sensor 103, estimates a self-position in a second state, and determines a self-position in a third state based on the position information of the weighted landmarks calculated by the landmark state determination unit 1003. situation The self-location is estimated, and the final self-location is estimated (Kalman filter processing) based on the first self-location, the second self-location, and the third self-location.

[0049] Next, a process flow of the self-location estimation unit 1005 described in the first embodiment will be described.

[0050] FIG. 2 is an explanatory diagram illustrating a process flow of the self-position estimation unit 1005 described in the first embodiment.

[0051] The self-position estimation unit 1005 has a self-position estimation unit (first self-position estimation unit) 201 based on lighting information, a self-position estimation unit (second self-position estimation unit) 202 based on an environmental sensor 103, a self-position synthesis unit 203, and a self-position synthesis unit 203 that uses the self-positions estimated by the self-position estimation unit 201 and the self-position estimation unit 202 to estimate a final self-position.

[0052] The self-position estimation unit 201 estimates the self-position based on the weighted illumination information (position information) calculated by the landmark state determination unit 1003.

[0053] The self-position estimation unit 202 estimates the self-position based on the position information output by the rotary encoder 105a and the rotary encoder 105b, the environmental map data stored in the memory unit 1002, and the surrounding conditions in the traveling direction detected by the environmental sensor 103.

[0054] The self-location synthesis unit 203 uses the self-locations estimated by the self-location estimation unit 201 and the self-location estimation unit 202 to estimate a final self-location, and outputs the estimated self-location to the control unit 1006.

[0055] The weighted lighting information (position information) calculated by the landmark state discrimination unit 1003 is calculated based on the position information output by the rotary encoder 105a and the rotary encoder 105b, the lighting information (position information) detected by the landmark detection processing unit 1001, and the lighting information (position information) stored in the memory unit 1002.

[0056] The self-position estimation unit 201 and the self-position estimation unit 202 share the position determined based on the position information output by the rotary encoder 105a and the rotary encoder 105b.

[0057] When the self-position estimation unit 201 and the self-position estimation unit 202 estimate their own positions, they output variances (reciprocals of weights) that take into account the detection accuracy.

[0058] For example, if the self-location estimation value of the self-location estimation unit 201 is r1 and the variance is σ^2_1, and the self-location estimation value of the self-location estimation unit 202 is r2 and the variance is σ^2_2, then the self-location estimation value r estimated by the self-location synthesis unit 203 is expressed as follows: r=(1-(σ^2_2 / (σ^2_1+σ^2_2)))r1+(σ^2_2 / (σ^2_1+σ^2_2))r2 It is calculated by the weighted average.

[0059] That is, the self-position estimation value estimated by the self-position estimation unit 201 is calculated based on the weighted illumination information (position information) calculated by the landmark state determination unit 1003.

[0060] Next, a method for estimating a self-position based on illumination information of the autonomous mobile device 1 according to the first embodiment will be described.

[0061] FIG. 3 is an explanatory diagram for explaining a method of estimating a self-position based on illumination information of the autonomous mobile device 1 according to the first embodiment.

[0062] The autonomous mobile device 1 uses the ceiling camera 101 to capture an image 304 of the ceiling, and detects lights 301, 302, and 303 as landmarks.

[0063] At this time, the positions of each of the lights 301, 302, and 303 are associated with the landmark information (the positions of the lights 301, 302, and 303) stored in the memory unit 1002 based on the position information output by the rotary encoder 105a and the rotary encoder 105b.

[0064] In other words, the lighting positions (L1', L2', L3') of lighting 301, lighting 302, lighting 303 in the environmental map data stored in memory unit 1002 are matched with the lighting positions (L1, L2, L3) of lighting 301, lighting 302, lighting 303 detected by ceiling camera 101.

[0065] The lighting positions (L1, L2, L3) detected by ceiling camera 101 are detected from the positions of lighting 301, lighting 302, and lighting 303 in image 304 of the ceiling captured by ceiling camera 101 and the distance to the ceiling detected by ceiling height measurement means 102.

[0066] At this time, the self-position estimation value r1 of the self-position estimation unit 201 is expressed as a redundant simultaneous equation (having more equations than variables) using the lighting positions L1', L2', and L3' stored in the memory unit 1002 and the lighting positions L1, L2, and L3 detected by the ceiling camera 101.

[0067] Then, the landmark state discrimination unit 1003 assigns weights to each of the lights 301, 302, and 303 according to their detection states, and when solving the redundant simultaneous equations, the self-position estimation unit 201 calculates the self-position estimation value r1 by the weighted least squares method using these weights.

[0068] The variance of the self-position estimation unit 201 is also calculated as the error variance of the weighted least squares method using the weights assigned to the respective lighting elements 301, 302, and 303.

[0069] In this way, in the first embodiment, the landmark state determination unit 1003 weights each light detected by the ceiling camera 101 according to its detection state. This prevents the estimation result of the self-position from deteriorating even in a situation where the installed light cannot be detected due to a reason such as a light failure.

[0070] By using lighting as a landmark, it is not necessary to install signs in advance on the ceiling of the environment in which the autonomous mobile device 1 moves.

[0071] According to the first embodiment, a plurality of landmarks (lighting) can be handled.

[0072] Next, a method for determining the illumination state of the autonomous mobile device 1 according to the first embodiment will be described.

[0073] FIG. 4 is an explanatory diagram for explaining a method for determining an illumination state of the autonomous mobile device 1 according to the first embodiment.

[0074] In Fig. 4, an image 405 is captured by ceiling camera 101, and lights 401, 402, 403, and 404 are detected as landmarks. In Fig. 4, the center line of image 405 is indicated by a dotted line, and vectors of lights 401, 402, 403, and 404 from the center of image 405 are indicated by arrows.

[0075] In the first embodiment, lighting information (for example, all lighting is ON as a normal state) is stored in the storage unit 1002 as landmark information.

[0076] 4, in image 405, lighting 401, lighting 402, and lighting 404 are in a state that matches the lighting information of a lighting ON state stored in storage unit 1002 as a normal state, while lighting 403 is in a state that does not match the lighting information stored as lighting information stored in storage unit 1002 due to a reason such as lighting failure (a state in which lighting 403 cannot be detected). In other words, lighting 403 is in a lighting OFF state.

[0077] In this way, when a light (light 403) with a different lighting state from the lighting states of the surrounding lights (lights 401, 402, and 404) is detected, the landmark state discrimination unit 1003 determines that there is an abnormality in this light (light 403), and notifies the user of the lighting information (lighting state) in which the abnormality was detected via the landmark state notification unit 1004.

[0078] When the storage unit 1002 stores the illumination ON state and the illumination OFF state, the illumination 401, the illumination 402, and the illumination 404 are detected as being in the illumination ON state, and the illumination 403 is detected as being in the illumination OFF state.

[0079] Then, it is determined that the lighting state of the lighting 403 is different (there is an abnormality) from the lighting states of the surrounding lighting 401, lighting 402, and lighting 404, and the lighting information (lighting state) in which the abnormality was detected is notified to the user via the landmark state notification unit 1004.

[0080] Next, a weighting method for each light, which is performed in the landmark state determination unit 1003 according to the detection state of each light, will be described.

[0081] In general, the lens used in the ceiling camera 101 has distortion, and the lens distortion increases the farther away from the center of the lens. In particular, when a wide-angle lens or the like is used, the distortion becomes noticeable.

[0082] Even if the lens distortion is corrected, there is distortion that cannot be completely corrected in the image 405. For this reason, in the first embodiment, a weight is assigned (a value between 0.0 and 1.0 is added) to each illumination depending on the magnitude of the vector from the center of the image 405 toward the center of the illumination 401, the illumination 402, the illumination 403, and the illumination 404.

[0083] That is, the weighting is performed so that the smaller the magnitude of the vector from the center of the image 405, the greater the weight. In this manner, the weighted illumination information (position information) is set based on the magnitude of the vector from the center of the image 405 captured by the ceiling camera 101.

[0084] Furthermore, the detection accuracy of ceiling camera 101 may differ depending on whether the light is ON or OFF. Basically, the light ON state has higher brightness and is more distinctive than the light OFF state, so the light ON state is weighted more heavily than the light OFF state.

[0085] Also, with regard to the distance to the ceiling (distance to the lighting) detected by the ceiling height measurement means 102, the greater the distance to the lighting installed on the ceiling, the worse the detection accuracy of the ceiling height measurement means 102 may be. For this reason, the lighting installed at a higher position on the ceiling is weighted so that the weight is smaller.

[0086] In addition, factors other than those mentioned above, for example, lighting by a window may be disturbed, which may reduce the detection accuracy of the ceiling camera 101. In such a case, it is preferable that the landmark state determination unit 1003 assigns weights according to the detection state (factor) of each lighting.

[0087] That is, it is preferable that the weighting in the landmark state determination unit 1003 is set according to the moving environment of the autonomous moving device 1. Furthermore, the weighting may be set based on, for example, the variance of the detection accuracy.

[0088] In this way, in the first embodiment, the lighting information can be handled according to the lighting state by the landmark state determination unit 1003. This prevents the estimation result of the self-location from being deteriorated.

[0089] In the first embodiment, the illumination to be detected is a rectangular shape such as a fluorescent lamp, but is not limited to a rectangular shape, and may be a round shape such as a downlight.

[0090] Next, a user interface for controlling the autonomous mobile device 1 according to the first embodiment will be described.

[0091] FIG. 5 is an explanatory diagram for explaining a user interface related to control of the autonomous mobile device 1 described in the first embodiment.

[0092] In Example 1, in order to control the autonomous moving device 1, the current position of the autonomous moving device 1, lighting 503 based on lighting information stored in the memory unit 1002, and an environmental data map 502 based on environmental map data stored in the memory unit 1002 are displayed on a screen (interface) 501 of a user's smartphone, tablet, PC, etc.

[0093] In the first embodiment, the environmental data map 502 is displayed with solid lines, the lights 503 are displayed with dotted squares, and the autonomous moving device 1 is displayed with a solid square, but this is not limited to this and can be changed as appropriate to make it easier for the user to see.

[0094] Next, a method for notifying the result of determination of the illumination state of the autonomous mobile device 1 according to the first embodiment will be described.

[0095] FIG. 6 is an explanatory diagram for explaining a method for notifying the result of determination of the illumination state of the autonomous mobile device 1 described in the first embodiment.

[0096] In the first embodiment, when it is determined that there is an abnormality in the lighting, information about the lighting in which the abnormality has been detected is notified to the user.

[0097] As a result, the light 604 in which an abnormality has been detected flashes on the screen 501. When the user performs an action such as clicking on the flashing light 604 in which an abnormality has been detected, a screen 605 displaying details of the abnormal state pops up on the screen 501.

[0098] Screen 605 pops up and displays the details of the abnormal condition, for example, “Status abnormality detected: non-lighting error” (lighting information (lighting status) in which the abnormality was detected) and an image captured by ceiling camera 101, for example, as shown in FIG. 4, are displayed.

[0099] In this way, when the landmark state determination unit 1003 detects an abnormality in the lighting, the details of the abnormal state are immediately notified to the user via the landmark state notification unit 1004. This allows the lighting in the moving environment of the autonomous mobile device 1 to be inspected.

[0100] Next, a user interface for editing illumination information when the autonomous mobile device 1 described in the first embodiment is offline will be described.

[0101] FIG. 7 is an explanatory diagram for explaining a user interface for editing illumination information when the autonomous mobile device 1 described in the first embodiment is offline.

[0102] In the first embodiment, when the autonomous mobile device 1 is offline and not moving, the user is shown, on the screen 501, lighting 503 based on the lighting information stored in the memory unit 1002 and an environmental data map 502 based on the environmental map data stored in the memory unit 1002.

[0103] In this way, the user edits the lighting information related to the lighting 503. By performing an operation such as clicking on the lighting 701 that the user wishes to edit, a screen 702 for editing the lighting information is popped up on the screen 501.

[0104] A pop-up screen 702 for editing lighting information displays lighting information such as "Lighting information: Position: . . ., Shape: . . ., Color: . . .".

[0105] In this way, the user edits the lighting information stored in the storage unit 1002. In this way, when there is a change in the lighting information due to replacement of lighting, etc., the change in the lighting information can be immediately reflected, and the estimation result of the self-position is not deteriorated in the moving environment of the autonomous moving device 1.

[0106] In this way, the autonomous mobile device 1 moves autonomously based on its own position estimated by the self-position estimation device 1000, and has the following configuration. (1) A ceiling camera 101 that photographs landmarks installed on the ceiling. (2) A ceiling height measuring means 102 that measures the distance from the ceiling camera 101 to the ceiling. (3) An environmental sensor 103 that acquires the surrounding conditions in the direction of travel. (4) An internal sensor (105) for detecting its own position information.

[0107] The self-location estimation device 1000 that estimates its own location has the following configuration. (1) A landmark detection processing unit 1001 that detects information about landmarks based on information detected by the ceiling camera 101 and the ceiling height measuring means 102. (2) A memory unit 1002 for storing landmark information and environmental map data. (3) A landmark state discrimination unit 1003 that discriminates the state of the landmark based on the landmark information detected by the landmark detection processing unit 1001 and the landmark information stored in the memory unit 1002, and calculates weighted landmark position information according to the detection accuracy of the landmark information. (4) A self-location estimation unit 1005 that estimates the self-location based on the location information detected by the internal sensor (105), the surrounding conditions in the direction of travel acquired by the environmental sensor 103, the environmental map data stored in the memory unit 1002, and the weighted landmark location information calculated by the landmark state discrimination unit 1003.

[0108] Furthermore, the autonomous mobile device 1 can correctly recognize landmarks whose conditions are changing and correctly estimate its own location without deteriorating the estimation result of its own location even in situations where the installed signs cannot be detected due to deterioration of the signs over time or changes in the surrounding lighting environment, or where the installed lights cannot be detected due to reasons such as a light failure.

[0109] Furthermore, the autonomous moving device 1, which estimates its own position using landmark information, can suppress deterioration in the accuracy of its own position estimation even when the information of pre-set landmarks cannot be used due to reasons such as lighting going out and the state of the landmarks changes. EXAMPLES

[0110] Next, a configuration of an autonomous mobile device 1 according to a second embodiment will be described.

[0111] FIG. 8 is an explanatory diagram illustrating the configuration of the autonomous mobile device 1 according to the second embodiment.

[0112] The autonomous mobile device 1 described in the second embodiment is different from the autonomous mobile device 1 described in the first embodiment in the installation position of the self-position estimation device 1000.

[0113] In the autonomous moving device 1 described in the first embodiment, the self-position estimation device 1000 is installed inside the autonomous moving device 1, whereas in the autonomous moving device 1 described in the second embodiment, the self-position estimation device 1000 is installed outside the autonomous moving device 1.

[0114] That is, in the second embodiment, the self-position estimation device 1000 is not mounted on the autonomous moving device 1, but is mounted on an external control controller (PC) that estimates the self-position of the autonomous moving device 1, or the like.

[0115] In the self-position estimation device 1000, the detection results of the ceiling camera 101, the ceiling height measurement means 102, the environmental sensor 103, the rotary encoder 105a and the rotary encoder 105b are communicated via the wireless communication means 801a and the wireless communication means 801b.

[0116] The self-location estimation device 1000 estimates its own location based on the detection result, similarly to the first embodiment.

[0117] Then, the estimation result of the self-position estimation device 1000 is communicated to the control unit 1006 of the autonomous mobile device 1 via the wireless communication means 801a and the wireless communication means 801b.

[0118] This eliminates the need to estimate the autonomous mobile device 1's own position, and the processing power of the PC installed in the autonomous mobile device 1 can be reduced, achieving power saving and miniaturization of the autonomous mobile device 1.

[0119] It should be noted that the present invention is not limited to the above-described embodiment, but includes various modified examples. For example, the above-described embodiment is specifically described in order to easily explain the present invention, and the present invention is not necessarily limited to the embodiment having all of the described configurations.

[0120] Also, a part of the configuration of one embodiment can be replaced with a part of the configuration of another embodiment. Also, the configuration of another embodiment can be added to the configuration of one embodiment. Also, a part of the configuration of each embodiment can be deleted, and a part of another configuration can be added or replaced with a part of another configuration. [Explanation of symbols]

[0121] 1. Autonomous mobile device 101 Ceiling Camera 102 Ceiling height measuring means 103 Environmental Sensor 104a, 104b...wheels 105a, 105b Rotary encoder 1000... Self-position estimation device 1001: Landmark detection processing unit 1002...Storage section 1003 Landmark state determination unit 1004: Landmark state notification unit 1005... Self-position estimation section 1006 Control section 201... Self-location estimation unit based on lighting information 202...Self-location estimation unit based on environmental sensors 203... Self-position synthesis section 301, 302, 303... Illuminations detected as landmarks 304: Images taken by the ceiling camera 401, 402, 403, 404... Illuminations detected as landmarks 405...Image taken by ceiling camera 501...Screen 502 Environmental Data Map 503: Illumination stored in the memory unit 604: An abnormality was detected in the lighting. 605: Screen showing details of abnormal condition 701...Light to edit 702...Screen for editing lighting information 801a, 801b...Wireless communication means

Claims

1. An autonomous mobile device that moves autonomously based on a self-position estimated by a self-position estimation device, The vehicle has a ceiling camera that captures images of landmarks, which are a plurality of lights installed on a ceiling, a ceiling height measuring means that measures the distance from the ceiling camera to the ceiling on which the landmarks photographed by the ceiling camera are installed, an environmental sensor that acquires the surrounding conditions in the direction of travel, and an internal sensor that detects its own position information, The self-location estimation device that estimates its own location, a landmark detection processing unit that detects information about landmarks based on information detected by the ceiling camera and the ceiling height measuring means; A storage unit that stores information about landmarks and environmental map data that are set in advance; a landmark state discrimination unit that discriminates a state of a landmark based on information of the landmark detected by the landmark detection processing unit and information of the landmark stored in the storage unit, and calculates weighted landmark position information by integrating a coefficient from 0.0 to 1.0 for each of the lights according to the magnitude of a vector from the center of the image to the center of each of the lights in accordance with the detection accuracy of the ceiling camera for each of the lights, such that the smaller the magnitude of the vector from the center of the image, the greater the weighting; a self-location estimation unit that estimates a self-location based on the self-location information detected by the internal sensor, the surrounding conditions in the traveling direction acquired by the environmental sensor, the environmental map data stored in the storage unit, and the weighted landmark location information calculated by the landmark state determination unit; An autonomous mobile device comprising:

2. An autonomous mobile device as described in claim 1, An autonomous mobile device, wherein the self-position estimation device is mounted inside the autonomous mobile device.

3. An autonomous mobile device as described in claim 1, The autonomous mobile device is characterized in that the self-position estimation device is mounted on the outside of the autonomous mobile device.

4. An autonomous mobile device as described in claim 1, The self-location estimation device is an autonomous mobile device characterized in that it has a landmark state notification unit that, when an abnormality is detected in the state of the landmark based on the state of the landmark detected by the landmark state determination unit, notifies a user of information about the landmark in which the abnormality has been detected.

5. An autonomous mobile device as described in claim 1, The self-location estimation unit is a first self-location estimation unit that estimates a self-location based on the weighted illumination information calculated by the landmark state determination unit; a second self-location estimation unit that estimates a self-location based on the self-location information detected by the internal sensor, the environmental map data stored in the storage unit, and the surrounding conditions in the traveling direction detected by the environmental sensor; a self-location synthesis unit that uses the self-locations estimated by the first self-location estimation unit and the second self-location estimation unit to estimate a final self-location and outputs the estimated self-location; An autonomous mobile device comprising:

Images