Moving body

Abstract

To increase precision of comparison between a photographed image by a camera and map data.SOLUTION: A moving body has a road surface photographing device for taking an image of a road surface. The road surface photographing device includes: a camera disposed so as to face the road surface; at least one light source for applying light to a photographing area of the camera; a storage device for storing an image of a previously photographed road surface as map data; and a processor for comparing map data with an image taken by the camera and estimating a self-position of the moving body. At least one light source is disposed such that reflection light of the applied light is photographed by the camera. The road surface photographing device further has a light-shielding plate disposed around the camera and the light source so as to shield the photographing area from external light.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a technique for estimating the self-location of a moving object. [Background technology]

[0002] Patent Document 1 discloses a technology for estimating the self-position of an autonomous vehicle using a road surface image. According to this technology, the self-position is estimated by comparing features extracted from an image of the road surface below the vehicle body captured by a camera with features extracted from a map image (map data) captured in advance. The autonomous vehicle is equipped with an adjustment unit that controls a light source corresponding to an area with low brightness in a group of light sources that irradiates light onto the camera's capture area to brighten the area. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-029706 Summary of the Invention [Problem to be solved by the invention]

[0004] According to the technology described in Patent Document 1, variations in the arrangement of the light sources or variations in the brightness of the light sources may cause variations in brightness in the image of the road surface captured by the camera, which may result in the capture of feature points that do not exist on the actual road surface.

[0005] The present disclosure has been made in consideration of the above-mentioned problems, and aims to provide a moving body that enables improved accuracy in matching images captured by a camera with map data. [Means for solving the problem]

[0006] The moving body according to the present disclosure includes a road surface photographing device that photographs an image of a road surface. The road surface photographing device includes a camera arranged to face the road surface, at least one light source that irradiates light onto an area photographed by the camera, a storage device that stores an image of the road surface photographed in advance as map data, and a processor that compares the map data with the image photographed by the camera to estimate the self-position of the moving body. The at least one light source is arranged so that reflected light of the irradiated light is photographed by the camera. The road surface photographing device further includes a light shielding plate arranged around the camera and the light source to block the photographing area from external light. Effect of the Invention

[0007] According to the present disclosure, the reflected light is used as a light source for capturing an image. This makes it possible to suppress uneven brightness in the captured image. Furthermore, by providing the above-mentioned light shielding plate, it is possible to suppress uneven brightness in the captured image caused by external light such as sunlight and illumination light entering the capture area. This improves the accuracy of matching the captured image with map data. [Brief description of the drawings]

[0008] [Figure 1] 1A and 1B are diagrams for explaining a configuration of a moving body and a self-position estimation method according to an embodiment; [Diagram 2] FIG. 2 is a diagram illustrating a configuration of the road surface photographing device shown in FIG. [Diagram 3] 10A and 10B are diagrams illustrating configurations of other examples of light source arrangements according to the embodiment. [Figure 4] 10A and 10B are diagrams illustrating the configuration of another example of the light blocking plate according to the embodiment. [Diagram 5] 5A to 5C are diagrams for explaining a first example of a method for adjusting the amount of light from a light source according to an embodiment. [Figure 6] 6A to 6C are diagrams for explaining a second example of the method for adjusting the light amount of a light source according to the embodiment. [Figure 7] 10A and 10B are diagrams for explaining a third example of the method for adjusting the light amount of a light source according to the embodiment. [Figure 8]13A to 13C are diagrams for explaining a fourth example of the method for adjusting the light amount of a light source according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0010] 1. Configuration of a moving object equipped with a road surface photographing device FIG. 1 is a diagram for explaining the configuration of a moving body 1 and a self-position estimation method according to an embodiment. FIG. 2 is a diagram showing the configuration of a road surface imaging device 10 shown in FIG. 1. The moving body 1 moves on a road surface 2. The moving body 1 is, for example, a vehicle. The vehicle may be an autonomous vehicle or a vehicle driven by a driver. Furthermore, the "moving body" according to the present disclosure may be, for example, a robot such as a logistics robot.

[0011] The moving object 1 is equipped with a road surface photographing device 10. The road surface photographing device 10 photographs an image of a road surface 2, and estimates a self-position P of the moving object 1 by using the photographed image. The road surface photographing device 10 includes a camera 12, a light source 14, and an electronic control unit (ECU) 16.

[0012] The camera 12 is positioned to face the road surface 2 as shown in Fig. 2. The road surface photographing device 10 has a top plate 18. The top plate 18 is fixed to, for example, the bottom surface of the moving body 1. The camera 12 is attached to, for example, the top plate 18. The top plate 18 is, for example, a white resin plate. The photographing area R (photographing surface) shown in Fig. 2 corresponds to the area of ​​the road surface 2 photographed by the camera 12.

[0013] The light sources 14 are arranged so as to irradiate light onto the shooting area R of the camera 12. The number of light sources 14 is not limited, and may be one or more. More specifically, as an example, in the case of a quadrangular top plate 18 (see FIG. 7(A) described later), the light sources 14 may be arranged corresponding to each side of the top plate 18. FIG. 2 shows two of the four light sources 14 arranged in this manner. The light sources 14 are, for example, LEDs (Light Emission Diodes). More specifically, the LEDs are, for example, tape LEDs.

[0014] The ECU 16 performs control related to estimation of the self-position P using the road surface photographing device 10. The ECU 16 includes a processor 20 and a storage device 22. The processor 20 executes various processes related to the control. The storage device 22 stores various information required for various processes by the processor 20. Furthermore, as shown in FIG. 2, the road surface photographing device 10 includes an illuminance sensor 24. The illuminance sensor 24 is used for light amount adjustment, which will be described later.

[0015] In order to estimate the self-position P, the storage device 22 stores a camera image of the road surface 2 captured in advance as map data. The map data (map image) is data obtained by mapping a point group (feature points; see FIG. 1) extracted from a camera image captured in advance onto a map space. The processor 20 compares the map data with an image captured by the camera 12 while the mobile body 1 is traveling, and estimates the self-position P. More specifically, the processor 20 estimates the self-position P by comparing the point group (feature points around the mobile body; see FIG. 1) extracted from the camera image with the feature points of the map data. An overview of a method for estimating the self-position P, such as a method for extracting feature points, is described in, for example, Patent Document 1.

[0016] In order to accurately estimate the self-position P by the above-mentioned method, uniformity of the quality of the camera images is required. In order to uniformize the quality of the camera images, it is desirable that the entire shooting area R is illuminated with a uniform and sufficient amount of light.

[0017] In consideration of this, in this embodiment, each of the light sources 14 is disposed so that the "reflected light" of the emitted light is captured by the camera 12. In other words, each of the light sources 14 is disposed in such a direction that the direct light is not incident on the road surface 2. Note that the reflected light referred to here includes light reflected by both specular reflection (regular reflection) and diffuse reflection (irregular reflection).

[0018] In order to realize the above-mentioned arrangement of the light sources 14, in the example shown in Fig. 2, each of the light sources 14 is arranged to face diagonally upward. Furthermore, each of the light sources 14 is arranged in a position where the light emitted from each light source 14 does not directly enter the shooting area R, but the light reflected from the inside of the housing such as the top plate 18 enters the shooting area R. More specifically, the housing 26 of the road surface photographing device 10 is configured to include the top plate 18 and a light shielding plate 28 described below. Each of the light sources 14 is attached to the light shielding plate 28 as an example, and emits light from the outside of the housing toward the inside.

[0019] Furthermore, in this embodiment, the road surface photographing device 10 includes a light shielding plate 28. As shown in Fig. 2, the light shielding plate 28 is disposed around the camera 12 and each light source 14 so as to block the photographing area R from external light. The external light includes, for example, sunlight and external illumination light. More specifically, the light shielding plate 28 is formed so as to extend downward from each side of the top plate 18. In other words, the light shielding plate 28 is formed so as to surround (cover) the photographing area R.

[0020] As described above, the mobile body 1 equipped with the road surface photographing device 10 according to this embodiment is configured so that the light source 14 is not aimed directly at the road surface 2, but rather uses only reflected light as a light source for photographing an image. This makes it possible to suppress the occurrence of uneven brightness in the camera image. Furthermore, by providing the light shielding plate 28, it is possible to suppress the occurrence of uneven brightness in the camera image caused by external light such as sunlight and illumination light entering the photographing area R. More specifically, even in an environment where the external illumination intensity changes, it is possible to suitably suppress the occurrence of uneven brightness in the camera image.

[0021] As described above, according to the present embodiment, it is possible to effectively suppress the occurrence of uneven brightness in the camera image. As a result, it is possible to improve the accuracy of matching the camera image with the map data. This leads to an improvement in the accuracy of estimating the self-location.

[0022] Furthermore, LEDs are suitable as light sources 14 for photographing road surfaces because they are inexpensive and consume little power. On the other hand, LEDs are generally highly directional. For this reason, when direct light is incident on the photographing area R of the road surface 2, there is a possibility that the LED placement marks will appear on the camera image as brightness unevenness due to reflection. In this regard, according to the light source arrangement of this embodiment, the light that is incident on the photographing area R is light that has been reflected and diffused inside the housing. For this reason, unevenness in the camera image can be suppressed.

[0023] 2.Other examples of light source arrangement 3(A) to 3(C) are diagrams showing configurations of other examples of light source arrangements according to the embodiment.

[0024] First, in the example shown in Fig. 3(A), each light source 14 is attached not to a light shielding plate 28 but to a fixed portion 30 provided around the camera 12 inside the housing 26. More specifically, the fixed portion 30 is formed so as to extend downward from the top plate 18, and covers the periphery of the tip of the camera 12. Each light source 14 is attached to the fixed portion 30 on the side opposite to the side facing the camera 12. Also in this example, in order to prevent direct light from entering the shooting area R, each light source 14 is arranged so as to face from the inside to the outside of the housing 26 (so as to face the side direction of the housing 26) and to face diagonally upward.

[0025] Next, in the example shown in Fig. 3(B), each light source 14 is attached to a light shielding plate 28 in the same manner as in the example shown in Fig. 2. In addition, in this example, as shown in Fig. 3(B), a diffusion plate 32 that diffuses the light emitted from each light source 14 is provided. The diffusion plate 32 is attached, for example, to the light emitting surface of each light source 14. As the diffusion plate 32, for example, a plate having a strong diffusion effect such as an acrylic resin is used. According to this example, the overall amount of light is reduced due to the installation of the diffusion plate 32, but a higher light diffusion effect can be expected.

[0026] Next, in the example shown in Fig. 3(C), each light source 14 is attached to the fixing portion 30 in the same manner as in the example shown in Fig. 3(A). In addition, in this example, as shown in Fig. 3(C), a diffusion plate 34 is disposed between each light source 14 and the road surface 2. The diffusion plate 34 is supported by a support member (not shown). In this example, the overall amount of light is reduced due to the installation of the diffusion plate 32, but a higher light diffusion effect can be expected.

[0027] 3.Other examples of light shields In the example shown in Fig. 2 described above, the light shielding plate (light shielding portion) of the road surface photographing device 10 is realized using only the light shielding plate 28. In order to prevent the road surface photographing device 10 from being damaged by the impact when the light shielding plate comes into contact with the road surface 2, the light shielding plate may be configured, for example, as follows. Fig. 4(A) and Fig. 4(B) are diagrams each showing the configuration of another example of the light shielding plate according to the embodiment.

[0028] First, in the example shown in Fig. 4(A), the light shielding plate 36 is composed of an upper portion 38 and a lower portion 40 that have different hardness in the up-down direction (vertical direction). Specifically, the upper portion 38 that is farther from the road surface 2 is made of a material having high hardness, such as an aluminum composite plate. Each light source 14 is attached to this upper portion 38. On the other hand, the lower portion 40 that is closer to the road surface 2 is made of a flexible material, such as rubber or cloth.

[0029] According to the light shielding plate 36 configured as described above, the portion close to the road surface 2 is shielded by the lower portion 40 made of a flexible material. This makes it possible to prevent damage to the road surface photographing device 10 caused by the impact when the light shielding plate 36 comes into contact with the road surface 2, while suppressing the incidence of external light into the photographing area R. Note that the light shielding plate (light shielding portion) in the example described here may be configured using three or more materials with different hardness in the vertical direction.

[0030] Also, as in the example shown in FIG. 4(B), a cover 42 made of a flexible material that is an exterior part of the moving body 1 may be used as a part (lower part) of the light blocking plate.

[0031] 4. Adjusting the light source

[0032] 4-1. First example of light adjustment method 5(A) and 5(B) are diagrams for explaining a first example of a method for adjusting the amount of light of the light source 14 according to the embodiment. The reflection intensity from the road surface 2 varies depending on the material of the road surface 2. For this reason, it is desirable that the road surface photographing device 10 has a function for adjusting the brightness within the photographing area R. More specifically, when the road surface photographing device 10 crosses a road surface area with different reflectivity, such as a change from concrete to asphalt, as the moving body 1 moves, the brightness within the photographing area R changes due to the difference in the reflection intensity of the road surface 2. In order to keep the quality of the camera image stable, it is desirable to quickly complete the light amount adjustment for suppressing such a change in brightness. In view of this, in the first example, the following feedback control of the amount of light is executed.

[0033] As already described, the road surface photographing device 10 includes an illuminance sensor 24. As shown in FIG. 5(A), the illuminance sensor 24 is attached to the top plate 18 so as to face the photographing area R of the road surface 2. A specific example of the arrangement of the illuminance sensor 24 will be described later with reference to FIG. 7(A). The illuminance sensor 24 is used to measure the illuminance of the photographing area R in the housing 26. The feedback control according to the first example is performed using the illuminance sensor 24, which has a measurement cycle shorter than the cycle of the frame rate of the camera 12 (for example, about 60 Hz), in order to adjust the brightness at a time faster than the frame rate of the camera 12. Specifically, as shown in FIG. 5(B), a control deviation between a preset target light amount and a light amount corresponding to the illuminance detected by the illuminance sensor 24 is calculated. Then, the light amount controller controls the intensity of the light source 14, for example, by controlling the current of the light source 14, so that the control deviation becomes smaller. This makes it possible to quickly adjust the light reflection (light amount) of the road surface 2 so as to suppress changes in illuminance in the photographed area R of the road surface 2, even when the road surface photographing device 10 crosses road surface areas with different reflectivities.

[0034] 4-2. Second example of light adjustment method 6(A) and 6(B) are diagrams for explaining a second example of a method for adjusting the amount of light of the light source 14 according to the embodiment. The second example relates to a measure for preventing the illuminance sensor 24 from directly picking up the light irradiated from the light source 14.

[0035] 6(A), a light shielding plate 44 may be provided so as to cover the periphery of the illuminance sensor 24 inside the housing 26. The light shielding plate 44 is formed so as to extend downward from the top plate 18 so as to be interposed between the light source 14 and the illuminance sensor 24. By additionally providing the light shielding plate 44, it is possible to block direct light from each light source 14 so that only light from directly below is picked up by the illuminance sensor 24.

[0036] 6(B), a polarizing plate 46 may be attached to the surface (detection surface) of the illuminance sensor 24. The polarizing plate 46 is configured to transmit only light from directly below. By additionally providing the polarizing plate 46, it is possible to block direct light from each light source 14 so that only light from directly below is picked up by the illuminance sensor 24.

[0037] 4-3. Third example of light adjustment method 7(A) and 7(B) are diagrams for explaining a third example of the light amount adjustment method of the light source 14 according to the embodiment. FIG. 7(A) is a diagram showing a specific example of the arrangement of the light source 14 and the illuminance sensor 24 in the housing 26. In this example, as shown in FIG. 7(A), four combinations of the light source 14 and the illuminance sensor 24 are arranged point-symmetrically with respect to the center position of the camera 12 as a reference, in other words, corresponding to the four sides of the top plate 18 having a rectangular shape. Each combination of the light source 14 and the illuminance sensor 24 adjusts the amount of light in the area of ​​the road surface 2 corresponding to each of the four divided areas A1 to A4. Here, the appropriate values ​​of the illuminances B1 to B4 in the housing 26 received by each illuminance sensor 24 and the intensity (output intensity) of the light source 14 cannot be obtained explicitly due to factors such as light diffusion and individual differences, and automatic adjustment by feedback control is desirable. In view of this, in the third example, feedback control of the amount of light shown in FIG. 7(B) is executed.

[0038] 7B is repeatedly executed by the processor 20. First, in step S100, the processor 20 acquires a value (actual illuminance) Bi (for example, i=1 to 4) of each illuminance sensor 24. Next, in step S102, the processor 20 calculates a difference Ei of the actual illuminance Bi from a target illuminance Bi_ref set in advance.

[0039] Next, in step S104, the processor 20 increases or decreases the light amount (intensity) Li of each light source 14 so as to reduce the calculated difference Ei. Next, in step S106, the processor 20 determines whether or not a predetermined end condition is satisfied. The end condition is, for example, that the difference Ei is sufficiently small, or that the light amount Li has reached a maximum or minimum value. If the result of this determination is No, the process returns to step S100. On the other hand, if the result of this determination is Yes, the process proceeds to the end.

[0040] According to the above-described feedback control of the amount of light, the output intensity of the light source 14 can be automatically adjusted (increased or decreased) for each divided region until the illuminance inside the housing 26 falls within the target range.

[0041] 4-4. Fourth example of light adjustment method 8(A) and 8(B) are diagrams for explaining a fourth example of the light amount adjustment method of the light source 14 according to the embodiment. FIG. 8(A) shows an example of a moving path of the moving body 1. In this example, the road surface on which the moving body 1 travels changes between asphalt, concrete, and an indoor resin-based road surface. It is assumed that the road surface environment may change in this way while the moving body 1 is traveling. With regard to the light amount adjustment of the light source 14, such a change in the road surface environment leads to, for example, the following problem. That is, the appropriate value of the intensity (output intensity) of the light source 14 for keeping the illuminance inside the housing 26 within the target range differs depending on the road surface environment. For this reason, when a sudden change in the road surface occurs (for example, from outdoor asphalt to indoor resin-based road surface), there is a high possibility that the quality of the camera image during the adjustment process when the intensity of the light source 14 is adjusted by feedback control will be degraded due to whiteout or the like. As a result, there is a possibility that the camera image cannot be used for image processing for self-position estimation. Therefore, it is required to shorten the time required for adjusting the intensity of the light source 14 and stabilize the illuminance inside the housing 26 early. In view of this, in the fourth example, feedback control of the light amount shown in FIG. 8(B) is executed.

[0042] The process shown in Fig. 8(B) is repeatedly executed by the processor 20 as the current position of the moving object 1 is updated. First, in step S200, the processor 20 sets an initial value I of the intensity of the light source 14. More specifically, the storage device 22 stores a map of the initial values ​​I corresponding to the position information of the moving object 1. The processor 20 reads out the initial value I corresponding to the updated current position from the initial value map and sets it as the current initial value I.

[0043] Next, in step S202, the processor 20 adjusts the intensity of the light source 14 so that it coincides with the set initial value I. Next, in step S204, it is determined whether or not the absolute value of the difference E between the illuminance sensor value (actual illuminance) and the target illuminance is less than a predetermined threshold value TH1. As a result, if the determination result is No, the process returns to step S202, and the processor 20 adjusts the intensity of the light source 14 so that the absolute value of the difference E becomes smaller.

[0044] On the other hand, when the determination result of step S204 is Yes (i.e., when the feedback control has converged), the process proceeds to step S206. In step S206, the processor 20 determines whether or not the absolute value of the difference between the previous value and the current value of the intensity of the light source 14 is greater than a predetermined threshold value TH2. The previous value here is the initial value I set by the process of step S200, and the current value is the value of the intensity of the light source 14 adjusted when the process proceeds to step S206. According to the process of step S206, if the difference between the previous value and the current value is small (S206; No), the process proceeds to the end. On the other hand, if the difference is large (S206; Yes), the process proceeds to step S208, where the processor 20 updates the initial value I corresponding to the current position with the above-mentioned current value. That is, the initial value I stored in the initial value map is updated.

[0045] As described above, according to this feedback control, the appropriate intensity of the light source 14 when this feedback control converges is stored in the initial value map in a state associated with the current position of the moving body 1. Then, when the moving body 1 travels in the same position, the appropriate intensity of the light source 14 corresponding to that position is read from the initial value map and used as the initial value I of this feedback control. By storing the appropriate value of the intensity of the light source 14 in the past travel route in this way, it is expected that the feedback control of the light amount will converge early. In other words, it is expected that the adjustment of the intensity of the light source 14 in the next and subsequent travels can be completed in a shorter time. As described above, according to this feedback control, the illuminance inside the housing 26 of the road surface photographing device 10 can be made to quickly follow the change in the road surface environment, and the light source environment inside the housing 26 can be kept uniform. [Explanation of symbols]

[0046] REFERENCE SIGNS LIST 1 moving object, 2 road surface, 10 road surface photographing device, 12 camera, 14 light source, 16 electronic control unit (ECU), 18 top plate, 20 processor, 22 storage device, 24 illuminance sensor, 26 housing, 28, 36, 44 light shielding plate, 30 fixing portion, 32, 34 diffusion plate, 38 upper portion of light shielding plate, 40 lower portion of light shielding plate, 42 cover, 46 polarizing plate

Claims

[Claim 1] A moving body including a road surface photographing device that photographs an image of a road surface, The road surface photographing device is A camera is placed facing the road surface, At least one light source that irradiates light onto a photographing area of ​​the camera; a storage device that stores images of road surfaces captured in advance as map data; a processor that compares the map data with an image captured by the camera and estimates a self-position of the moving object; Including, The at least one light source is disposed so that reflected light of the emitted light is captured by the camera; The road surface photographing device further includes a light shielding plate disposed around the camera and the light source so as to shield the photographing area from external light. A moving object characterized by:

Images