A self-propelled vehicle

Abstract

The invention relates to a self-propelled vehicle (100) comprising a light emitting device for emitting light pulses outside the self-propelled vehicle (100) and an imaging device (1) for capturing images of an area located outside the self-propelled vehicle (100), the imaging device (1) comprising a shutter; wherein the light emitting device is coordinated with the shutter of the imaging device (1) such that the imaging device (1) captures an image (50) during at least part of the duration of a light pulse; the light emitting device is configured to emit light pulses, each light pulse having a duration of less than 5000 μs. The invention also relates to a method of capturing an image (50) from a self-propelled vehicle (100).

Description

Technical Field

[0001] This invention relates generally to the field of self-propelled vehicles, and more specifically to the field of navigation of self-propelled vehicles. Background Technology

[0002] Systems that assist in the navigation of self-propelled vehicles, and more specifically, systems that assist in the navigation of autonomous self-propelled vehicles, are known in the art. Some of these systems allow navigation in darkness (e.g., in unlit underground mines). However, most of these systems are not suitable for small, lightweight vehicles, such as aerial drones used for indoor inspections.

[0003] Kauhanen, H. 2008, “Close range photogrammetry-Structured light approach for machine vision aided harvesting,” ISPRS Archives, vol. XXXVII, part B5, pp. 75-80 (hereinafter referred to as “Kauhanen”), discloses a machine vision-aided harvester with two cameras and a projector. The projector projects patterned pulses of NIR light onto a tree, and the cameras capture images of the portions of the tree onto which the light pulses are projected. Spectral filters allow the differentiation of the patterned light in the captured images, which is in binary form, more specifically, black and white. Since the orientation of the cameras is known, images of the same target taken with both cameras can be used for photogrammetric tasks. In this way, by simultaneously capturing images of the same target with two cameras, the resulting data can be used for photogrammetric tasks, allowing the measurement of individual tree characteristics before contact with the individual tree. Using a database of felled trees is advantageous for planning subsequent harvesting events. However, the patterned light disclosed in Kauhanen does not allow for tracking of objects projected onto it because the projector follows the vehicle's movement, changing the position of the light source from one frame to the next. The result of this changing light source position is that the same object may appear very different in different frames, thus hindering object tracking. Furthermore, Kauhanen discloses the combined use of patterned light with two cameras. The machine vision-assisted harvester in Kauhanen requires multiple (more than one) cameras to simultaneously capture the same image from different viewpoints. Moreover, if the machine-assisted vision system disclosed in Kauhanen is implemented in a small vehicle (such as a small drone or the like), the distance between the cameras will be limited to only a few centimeters, thus limiting the depth of field to a very short range. This limitation makes it impossible for the vehicle to navigate in large spaces (e.g., a space where the distance from the small vehicle to the nearest wall is approximately ten times the distance between the cameras).

[0004] WO2019 / 084595A1 discloses a system and method for improving the signal-to-noise ratio in target tracking under low-light conditions. WO 2020 / 014706 A1 discloses visual navigation for a mobile device capable of operating under varying ambient lighting conditions. US 2017 / 083748 A1 discloses a system and method for detecting and tracking movable objects. US2018 / 348764 A1 discloses a system and method for providing easy-to-use release and automatic positioning for unmanned aerial vehicle (UAV) applications. US2018 / 143018A1 discloses a distance sensor that projects parallel patterns. Mueller K et al., “Combination of Wide Baseline Image Matching and Tracking for Autonomous UAV Approaches to a Window”, GYROSCOPY AND NAVIGATION, PLEIADES PUBLISHING, MOSCOW, vol. 10, no. 4, 1 October 2019 (2019-10-01), pages 206-215, XP037006472, disclose a combination of wide baseline image matching and tracking for autonomous UAV approach windows. Summary of the Invention

[0005] A first aspect of the invention relates to a self-propelled vehicle, preferably a small, lightweight vehicle, such as an aerial drone for indoor inspection, comprising:

[0006] A light emitting device for emitting light pulses beyond a self-propelled vehicle, such as visible light pulses and / or infrared light pulses; and

[0007] An imaging device for capturing images of an area located outside a self-propelled vehicle, the imaging device including a shutter; wherein

[0008] The shutters of the light emitting device and the imaging device are coordinated such that the imaging device captures an image during at least a portion of the duration of the light pulse, for example, during at least 25%, 50%, 75%, 90%, or 100% of the duration of the light pulse, preferably during all light pulses or during most of the light pulses. Thus, the coordination between the light emitting device and the imaging device ensures that an image of a region located outside the self-propelled vehicle is captured by the imaging device during at least a portion of the duration of the light pulse.

[0009] According to the first aspect:

[0010] The optical emitting device is configured to emit optical pulses, wherein the duration of each optical pulse is less than 5000 μs (e.g., less than 4000, 3000, 2000 or 1000 μs), but preferably greater than 10, 15, 20 or 25 μs.

[0011] It has been found advantageous to combine light pulses with durations less than 5000 μs with sufficiently high shutter speeds to avoid or adequately reduce image blur, such as exposure times less than 120, 100, 80, or 60 μs, but preferably greater than 10, 15, 20, or 25 μs. This combination ensures low blur and blurriness (i.e., sharp and clear images) in images captured when the self-propelled vehicle is moving in the dark, making it possible to process the images in image-matching-based applications. Using light pulses with short durations also minimizes energy consumption corresponding to the operation of the light source, which can be particularly advantageous in the case of small aerial vehicles. It also minimizes heat generation, which is sometimes an advantage, minimizing overheating of components, especially electronic components, and even more so, the light pulse source. Particularly advantageous is minimizing overheating of electronic components in small aerial vehicles, which cannot be located as far from the light pulse source as in larger vehicles, and which cannot use certain heat sinks that could be used in vehicles where the requirements for low weight and small size are not as stringent as in the case of small aerial vehicles. In some embodiments of the present invention, it is anticipated that future improvements to the image sensor will allow exposure times of less than 10 μs.

[0012] In some embodiments, the duration of each optical pulse is less than 4000, 3000, 2000, or 1000 μs. This results in the self-propelled vehicle generating less heat per optical pulse and thus allowing for the emission of more optical pulses in the same amount of time. This enables improved performance from a positioning estimation perspective and provides enhanced navigation capabilities.

[0013] In some embodiments, the duration of each light pulse is between 20 and 2000 μs. In some of these embodiments, the duration of each light pulse is between 20 and 50 μs. It has been found that a duration between 20 and 50 μs is particularly advantageous during most of the navigation time based on close-range photogrammetry (e.g., typically in indoor navigation) because very little energy is consumed in generating the light pulses, and the images captured when the self-propelled vehicle is moving in the dark have low blur and fuzziness (i.e., sharp and clear images), making it possible to process the images in image-matching-based applications.

[0014] In some embodiments, the self-propelled vehicle is configured to adapt the duration of light pulses during movement of the self-propelled vehicle to the light reflected by an object located within the field of view of the imaging device, the duration being adapted based on at least one image of the object previously captured by the imaging device. In this way, if the object shown in the image is too bright, which implies a loss of detail in the captured image, the duration of subsequent light pulses can be reduced to decrease the amount of light reaching the imaging device, thus reducing the brightness of the object shown in the captured image during the subsequent light pulses. Thus, the amount of light reflected by the object and detected by the image sensor of the imaging device can be adapted in real time. The amount of light reflected by the object depends on parameters such as the light reflectivity of the target surface of the object, the amount of natural light (if present), and the distance from the object to the image sensor of the imaging device. Preferably, the aperture and gain of the imaging device remain constant.

[0015] In some embodiments, each light pulse is initiated almost simultaneously with the exposure time, such that the time difference between the initiation of the light pulse and the initiation of the exposure time is less than 5, 2, or 0.5 μs. In some embodiments of these embodiments or in others, each light pulse ends almost simultaneously with the end of the exposure time, such that the time difference between the end of the light pulse and the end of the exposure time is less than 5, 2, or 0.5 μs. This can be achieved, for example, by ending the exposure time once the light sensor of the imaging device receives light intensity per unit time below a certain threshold. In some embodiments, the light pulse is initiated simultaneously with the start of the exposure time and / or ends simultaneously with the end of the exposure time. This demonstrates the advantage of fully utilizing the light pulses, because light emitted outside the exposure time, although reaching the imaging device, is not detected by the image sensor of the imaging device.

[0016] In some embodiments, the shutter is synchronized with at least some light pulses emitted by the light emitting device, preferably with all light pulses emitted by the light emitting device. In these embodiments, synchronization between the light pulses and the shutter means:

[0017] Each light pulse is activated simultaneously with the start of the exposure time, and

[0018] The light pulse ends at the same time as the exposure time ends.

[0019] The advantage of a shutter that is synchronized with all the light pulses emitted by the light emitting device is that it captures an image for each light pulse, thereby reducing the energy consumed by the light emitting device and the heat generated by the device.

[0020] During the normal operation of a self-propelled vehicle, shorter duration light pulses exhibit a lower duty cycle. In some embodiments, the self-propelled vehicle is configured to emit light pulses with a duty cycle between 0.05% and 10%. In some embodiments, the duty cycle is less than 10%, 5%, 1%, or 0.5%. It has been found advantageous to use a duty cycle between 0.1% and 0.3% for most of the time in close-range photogrammetry-based navigation in darkness (e.g., typically in indoor navigation), and more specifically, to use a duty cycle of 0.2% (e.g., 50 pulses per second, each pulse with a duration of 40 μs). In this way, the light emitting device generates less heat and has sufficient time to dissipate it. Furthermore, the light emitting device requires less energy for illumination. Moreover, the lower heat dissipation and lower illumination power requirements enable the use of smaller components with lower weight, which is advantageous, particularly in the fields of small vehicles and aircraft. Because the light emitting device emits light pulses, it undergoes cyclic heating (i.e., when emitting light pulses) and cooling (i.e., when not emitting light pulses). Advantageously, the duration of a light pulse is significantly shorter than the time between two consecutive light pulses (i.e., the time during which the light emitting device does not emit light). Therefore, the light emitting device has more time to dissipate the heat generated by the light source during the emission of the light pulse.

[0021] In some embodiments, the imaging device is configured to capture at least 50 frames or images per second. In other words, the imaging device is configured to operate at a frame rate of at least 50 frames per second. In some of these embodiments, the imaging device is configured to capture one of the following: at least 60, 70, 80, 90, or 100 frames per second, depending on the application of the frames.

[0022] Because the light emitting device has sufficient cooling time between two consecutive light pulses, the short duration of the light pulses emitted by the light emitting device allows for a high frame rate per unit time. Furthermore, the high frame rate per unit time helps reduce illumination and exposure variations between images captured while the self-propelled vehicle is moving. In this way, a large number of matches can occur between the captured images. This increases the efficiency of image matching and thus improves the performance of photogrammetric and / or computer vision algorithms, reducing computation time. These characteristics enable real-time processing of information, which is advantageous in the navigation of self-propelled vehicles. Therefore, this improvement in efficiency can improve the navigation of self-propelled vehicles.

[0023] In some embodiments, image matching is performed on images captured row by row (one after another). Therefore, images processed in image matching-based applications exhibit lower variation in lighting and exposure, which increases the matching in the image matching process and, in particular, allows for the identification of more connection points because more objects displayed in different images are common to more than one of those different images in a series of images, and thus more reference points can be used to position the self-propelled vehicle relative to those objects.

[0024] In some embodiments, the self-propelled vehicle is configured to be guided based on the relative positions between a first plurality of connection points in an image captured by an imaging device and a second plurality of connection points in another image captured by the imaging device, wherein the first plurality of connection points are a visual representation of the position of the self-propelled vehicle's surroundings, and the second plurality of connection points are a visual representation of the position of the self-propelled vehicle's surroundings represented by the first plurality of connection points. In some of these embodiments, each of the two images is captured sequentially.

[0025] In some embodiments, the self-propelled vehicle is configured to perform matching of tie points present in different images, wherein these tie points are preferably identifiable within the image captured during pulses of unpatterned light. It has been found that using tie points in image matching for navigation of the self-propelled vehicle is particularly advantageous because it allows for real-time navigation; in other embodiments, other image processing methods may be used. In some embodiments, the tie points of the images correspond to features of objects that are within the field of view of the imaging device at the time of image capture and are illuminated by pulsed but unpatterned light. Therefore, the image captured by the imaging device is not distorted relative to reality as the light emitted by the light emitting device is patterned.

[0026] In some embodiments, the light emitting device is configured to project unpatterned light, i.e., light without any shaped pattern (e.g., circles, lines, squares, and / or any other shape). In some embodiments, the light emitting device is configured to project light without any graphic form, which is intended to be captured wholly or partially by the imaging device. These embodiments are advantageous because patterned light or light including graphic forms may obscure details of the environment surrounding the self-propelled vehicle in the image captured by the imaging device. In other words, since the light emitting device is used to emit light pulses outside the self-propelled vehicle (i.e., directed towards the exterior of the self-propelled vehicle), and the imaging device is used to capture an image of the area outside the self-propelled vehicle, patterned light or light including graphic forms emitted by the light emitting device may obscure details of the area outside the self-propelled vehicle in the image captured by the imaging device. Therefore, the projection of patterned light or light including graphic forms may undesirably hinder the process of positioning the self-propelled vehicle based on the image relative to its surrounding environment.

[0027] In some embodiments, the shutter of the imaging device is a global shutter. In other embodiments, the shutter is a rolling shutter, where the rolling shutter effect present in the captured image simply needs to be appropriately accounted for (e.g., avoided or corrected). At the time of writing, in the context of this invention, rolling shutters are disadvantageous compared to global shutters because global shutters require less time to capture images (i.e., lower exposure time), thus requiring less time to be illuminated by the light-emitting device, and can achieve a higher frame rate per unit time. Therefore, rolling shutters are disadvantageous relative to global shutters due to their lower energy efficiency and their adverse effects on real-time triangulation. However, global shutters are much more expensive than rolling shutters, and rolling shutters and their associated software are currently being improved to handle the rolling shutter effect, so in the near future, rolling shutters may be advantageous in the context of this invention. Nevertheless, capturing images for navigation based on close-range photogrammetry (e.g., typically in indoor navigation) does not require a high-resolution global shutter that is much more expensive than a low-resolution global shutter.

[0028] In some embodiments, the imaging device is configured to convert light into electrical signals without using any electrical amplifiers to provide gain during the conversion, and more specifically, without using any electrical amplifiers to provide gain for the image signal generated by the imaging device's light sensor. The use of high-power light pulses avoids the need for electrical amplifiers associated with the imaging device's light sensor. High-power light pulses can be generated by providing high-power pulses to the light emitting device. The average electrical power provided to the light emitting device in each high-power light pulse (which includes electrical power converted into heat due to illumination) can be, for example, at least 70, 150, 300, or 400 W. In some of these embodiments, less than 35%, 30%, or 25% of the average electrical power of each high-power pulse is converted into heat. Electrical amplifiers are typically used to provide gain when converting light into electrical signals in an image sensor. The purpose of amplifiers is to artificially increase the brightness and exposure of an image, especially when shooting images in darkness. However, the use of amplifiers (image gain or ISO in conventional photography) introduces noise into the image, distorts pixel colors, and thus negatively affects image matching. Due to the high power of the light pulses, the system reduces image noise, improves image matching, enhances the reliability of vehicle positioning and navigation, and improves the quality of 3D models of image surfaces generated using photogrammetry.

[0029] In some embodiments, the imaging device includes an image sensor comprising an array of light sensors, and the imaging device is configured to bin the pixels of adjacent light sensors in the light sensor array to reduce the pixel resolution of the captured image. In this sense, the imaging device is configured to combine information detected by adjacent light sensors (or by adjacent pixels of the image sensor) to create a single pixel in the captured image. The advantage of this is that the same image sensor can be used to capture both images with low pixel resolution and images with higher pixel resolution. In some of these embodiments, the same image sensor is used both to capture low-resolution images for navigation purposes and to capture full-resolution images intended for subsequent processing tasks. Therefore, in these embodiments, a single image sensor can be used. The low-resolution image may be a grayscale image to allow for fast processing of low-resolution images while still allowing the utilization of the image sensor's sensitivity. The full-resolution image captured by the same image sensor may be a color image. In the context of this invention, a full-resolution image is an image captured without binning adjacent light sensors.

[0030] To capture low-resolution images, it is preferable to perform pixel merging on adjacent photosensitive elements of the image sensor, for example, in a 4x4 matrix. The purpose of this pixel merging is to increase the sensitivity of the image sensor by increasing the area of ​​each pixel in the image, reducing the amount of light required per photosensitive element to capture the image, thereby reducing the overall system exposure time, energy consumption, and heat generation. Therefore, pixel merging allows for the capture of more images in the same amount of time. Furthermore, pixel merging allows for maintaining the same field of view. Thus, because the number of pixels constituting the image is reduced through pixel merging, image processing for identifying connection points and for matching with other images is faster than the same processing applied to full-resolution images. This allows for improved performance of image matching algorithms and allows for reduced time between capturing an image and calculating the position of the self-propelled vehicle based on image matching, which is advantageous for proper vehicle navigation. In some of these embodiments, the resolution of the low-resolution image used for navigation purposes is expected to be less than one megapixel. In some embodiments, color information of the image pixels, other than grayscale tones, is not used when processing the image for navigation purposes.

[0031] In some embodiments, the same image sensor is used to capture full-resolution images (e.g., typically higher than five megapixels at nominal sensor resolution), removing pixel binning from the light sensor where necessary. After the self-propelled vehicle completes its mission, such as a tunnel inspection, the full-resolution images can be further post-processed.

[0032] In some embodiments, the onboard processing unit may be configured to measure the amount of surface overlap, i.e., the amount of overlap of image regions, between the most recently captured full-resolution image and the most recently captured image. When an overlap threshold (typically determined by the user based on the application) between the most recently captured full-resolution image and the most recently captured image is reached, pixel binning of the light sensor can be removed, and a full-resolution image that can be stored in onboard memory can be captured. In these embodiments, a higher-resolution image can be used instead of a full-resolution image. This method ensures overlap between full-resolution images (or higher-resolution images when capturing a higher-resolution image instead of a full-resolution image), which is advantageous for generating high-resolution photogrammetric 3D models from full-resolution images. Furthermore, this method allows minimizing the amount of full-resolution images captured, reducing the energy consumed by the self-propelled vehicle, as capturing higher-resolution images consumes more energy than capturing low-resolution images. Moreover, this low / high-resolution switching technique integrates two features: the generation of images for navigation and the generation of full-resolution images for 3D modeling, using the same image sensor, reducing the weight and cost of the self-propelled vehicle, and increasing its 3D modeling capabilities.

[0033] In some embodiments, the optical emitting device also emits optical pulses (or pulses) of a duration of up to 20,000 μs. These pulses (or multiple pulses) with longer durations are emitted after or before a pulse sequence, wherein each pulse has a duration of less than 5,000 μs. Unlike pulses with durations less than 5,000 μs, pulses with longer durations have been found to be generally unsuitable for locating vehicles during navigation via image-matching applications. Pulses with longer durations are suitable for capturing images with higher resolution, which are suitable for 3D modeling.

[0034] In some embodiments, the light emitting device uniformly illuminates the field of view of the imaging device while capturing an image. In this way, image matching depends on objects shown in the image that are inherently present in the environment surrounding the self-propelled vehicle and are not artificially created by the self-propelled vehicle, for example, by projecting patterned light.

[0035] In some embodiments, the light emitting device includes a light diffuser. The lower weight and space requirements of the light emitting components make it possible to incorporate additional components such as light diffusers into the self-propelled vehicle without unduly exceeding the weight and / or space requirements of the self-propelled vehicle. A light diffuser is advantageous because it allows the imaging device to be exposed to light with enhanced uniformity, reducing overexposure of the image sensor and thus exposing areas of excessive light that would lead to loss of detail in the captured image. A light diffuser is particularly advantageous in some embodiments of the invention where the light source of the light emitting device is much smaller than the field of view of the imaging device. Thus, the light diffuser can minimize overexposure of regions represented in the image (regions that are close to the light source during image capture). This is especially advantageous when the region represented in the central portion of the image is closer to the light source than the rest of the image. Furthermore, this enhanced uniformity reduces variations in shading, lighting, and exposure between different captured images, which improves image matching performance. This enhanced uniformity is particularly advantageous in close-range imaging, such as when the vehicle approaches an obstacle it wants to avoid.

[0036] In some embodiments, the light emitting device is configured to emit light pulses that illuminate the road in front of the propulsion vehicle and simultaneously illuminate the field of view of the imaging device.

[0037] In some embodiments, the imaging device includes an imaging lens, and the light emitting device includes a plurality of light sources, each oriented at an angle between 35° and 55° (e.g., an angle between 40° and 50°, such as 45°) to the optical axis of the imaging lens of the imaging device. In this way, the field of view of the imaging device is subjected to light with enhanced uniformity. Additionally, in this manner, the shadows of the particles generated by the light pulses are minimized, thus making it less likely that particles suspended in the air will appear in the image, thereby improving the quality of the image captured by the imaging device.

[0038] In some embodiments, the self-propelled vehicle is propelled by a thruster (e.g., a quadcopter), and the light source (e.g., an LED) of the light emitting device is located near the thruster, thereby improving the heat dissipation of the light source.

[0039] In some embodiments, the self-propelled vehicle is preferably a relatively small self-propelled vehicle, for example, a self-propelled vehicle whose size allows it to be housed in a cube having a side length of less than 1m (e.g., less than 50cm) and / or a weight of less than 4kg (e.g., less than 2kg).

[0040] According to another aspect of the invention, the present invention relates to a method for capturing an image of a target located outside a self-propelled vehicle from a self-propelled vehicle, the method comprising:

[0041] Light pulses are emitted towards the target, each pulse lasting less than 5000 μs, so that the target, located in darkness, is intermittently illuminated by the light pulses.

[0042] Capture the target image,

[0043] The step of capturing an image of the target is coordinated with a light pulse emitted toward the target, such that the image is captured for at least a portion of the duration of the light pulse.

[0044] In some embodiments, the method further includes a step of calculating vehicle positioning information for providing guidance instructions to the self-propelled vehicle, wherein the step of calculating the vehicle positioning information includes processing captured images according to an image matching algorithm (preferably Kalman filtering). In this way, the self-propelled vehicle can respond to obstacles in its surrounding environment by adjusting the guidance instructions and thus its future motion. Preferably, these steps are performed on the vehicle to which the guidance instructions are addressed, so that the vehicle can be guided even if satellite-based geolocation is denied.

[0045] The different aspects and embodiments of the present invention defined above can be combined with each other as long as they are compatible with each other.

[0046] Other advantages and features of the invention will become apparent from the following detailed description and will be particularly pointed out in the appended claims. Attached Figure Description

[0047] To provide a complete description and a better understanding of the invention, a set of accompanying drawings is provided. These drawings form an integral part of the specification and illustrate embodiments of the invention. These embodiments should not be construed as limiting the scope of the invention, but are merely examples of how the invention can be practiced. The drawings include the following figures:

[0048] Figure 1 This is a perspective view of a schematic representation of an imaging device for a self-propelled vehicle according to an embodiment of the present invention, wherein the imaging device holds a PCB.

[0049] Figure 2 This is a perspective view of an imaging device and a light emitting device of a self-propelled vehicle according to an embodiment of the present invention.

[0050] Figure 3 This is a schematic diagram illustrating an operational example of multiple components of a self-propelled vehicle according to an embodiment of the present invention.

[0051] Figure 4A first example is shown of an illumination power pulse applied to a light emitting device (top) and a trigger voltage pulse applied to a high-speed triggering unit (bottom) for triggering the illumination power pulse; the illumination power pulse repeats at the same period as the trigger voltage pulse; the light emitting device and the high-speed triggering unit are part of a self-propelled vehicle according to an embodiment of the present invention.

[0052] Figure 5 A second example is shown, illustrating an illumination power pulse applied to a light emitting device (top) and a trigger voltage pulse applied to a high-speed triggering unit (bottom) for triggering the illumination power pulse; the illumination power pulse repeats at the same period as the trigger voltage pulse; the light emitting device and the high-speed triggering unit are part of a self-propelled vehicle according to an embodiment of the invention.

[0053] Figure 6 A third example is shown, illustrating an illumination power pulse applied to a light emitting device (top) and a trigger voltage pulse applied to a high-speed triggering unit (bottom) for triggering the illumination power pulse; the illumination power pulse repeats at the same period as the trigger voltage pulse; the light emitting device and the high-speed triggering unit are part of a self-propelled vehicle according to an embodiment of the invention.

[0054] Figure 7 An example is shown of an illumination power pulse applied to a light emitting device (top) and a trigger voltage pulse applied to a high-speed triggering unit (bottom) for triggering the illumination power pulse; the light emitting device and the high-speed triggering unit are part of a self-propelled vehicle according to an embodiment of the invention.

[0055] Figure 8 This is a schematic diagram of a first image captured by an imaging device of a self-propelled vehicle according to an embodiment of the present invention.

[0056] Figure 8A It shows Figure 8 A schematic representation, in which example connection points for image matching applications are marked.

[0057] Figure 8B It shows in Figure 8A The image shown is a grayscale image with green connection points.

[0058] Figure 9 This is a schematic diagram of a second image captured by an imaging device of a self-propelled vehicle according to an embodiment of the present invention.

[0059] Figure 9A It shows Figure 9 A schematic representation, in which example connection points for image matching applications are marked.

[0060] Figure 9B It shows in Figure 9A The image shown is a grayscale image with green connection points.

[0061] Figure 10 A self-propelled vehicle according to an embodiment of the present invention is shown. Detailed Implementation

[0062] The following description should not be considered limiting, but only used to describe the main principles of the invention. Embodiments of the invention will be described by way of example with reference to the above-described drawings.

[0063] Figure 1 An imaging device 1 including a housing 12 is disclosed. The housing 12 of the imaging device 1 may have a front side, a rear side, an upper side, a lower side, and two lateral sides. The lens of the imaging device 1, preferably a fixed-focus lens 11, may be arranged on the front side of the imaging device 1. The housing 12 of the imaging device 1 houses an image sensor 14 (in... Figure 3 (Illustrated schematically) This housing protects the image sensor 14 from environmental factors and vibrations that may occur during navigation of the self-propelled vehicle. The housing 12 of the imaging device 1 includes a vehicle attachment device 13 for attaching the housing 12 of the imaging device 1 to a portion of the self-propelled vehicle. Preferably, the vehicle attachment device 13 is located on the rear and / or upper and / or lower side of the housing 12 of the imaging device 1.

[0064] In addition, the housing 12 of the imaging device 1 can hold four PCBs 2. Figure 1 Only two of the four PCBs 2 are shown. The other two PCBs 2 can be arranged around the fixed-focus lens 11 such that the four PCBs 2 are evenly distributed around the fixed-focus lens 11, and the angular distance between the PCB 2 and the nearest PCB 2 held by the housing 12 is 360° / 4 = 90°. Since the light source is intended to be mounted on the PCBs 2, the uniform distribution of the PCBs 2 around the fixed-focus lens 11 is advantageous because it allows the light source to be easily and uniformly distributed around the fixed-focus lens 11, so that the field of view of the fixed-focus lens 11 is illuminated by light with enhanced uniformity.

[0065] In other embodiments, the housing 12 of the imaging device 1 holds N PCBs 2, where N>4. As described above, the N PCBs 2 are preferably evenly distributed around the fixed-focus lens 11, such that the angular distance between the PCBs 2 and the nearest PCB 2 is 360° / N.

[0066] Furthermore, it is advantageous that the PCB 2 is located outside the field of view of the imaging device 1. As a result, the imaging device 1 is able to capture more details of the surrounding environment of the propulsion vehicle, thereby improving the performance of the image matching algorithm.

[0067] Preferably, the PCB 2 is attached to the housing 12 of the imaging device 1 such that the PCB 2 does not directly contact the housing 12. This results in less heat generated by the light source being transferred to the housing 12 of the imaging device 1. More preferably, most of the surface of each PCB 2 is uncovered to improve heat dissipation. This is particularly advantageous because more heat is transferred into the air via convection as the self-propelled vehicle moves.

[0068] continue Figure 1 Each PCB 2 may be adapted to mount a light source, such as multiple LEDs (e.g., four to sixteen LEDs), for illuminating the field of view of the imaging device, rather than LEDs with lower illumination power, such as LEDs used only for signaling purposes. For example, the light source may include multiple high-power LEDs, such as multiple LEDs, where each LED is capable of handling a rated current of at least 1, 2, or 3 A and / or at least 3, 5, or 10 W of electrical power. Preferably, each PCB 2 defines a plane forming a 45° angle with respect to the optical axis of the fixed-focus lens 11. Each PCB 2 may be thermally optimized, including heat transfer paths between PCB 2 layers and large areas of copper to reduce thermal resistance between the LED junctions and the air.

[0069] like Figure 2 As shown, LEDs 3 can be arranged uniformly around the fixed-focus lens 11. Thus, LEDs 3 can be mounted on several PCBs 2 (e.g., on four PCBs 2), evenly arranged around the lens 11. The LEDs 3 on each PCB 2 can be arranged as close as possible to the edge of the PCB 2 closest to the fixed-focus lens 11. On each PCB 2, eight LEDs 3 can be arranged in parallel rows, four LEDs 3 per row. To improve the identification of connection points, the light emitted by the LEDs 3 can be diffused. All LEDs 3 can emit visible light of the same color. To improve the identification of connection points, the light emitted by the LEDs 3 can be diffused light. Figure 2 As shown, the plane defined by each PCB 2 can form a 45° angle with respect to the optical axis of the lens 11.

[0070] like Figure 2 As shown, LED 3 (or other light sources) can be arranged such that each LED 3 is oriented perpendicular to the plane defined by the PCB 2 on which it is mounted, thus allowing the LED 3 to be oriented at a 45° angle to the optical axis of the fixed-focus lens 11. Figure 1 and Figure 2 In this configuration, LED3 is located on the front side of the housing 12 of the imaging device 1, that is, on one side of the imaging device 1 where the fixed-focus lens 11 is positioned. Preferably, LED3 is located slightly behind the fixed-focus lens 11 (this is in...). Figure 3(This can be more easily understood in the text). This does not preclude other embodiments (not shown) in which an additional light source (e.g., LED 3) is arranged in a different position relative to the fixed-focus lens 11 (e.g., on the rear side of the housing 12 of the imaging device 1, preferably facing the opposite direction to that of the fixed-focus lens 11), so that the light travels more indirectly because it is reflected more times on the object surrounding the self-propelled vehicle.

[0071] like Figure 3 As shown, a high-speed trigger unit 21 can be mounted on each PCB 2. The high-speed trigger unit 21 turns on and off LEDs 3 mounted on the same PCB 2. The high-speed trigger unit 21 has electronics that send lighting power pulses to the LEDs 3 for illumination. The lighting power pulse is initiated when the high-speed trigger unit 21 receives a signal and ends when the high-speed trigger unit 21 receives another signal, preferably when a low-power signal such as a TTL signal is received. The lighting power pulse can be less than 6, 5, or 4 A, although it is preferred to be greater than an average current of 0.5, 1, or 2 A. The high-speed trigger unit 21 has a fast response, meaning that the high-speed trigger unit 21 requires only a short time between receiving a signal and the start or end of the corresponding lighting power pulse, for example, less than 5, 2, or 0.5 μs. This can be achieved by minimizing the electrical interference between the portion of the high-speed trigger unit 21 used to send the lighting power pulse and the portion of the high-speed trigger unit 21 used to process the signal received by the high-speed trigger unit 21. In embodiments where the exposure time ends just before the high-speed trigger unit 21 receives a corresponding signal to end the illumination power pulse, a fast response is advantageous because this minimizes the amount of time the light source remains on outside the exposure time. In some of these embodiments where a fast response is advantageous, the imaging device 1 generates a signal to end the illumination power pulse, such as a signal consisting of a sudden voltage drop, and sends this signal to the high-speed trigger unit 21 when the image sensor of the imaging device 1 has ended the predetermined exposure time of the processing unit 4 to which the imaging device 1 is connected. In other embodiments (not shown), a light source other than the LED 3 can be used, which must respond quickly enough to the signal controlling the emission of the light pulse and preferably be as efficient as possible to consume very little energy, especially when emitting the light pulse.

[0072] A power supply 5, such as a lithium-ion battery, for providing illumination energy to the LED 3 can be connected to the high-speed trigger unit 21 of the LED 3. Additionally, the high-speed trigger unit 21 can be connected to an image sensor, such as a global shutter image sensor 14 (e.g., a CMOS image sensor), which has high sensitivity to light and responds quickly to the signal triggering image capture. For simplicity, Figure 3The connection shown is only with PCB 2 located in the lower half of the imaging device 1 shown; however, it will be understood that similar connections may exist between other PCBs 2, their high-speed trigger units 21, image sensor 14, and power supply 5. Power supply 5 provides the energy required for light pulses to be emitted by LED 3.

[0073] Part of the light pulse emitted by LED 3 can be reflected, for example, by the wall of an underground cave that the self-propelled vehicle is navigating, and then the light pulse can pass through lens 11 to reach image sensor 14. Image sensor 14 can be connected to processing unit 4 mounted on the self-propelled vehicle. Processing unit 14 can be configured to send trigger pulse 8 to image sensor 14 to trigger image capture. In this way, processing unit 4 controls the activation of shutter of imaging device 1 and can set the exposure time, although preferably, the trigger pulse 8 that triggers image capture only initiates image capture, and the duration (or width) of trigger pulse 8 does not provide the shutter with information about the duration of the exposure time for image capture. Preferably, image sensor 14 is automatically deactivated when it stops receiving light.

[0074] Upon receiving the trigger pulse 8 that triggers image capture, the global shutter image sensor 14 can send the trigger pulse 6 to the high-speed trigger unit 21 of the LED 3. Upon receiving the trigger pulse 6, the high-speed trigger unit 21 can cause the power supply 5 to provide an illumination power pulse to the LED 3 for the LED 3 to emit a light pulse. Preferably, the duration (or width) of each trigger pulse 6 received by the high-speed trigger unit 21 can determine the duration of the illumination power pulse caused by the high-speed trigger unit 21, and thus determine the duration of the light pulse. To prevent overheating of the LED 3 due to excessively long durations of the trigger pulse 6, a pulse duration limiter 7, also known as a pulse width limiter, can be arranged between the image sensor 14 and the high-speed trigger unit 21 of the LED 3.

[0075] exist Figure 3 The diagram schematically illustrates the path 23 of the light pulse following the light pulse that leaves the LED 3 before being deflected by the surrounding environment of the self-propelled vehicle (e.g., refracted or reflected by objects in the surrounding environment). Preferably, this path 23 does not enter the central volume 9 of the field of view of the imaging device 1, which extends from the fixed-focus lens 11 parallel to the optical axis of the fixed-focus lens 11.

[0076] In image capture, the light detected by image sensor 14 is converted into an electrical signal 22 defining the original image. The original image is transmitted to processing unit 4, for example, via a high-speed data bus. It may be advantageous not to compress the original image, so that the original image retains details that would be lost if the original image were compressed. Processing unit 4 can be configured to process the original image received from image sensor 14. For example, processing unit 4 can be configured to identify connection points in subsequent original images during the movement of the self-propelled vehicle to determine the trajectory followed by the self-propelled vehicle relative to its surrounding environment, and correct its trajectory, for example, to avoid collisions with its surrounding environment. Processing unit 4 can be configured to adapt the image rate per unit time when analyzing the original image (e.g., when performing image matching on the original image). Processing unit 4 can be configured to adapt pixel binning of imaging device 1 when analyzing the captured original image (e.g., when performing image matching on the original image). The processing unit 4 can be configured to adapt the exposure time and / or the duration of the light pulse in subsequent captures of one or more images when analyzing the captured original image (e.g., when performing image matching on the original image).

[0077] Processing unit 4 can be configured to store full-resolution images and / or higher-resolution images in a different onboard memory than the onboard memory that stores images with lower pixel resolution.

[0078] Figure 10 An example self-propelled vehicle 100, more specifically, an unmanned aerial vehicle, is disclosed, which includes... Figure 2 The imaging device 1 shown. (As shown) Figure 10 As shown, the imaging device 1 of the self-propelled vehicle 100 is an imaging device 1 used to capture images of the area outside the self-propelled vehicle 100. The drone 100 includes four arms, each arm having a motor 102 arranged on its top and near the end of the arm. Each motor 102 can be coupled to a blade in such a way as to form a propeller.

[0079] like Figure 10 As shown, the light emitting device 1 of the self-propelled vehicle 100 includes a light emitting device for emitting light pulses to the outside of the self-propelled vehicle 100, and more specifically, includes a light source, such as an LED, arranged along each arm of the self-propelled vehicle 100. Therefore, the self-propelled vehicle 100 has the advantage of enhanced illumination power and illumination uniformity. To further enhance illumination uniformity, it is advantageous that the arms have different orientations (i.e., the arms are not parallel to each other). Figure 10All four arms with light sources are disclosed, but in other embodiments, fewer arms may have light sources. In other different embodiments, the self-propelled vehicle 100 may have more than four arms; in some of these embodiments, the light source extends along each arm, while in other embodiments, the light source extends along only some of the arms.

[0080] like Figure 10 As shown, the field of view of the light source preferably facing the imaging device 1 is oriented toward one side of the drone 100. Figure 10 In other embodiments not shown, in order to enhance illumination uniformity, some light sources facing the UAV 100 are oriented to a different side from the field of view of the imaging device 1.

[0081] like Figure 10 As shown, the light sources are preferably arranged in two rows parallel to each arm of the UAV 100.

[0082] A light source arranged along the arm of the self-propelled carrier 100 can be mounted on PCB 101 and configured to emit light pulses simultaneously and for the same duration as a light source mounted on PCB 2. This increases the illumination power of the light pulses, which allows for the capture of images with higher quality and / or allows for a reduction in the exposure time of the imaging device 1 during image capture.

[0083] Furthermore, since the arm is close to the propeller, the airflow generated by the propeller helps to dissipate heat from the light source, so it is advantageous to place the light source in the arm.

[0084] The self-propelled vehicle 100 includes a leg 103 arranged between the longitudinal center of each arm and the end of the arm, and a motor 102 is provided at the top of the leg 103. The leg 103 is advantageously used to support the weight of the self-propelled vehicle 100 and to provide stability to the self-propelled vehicle 100 when it is not in flight.

[0085] The self-propelled vehicle 100 also includes a center plate 105 attached to the four arms and the imaging device 1. The imaging device 1 can be attached to the edge of the center plate 105 via a rear plate 106, a vibration damping device 108, an upper plate 107, and a lower plate (not shown). The upper plate 107 and the lower plate are preferably L-shaped plates.

[0086] The upper plate 107 can be threaded to the upper side of the housing 12 of the imaging device 1. The lower plate can be threaded to the lower side of the housing 12 of the imaging device 1. The upper plate 107 and the lower plate can be attached to the rear plate 106 via a vibration damping device 108. The vibration damping device 108 is preferably made of rubber. The rear plate 106 is attached to the plate 105.

[0087] In some embodiments (not shown), the connection between the rear plate 106 and the upper plate 107 allows the rear plate 106 to achieve a specific orientation angle relative to the upper plate 107 (e.g., with). Figure 10 (Different orientation angles are shown). This achieves specific orientation angles for the imaging device 1 and the light source attached to the housing 12 relative to the self-propelled vehicle 100, enabling the imaging device 1 to operate from angles different from those shown. Figure 10 The image is captured in the direction indicated. For example, the imaging device 1 may be oriented downwards, such that the image captured by the imaging device 1 shows a larger portion of the ground above which the self-propelled vehicle 100 is flying. Alternatively, for example, the imaging device 1 may be oriented upwards, such that the image captured by the imaging device 1 shows a larger portion of the airspace above which the self-propelled vehicle 100 is flying below.

[0088] The plate 105 can support the housing 104, which houses electronic components, such as the processing unit 4 or onboard memory for storing full-resolution and / or low-resolution images. The processing unit 4 can be electrically connected to the imaging device 1 via wire 109.

[0089] The self-propelled vehicle 100 includes a battery connector 110 for supplying power to the electronic components of the self-propelled vehicle 100 (e.g., motor 102, processing unit 4, imaging device 1, and light emitting device).

[0090] Figure 4 An example distribution of the illumination power pulse for the light emitting device 3 is shown. For example, the illumination power pulse can be sent to the light source 3 by the trigger unit 21. The trigger unit 21 can obtain the energy required to activate the illumination power pulse from the power supply 5. Figure 4 Within this, four zones corresponding to different time ranges can be distinguished. Zones 31 and 33 have more lighting power pulses per unit time, while zone 32 has fewer lighting power pulses per unit time. Figure 4 In the process, when the voltage increases to a certain voltage value (i.e., in each approximately vertical line), the light source 3 is turned on.

[0091] like Figure 4 As shown, the first region 31 begins at 0 ms and ends at approximately 240 ms. The illumination power pulses in the first region 31 and the third region 33 have a period of 20 ms and reach a maximum voltage of approximately 11.0 V. The third region 33 begins at approximately 590 ms and extends to the end of the curve, but its actual duration may be longer than that. Figure 4The length shown is longer. When subjected to illumination power pulses from the first region 31 or the third region 33, the light pulses emitted by the light source 3 are suitable for capturing images with low resolution because the number of illumination power pulses per unit time is high (and therefore there are a high number of light pulses).

[0092] The second region 32 presents two illumination power pulses at approximately 310 ms and 370 ms, respectively. The second region 32 has a duration of approximately 120 ms. During this time, the processing unit 4 refreshes the image sensor buffer and performs some configuration related to the shutter time for subsequent high-resolution image capture. The illumination power pulse of the second region 32 at 310 ms reaches a maximum voltage of approximately 11.0 V. The illumination power pulse of the second region 32 at 370 ms reaches a maximum voltage of approximately 11.0 V. When subjected to the illumination power pulses of the second region 32, the light pulses emitted by the light source 3 are suitable for capturing images with full resolution because the number of illumination power pulses per unit time is small (and therefore there is a low number of light pulses). Thus, the light source 3 can remain on for the longer duration without burning out, allowing more time to capture each image (i.e., allowing longer exposure times for each captured image). These full-resolution images can be used to generate 3D models of the surface of the environment surrounding the self-propelled vehicle.

[0093] As described above, each illumination power pulse in the second region 32 lasts longer than the illumination power pulses in the first region 31 or the third region 33. The light source 3 produces light throughout the entire duration (or width) of each illumination power pulse. Figure 4 The diagram shows a first example point 71 where the lighting power pulse of light source 3 is turned on and a second example point 72 where the pulse of light source 3 is turned off. Additionally, Figure 4 The duration 73 of the light pulse generated by the illumination power pulse is shown. Once the light source is turned off at point 72, energy consumption is greatly reduced until the next illumination power pulse, because the light sources 3 do not consume illumination energy while they are off. From point 72, where the light sources 3 are turned off, until the next illumination power pulse, most of the energy consumption is likely due to the unloading of parasitic capacitance. Reference Figure 7 This will allow for a better understanding of these different parts of the lighting power pulse.

[0094] Figure 4 The fourth region 34 extends approximately between 420ms and 580ms. During the fourth region 34, the full-resolution image captured in the second region 32 is transmitted to the processing unit 4 for storage in the onboard memory.

[0095] also, Figure 4Trigger voltage pulses 41 and 42 are shown with the same cycle as the lighting power pulses 32, 31, and 33. From... Figure 4 As can be seen, trigger pulses 41 and 42 start and end almost simultaneously with the light pulse. Trigger voltage pulses 41 and 42 are applied by the imaging device 1 to the high-speed trigger unit 21, causing the high-speed trigger unit to send illumination power pulses 32, 31, and 33. Trigger pulse 41 is suitable for capturing full-resolution images. Trigger pulse 42 is suitable for capturing lower-resolution images. Figure 4 The diagram shows that the trigger pulse 41 corresponding to the second region 32 lasts for a longer time than the trigger pulse 42 corresponding to the first or third regions 31, 33.

[0096] Figure 4 The diagram shows that each trigger pulse 41, 42 received by the high-speed trigger unit 21 begins almost simultaneously with the illumination power pulse turning on the light source 3, as it can be observed that the beginning of the trigger pulses 41, 42 is almost collinear with the vertical portion of the illumination power pulse. In embodiments where the high-speed trigger unit 21 receives the trigger voltage pulses 41, 42 as soon as the exposure time begins, higher energy efficiency is achieved compared to turning on the light source 3 before the exposure time begins.

[0097] also, Figure 4 The trigger pulses 41 and 42 of the imaging device 1 are shown to reach a maximum value of approximately 2.5V.

[0098] exist Figure 5 The light source 3 is divided into two regions corresponding to different time ranges. The first region 31 has a high number of illumination power pulses per unit time (and therefore a high number of light pulses), and the second region 32 has a low number of illumination power pulses per unit time (and therefore a low number of light pulses). When the voltage of the illumination power pulse increases to above a certain voltage value (i.e., in each approximately vertical line), the light source 3 is turned on.

[0099] like Figure 5 As shown, the first region 31 starts at 0 ms and ends at approximately 150 ms. Each illumination power pulse in the first region 31 reaches a maximum voltage of approximately 11.0 V. The second region 32 starts at approximately 230 ms and ends at approximately 310 ms.

[0100] Since there are many illumination power pulses (and light pulses) per unit time, the light pulses emitted by the light source 3 when it is subjected to the illumination power pulse of the first region 31 are suitable for capturing images with low resolution.

[0101] The second region 32 presents two light pulses at approximately 230 ms and 300 ms, respectively. The illumination power pulse of the second region 32 at 230 ms reaches a maximum voltage of approximately 11.0 V. The illumination power pulse of the second region 32 at 300 ms also reaches a maximum voltage of approximately 11.0 V.

[0102] The light pulses emitted by light source 3 when subjected to illumination power pulses from the second region 32 are suitable for capturing images with full resolution because the number of illumination power pulses (and light pulses) per unit time is small. Therefore, light source 3 can remain on for longer periods without burning out, allowing more time to capture each image.

[0103] Figure 5 Trigger pulses 41 and 42 are shown, repeating with the same period as the lighting power pulses 32, 31, and 33. Figure 5 As can be seen, trigger pulses 41 and 42 start and end almost simultaneously with the light pulse. Trigger voltage pulses 41 and 42 are applied by the imaging device 1 to the high-speed trigger unit 21, causing the high-speed trigger unit to send illumination power pulses 32 and 31. Trigger pulse 41 is suitable for capturing full-resolution images. Trigger pulse 42 is suitable for capturing lower-resolution images. Figure 5 The diagram shows that the duration of the trigger pulse 41 in the second region 32 is longer than the duration of the pulse 42 in the first region 31.

[0104] Figure 6 It shows Figure 4 or Figure 5 A magnified view of region 31 in the middle. Figure 6 Disclosed is a method where each trigger pulse 42 received by the high-speed trigger unit 21 is initiated almost simultaneously with the illumination power pulse turning on the light source 3. In embodiments where the high-speed trigger unit 21 receives the trigger voltage pulse 42 as soon as the exposure time begins, higher energy efficiency is achieved compared to turning on the light source 3 before the exposure time begins.

[0105] also, Figure 6 The trigger pulse 42 received by the high-speed trigger unit 21 has a frequency of 50 pulses per second, and the illumination power pulse of the light source 3 exhibits the same frequency, i.e., 50 pulses per second. The trigger pulse of shutter 1 reaches a maximum value of 2.5V.

[0106] Figure 7 It shows Figure 6A magnified view of the illumination power pulse 311 in the first region 31. The illumination power pulse 311 turns on the light source 3 for approximately 100 μs and turns off the light source 3 for approximately 140 μs. The trigger pulse 43 received by the high-speed trigger unit 21 is activated almost simultaneously with the illumination power pulse 311 turning on the light source 3. The imaging device 1 stops capturing images before the electronics associated with the light emitting device are completely turned off. If Figure 7 The lighting power pulse 311 and trigger pulse 43 shown are... Figure 6 If the frequency disclosed in the report occurs, the light emitting device 3 will exhibit a duty cycle of approximately 0.2%.

[0107] The energy consumed by illumination occurs when light source 3 is turned on (e.g., when...). Figure 7 (Approximately between 100 and 140 μs). When the light source 3 is turned off, a portion of the remaining electronic components requires electrical energy (e.g., it may be necessary to unload parasitic capacitance), but the amount of energy consumed by that portion of the electronic components when the light source 3 is turned off is negligible compared to the energy consumed when the light source 3 is turned on.

[0108] Figure 8 and Figure 9 Schematic diagrams 50 and 60 of images captured by imaging device 1 are disclosed. These images are examples of low-resolution grayscale images suitable for rapid processing by processing unit 4 when performing image matching applications.

[0109] After capturing an image (such as a grayscale image represented by 50 or a grayscale image represented by 60), the processing unit 4 searches for and identifies connection points 511 and 611 within the captured image and saves connection points 511 and 611 in the onboard memory. Figure 8A and 9A Example connection points 511 and 611 are shown respectively, which have been identified by processing unit 4 in the images represented by 50 and 60 respectively.

[0110] exist Figure 8A and 9A As can be observed, connection points 511 and 611 correspond to features in the images represented by 50 and 60, respectively, which have high color contrast (e.g., high grayscale contrast) with their adjacent contours, and simultaneously have a specific shape different from the shape adjacent to the features. If the representations (figures) were black and white, the high contrast of gray shadows might not be noticeable in representations 50 and 60; however, they are respectively... Figure 8B and 9BThis can be understood. In this way, the processing unit 4 identifies the features of the surrounding environment of the self-propelled vehicle represented by connection points 511, 611, and performs matching of connection points between subsequently captured images to calculate the position of the vehicle relative to its surrounding environment, thus enabling navigation based on the calculated position.

[0111] More specifically, Figure 8A Multiple interconnected bars are shown, wherein the processing unit 4 has identified a first plurality of connection points 511 of the intersecting bars and a second plurality of connection points 511 of the protruding portions of the bars.

[0112] Figure 9A Multiple interconnected bars are shown, wherein the processing unit 4 has identified a first plurality of connection points 611 corresponding to the area where the bar changes direction, a second plurality of connection points 611 corresponding to the area where the bar intersects with bars having different longitudinal directions, a third plurality of connection points 611 of the intersecting bars, and a fourth plurality of connection points 611 of the protruding portions of the bars.

[0113] In this way, the same features of the surrounding environment of the self-propelled vehicle can be identified in continuously captured images, so the processing unit 4 of the self-propelled vehicle can calculate the position and relative motion of the self-propelled vehicle with respect to the features of the surrounding environment.

[0114] In this document, the term “comprising” and its derivatives (such as “including”) should not be understood in the sense of exclusion; that is, these terms should not be interpreted as excluding the possibility that the content described and defined may include other elements, steps, etc.

[0115] On the other hand, the present invention is obviously not limited to the specific embodiments described herein, but also includes any variations that can be considered by those skilled in the art within the general scope of the invention as defined in the claims (e.g., choices regarding materials, dimensions, components, configurations, etc.).

Claims

1. A self-propelled vehicle (100), comprising: A light emitting device used to emit light pulses; as well as An imaging device (1) for capturing images of the environment surrounding the self-propelled vehicle (100), the imaging device (1) comprising a light sensor and a shutter; wherein, The light emitting device is coordinated with the shutter of the imaging device (1) such that the imaging device (1) captures an image (50) of the surrounding environment of the self-propelled vehicle (100) for at least a portion of the duration of the light pulse. The light emitting device is configured to emit light pulses, wherein the duration of each light pulse is less than 5000 μs and the self-propelled vehicle is a drone; Its features are, The imaging device (1) is configured to convert light into electrical signals without using any electrical amplifier to provide gain for the image signal generated by the light sensor of the imaging device, and The light pulse is a high-power light pulse to reduce image noise in the image captured by the imaging device (1).

2. The self-propelled vehicle (100) according to claim 1, wherein, The duration of each optical pulse is less than 2000 μs, and the duration of each optical pulse is greater than 20 μs.

3. The self-propelled vehicle (100) according to claim 1, wherein, The self-propelled vehicle (100) is configured to adapt the duration of the light pulse to light reflected by an object located within the field of view of the imaging device (1) during movement of the self-propelled vehicle (100), the duration being adapted based on at least one image (50) of the object previously captured by the imaging device (1).

4. The self-propelled vehicle (100) according to any one of the preceding claims, wherein, The self-propelled vehicle (100) is configured to emit light pulses with a duty cycle between 0.05% and 10%.

5. The self-propelled vehicle (100) according to any one of claims 1-3, wherein, The imaging device (1) is configured to capture at least 50 images (50) per second.

6. The self-propelled vehicle (100) according to any one of claims 1-3, wherein, The self-propelled vehicle (100) is configured to be guided by the relative position between a first plurality of connection points (511) of an image (50) captured by the imaging device (1) and a second plurality of connection points (511) of another image (50) captured by the imaging device (1), wherein the first plurality of connection points (511) is a visual representation of the position of the surrounding environment of the self-propelled vehicle (100) and the second plurality of connection points (511) is a visual representation of the position of the surrounding environment of the self-propelled vehicle (100) represented by the first plurality of connection points (511).

7. The self-propelled vehicle (10) according to any one of claims 1-3, wherein, The self-propelled vehicle (100) is configured to perform matching of connection points (511) present in different images (50), wherein the connection points (511) are identifiable within the images (50) captured during pulses of unpatterned light.

8. The self-propelled vehicle (100) according to any one of claims 1-3, wherein, The shutter of the imaging device (1) is a global shutter.

9. The self-propelled vehicle (100) according to any one of claims 1-3, wherein, The imaging device (1) includes an image sensor (14) which includes an array of light sensors, and the imaging device (1) is configured to perform pixel merging on adjacent light sensors of the light sensor array to reduce the pixel resolution of the captured image (50).

10. The self-propelled vehicle (100) according to any one of claims 1-3, wherein, The light emitting device includes a light diffuser.

11. The self-propelled vehicle (100) according to any one of claims 1-3, wherein: The imaging device (1) includes an imaging lens, and The light emitting device includes a plurality of light sources (3), each light source (3) being oriented in a direction forming an angle between 35° and 55° relative to the optical axis of the imaging lens (11) of the imaging device (1).

12. A method for capturing images (50) from a self-propelled vehicle (100), the method comprising: Light pulses are emitted toward the target, wherein the duration of each light pulse is less than 5000 μs; as well as Capture an image (50) of the target, which is located in the environment surrounding the self-propelled vehicle (100); in, The image was captured by an imaging device including a light sensor; The step of capturing an image of the target (50) is coordinated with a light pulse emitted toward the target, such that an image (50) is captured during at least a portion of the duration of the light pulse; and The self-propelled vehicle is an unmanned aerial vehicle (UAV). Its features are, The imaging device (1) converts light into electrical signals without using any electrical amplifier to provide gain for the image signal generated by the light sensor of the imaging device, and The light pulse is a high-power light pulse to reduce image noise in the image captured by the imaging device (1).

13. The method of claim 12, further comprising the step of calculating vehicle positioning information to provide guidance instructions to the self-propelled vehicle (100), wherein the step of calculating the vehicle positioning information includes processing the captured image (50) according to an image matching algorithm.

14. The method according to any one of claims 12-13, wherein, The self-propelled vehicle (100) is as described in any one of claims 1-11.

Images