Tracking device that increases the usability of the work surface

Abstract

A trajectory tracking device is provided for detecting the displacement relative to a working surface, the trajectory tracking device comprising: a first image sensor, the first image sensor being used to output a first image frame; a second image sensor, the second image sensor being used to output a second image frame; a lens; and a first light source, the first light source being used to illuminate the working surface to generate reflected light and scattered light, wherein the reflected light is incident on the first image sensor without passing through the lens, and the scattered light is incident on the second image sensor through the lens.

Description

[0001] This application is a divisional application of the Chinese invention patent application with application number 201810497372.8, application date May 22, 2018, and name “Trajectory tracking device capable of increasing the applicability of working surface”. Technical Field

[0002] The present invention relates to an optical trajectory tracking device, and more particularly to an optical trajectory tracking device capable of increasing the applicability of a working surface. Background Art

[0003] The optical displacement detection device generally comprises a light source, an image sensor and a processor. The light source is used to illuminate the working surface. The image sensor is used to obtain reflected light from the working surface and output pixel data. The processor calculates the displacement of the displacement detection device relative to the working surface based on the pixel data.

[0004] However, conventional optical displacement detection devices have the limitation that they cannot operate normally on all working surfaces. For example, a displacement detection device suitable for a reflective surface may not be suitable for a light-absorbing surface, and vice versa.

[0005] Therefore, an optical displacement detection device that can operate on any working surface is needed. Summary of the invention

[0006] The invention provides a track tracking device which can be applied to both smooth and rough working surfaces.

[0007] The present invention also provides a track tracking device capable of calculating the distance between an image sensor and a work surface. The track tracking device can also adjust the displacement ratio according to the distance calculation to output substantially the same counts per inch, thereby improving user experience.

[0008] The present invention provides a trajectory tracking device for detecting the displacement relative to a work surface. The trajectory tracking device comprises a first image sensor, a second image sensor, a lens and a first light source. The first image sensor is used to output a first image frame. The second image sensor is used to output a second image frame. The first light source is used to illuminate the work surface to generate reflected light and scattered light, wherein the reflected light is incident on the first image sensor without passing through the lens, and the scattered light is incident on the second image sensor through the lens.

[0009] In the present invention, the working surface is, for example, a desktop, a floor, a carpet surface, a glass surface, a tile surface or other surfaces that the track tracking device can travel on. The track tracking device can use different operation modes to be applicable to different working surfaces.

[0010] In order to make the above and other purposes, features and advantages of the present invention more obvious, the following will be described in detail with reference to the accompanying drawings. In addition, in the description of the present invention, the same components are represented by the same symbols, which are first described here. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A schematic diagram of a trajectory tracking device according to an embodiment of the present invention;

[0012] Figure 2 is a schematic diagram of a trajectory tracking device according to another embodiment of the present invention;

[0013] Figure 3 A schematic diagram of a trajectory tracking device according to another embodiment of the present invention.

[0014] Description of Reference Numerals

[0015] 100 Tracking Device

[0016] 11 Image Sensor

[0017] 12. First Light Source

[0018] 13 Second Light Source

[0019] 14 Lens

[0020] 15 Processor

[0021] 16 The third light source

[0022] 17 Housing DETAILED DESCRIPTION

[0023] The present invention is applied to an optical trajectory tracking device, which can be suitable for any working surface, including strongly reflective surfaces such as glass surfaces and light-colored tile surfaces, and weakly reflective surfaces such as carpet surfaces and dark-colored tile surfaces, so as to effectively increase the operable working surface of the trajectory tracking device.

[0024] Please refer to Figure 1 , which is a schematic diagram of a trajectory tracking device 100 according to an embodiment of the present invention. The trajectory tracking device 100 is, for example, an optical mouse, a sweeping robot, or an optical device that can be used to move on a work surface S and detect a displacement or trajectory relative to the work surface S. The work surface S is, for example, a desktop, a floor, a carpet surface, a glass surface, a tile surface, or other surfaces on which the trajectory tracking device 100 can travel, which is determined according to different applications.

[0025] The trajectory tracking device 100 includes a housing 17, and the material thereof is not limited. The bottom surface of the housing 17 has an opening 100H for the light source and sensor inside to detect the surface features of the working surface S and calculate the displacement accordingly.

[0026] An image sensor 11, a first light source 12, a second light source 13, a lens 14, and a processor 15 are disposed in a housing 17 of the trajectory tracking device 100. In some embodiments, the image sensor 11, the first light source 12, the lens 14, and the processor 15 are formed in the same packaging structure. In other embodiments, the second light source 13 is also formed in the same packaging structure. The processor 15 is electrically connected to the image sensor 11, the first light source 12, and the second light source 13.

[0027] The image sensor 11 includes, for example, a CCD image sensor, a CMOS image sensor or other optical sensor, and is used to generate an image frame IF at a preset or sampling rate according to the incident light received by its pixel array. The image sensor 11 is preferably suitable for sensing invisible light (such as infrared light), or can sense full spectrum light but has an optical filter to filter out visible light.

[0028] The first light source 12 is used to shine light toward the working surface S through the opening 100H to generate reflected light Lr, and the reflected light Lr passes through the opening 100H and is incident on the image sensor 11 without passing through the lens 14 or any other lens. In other words, the image sensor 11 can be arranged on the main reflection light path of the first light source 12 (i.e., receiving the main reflection light beam, whose reflection angle is equal to the incident angle of the main incident light beam), so as to facilitate the detection of the reflected light Lr generated by the first light source 12 on a strong reflective surface (such as a light-colored floor tile surface, a glass surface, etc.). The first light source 12 is, for example, a laser diode, which is used to emit invisible light.

[0029] The second light source 13 is used to shine light toward the working surface S through the opening 100H to generate scattered light Ls. After passing through the opening 100H, the scattered light Ls is incident on the image sensor 11 through the lens 14. The lens 14 is preferably a convex lens, which is used to converge the scattered light Ls to the sensing array of the image sensor 11. In other words, the image sensor 11 is not arranged on the main reflection light path of the second light source 13 (does not receive the main reflection light beam relative to the main incident light beam of the second light source 13), and is used to detect the scattered light Ls generated by the second light source 13 on a weak reflective surface (such as a dark floor tile surface, a carpet surface, etc.). The second light source 13 is, for example, a light emitting diode (LED) or a laser diode (laser diode), which is used to emit invisible light. The laser diode has an emission angle of 18 to 30 degrees; the light emitting diode has an emission angle of about 30 degrees, but is not limited thereto.

[0030] It must be noted that, although the light passing through the lens 14 is referred to as scattered light Ls in this embodiment, the scattered light Ls is actually still formed by the work surface S reflecting the light emitted by the second light source 13, but the transmission of the scattered light Ls is not located on the main reflection light path of the second light source 13 (i.e., the path of the main reflection light beam whose reflection angle is equal to the incident angle of the main incident light), so as to facilitate separation from the reflected light Lr. The reflected light Lr refers to the light reflected by the work surface S and transmitted to the main reflection light path.

[0031] The processor 15 is, for example, a digital signal processor (DSP), a microprocessor (MCU), an application specific integrated circuit (ASIC), a central processing unit (CPU) or other processing devices for processing the image frame IF, and can implement its functions using software, hardware, firmware or a combination thereof.

[0032] The processor 15 is used to control the first light source 12 or the second light source 13 to light up. For example, when the trajectory tracking device 100 moves on a strongly reflective surface, it is preferred to control the first light source 12 to light up and control the second light source 13 to turn off, so as to calculate the displacement amount according to the image frame IF acquired by the image sensor 11 when the first light source 12 is turned on. When the trajectory tracking device 100 moves on a weakly reflective surface, it is preferred to control the first light source 12 to turn off and control the second light source 13 to light up, so as to calculate the displacement amount according to the image frame IF acquired by the image sensor 11 when the second light source 13 is turned on. The method for calculating the displacement amount can use known methods, such as comparing two image frames, calculating the correlation between image frames, etc., and there is no specific limitation.

[0033] For example, the processor 15 calculates the image features of the image frame IF, and controls the first light source 12 or the second light source 13 to light up according to the image features. In this embodiment, the image features are, for example, the count of grayscale value differences of adjacent pixels in the image frame IF exceeding a preset value, image contrast, image clarity, peak value or boundary number, and other parameters that can represent image quality, but are not limited thereto. The processor 15 turns on the first light source 12 or the second light source 13 according to the image frame IF having better image features.

[0034] For example, after the trajectory tracking device 100 is turned on or ends sleep, the processor 15 is preset to directly light up the first light source 12 (or the second light source 13) for operation. When the image feature of the image frame IF obtained by the processor 15 is lower than the preset value or the image feature changes by more than the preset change threshold, the switching mode is entered. In the switching mode, the processor 15 sequentially controls the first light source 12 to light up to obtain the first image frame and the second light source 13 to light up to obtain the second image frame. Then, the processor 15 compares the first image frame and the second image frame to confirm the one with better image features. When the image feature of the first image frame is better, the processor 15 controls the first light source 12 to light up with the image acquisition of the image sensor 11 and returns to the normal mode for continuous operation. When the image feature of the second image frame is better, the processor 15 controls the second light source 13 to light up with the image acquisition of the image sensor 11 and returns to the normal mode for continuous operation. Then, when the processor 15 detects that the image feature of the image frame IF is lower than the preset value or the image feature changes by more than the preset change threshold in the normal mode again, the switching mode is entered again. When the surface condition changes, the track tracking device 100 enters the switching mode from the normal mode to select the preferred operating state. In this way, the track tracking device 100 can be applied to different working surfaces.

[0035] In this embodiment, the normal mode refers to a mode in which only one of the two light sources is lit and the displacement is calculated. The switching mode refers to a mode in which the light source to be used is determined and the displacement is not calculated.

[0036] In some embodiments, the trajectory tracking device 100 may also include a third light source 16, which is preferably a point light source formed by a laser diode for height determination. The point light source refers to a bright spot formed on the working surface S. Preferably, when the third light source 16 is turned on, the first light source 12 and the second light source 13 are off. The processor 15 calculates the distance D relative to the working surface S based on the image frame IF acquired by the image sensor 11 when the third light source 16 is turned on. For example, the processor 15 calculates the distance D of the working surface S using a trigonometric function algorithm based on the imaging position of the third light source 16 in the image frame IF, calculates the distance D of the working surface S based on a time-of-flight (TOF) algorithm, or calculates the distance D based on other known distance measurement methods, without specific limitations.

[0037] The processor 15 also adjusts the magnification of the obtained displacement according to the distance D, so as to output approximately the same counts per inch when the trajectory tracking device 100 moves at a fixed speed, for example, assuming Δs×R. For example, the trajectory tracking device 100 may also include a memory, such as a non-volatile memory, for storing multiple magnifications R corresponding to different distances D. When the distance D obtained by the processor 15 is farther, the displacement Δs is multiplied by a higher magnification R; and when the distance D obtained by the processor 15 is closer, the displacement Δs is multiplied by a lower magnification R, so that the processor 15 can output approximately the same counts per inch when the trajectory tracking device 100 moves at a fixed speed. Thereby, even if the processor 15 uses different light sources to calculate the displacement, the user can still feel the same speed to have a better user experience.

[0038] It must be noted that the height calculation function corresponding to the third light source 16 and the processor 15 can be selectively implemented according to different applications.

[0039] Please refer to Figure 2 FIG. 2 is a schematic diagram of a trajectory tracking device 200 according to another embodiment of the present invention. Figure 1 The embodiments are substantially the same, and are all used to detect the displacement relative to the working surface S. The difference is that the housing 27 (which is similar to Figure 1 The operation of each component within the housing 17).

[0040] The trajectory tracking device 200 includes an image sensor 21, a partial reflector 28, a first light source 22, a second light source 23, a lens 24, and a processor 25. The processor 25 is electrically connected to the image sensor 21, the first light source 22, and the second light source 23.

[0041] The image sensor 21 includes, for example, a CCD image sensor, a CMOS image sensor or other optical sensor, for generating an image frame IF at a preset or sampling rate according to incident light received by its pixel array. Similarly, the image sensor 21 is preferably suitable for sensing invisible light, such as infrared light.

[0042] The first light source 22 is used to shine light toward the working surface S through the opening 200H to generate scattered light. After passing through the opening 200H, the scattered light is incident on the image sensor 21 without passing through the partial reflection plate 28 and the lens 24 or any lens. The first light source 22 can be selected as a light emitting diode or a laser diode, which has a light emission angle. The image sensor 21 is not arranged on the main reflection light path of the first light source 22 (i.e., it does not receive the main reflection light beam), and is used to detect the scattered light generated by the first light source 22 on the weak reflective surface. In this embodiment, the definitions of the main reflection light path and the scattered light are as described above, so they are not repeated here.

[0043] The second light source 23 is used to shine light toward the partial reflection plate 28 to generate partial reflected light perpendicular to the working surface S, which illuminates the working surface S through the opening 200H. The partial reflected light is incident on the image sensor 21 after being reflected by the working surface S. In this embodiment, the partial reflection plate 28 is, for example, a plastic plate or a glass plate, which is used to reflect part of the light emitted by the second light source 23. There is no specific restriction on the reflectivity and transmittance of the partial reflection plate 28, as long as it can partially reflect the light emitted by the second light source 23 perpendicularly to the working surface S. After being reflected by the working surface S, the partial reflected light still vertically upwardly irradiates the partial reflection plate 28. After a part of the partial reflected light passes through the partial reflection plate 28, it passes through the lens 24 (for example, a convex lens) located between the partial reflection plate 28 and the image sensor 21 to reach the pixel array of the image sensor 21. Since the second light source 23 is used to generate reflected light perpendicularly to the working surface S and the image sensor 21, it is suitable for strongly reflective surfaces.

[0044] By providing the partial reflector 28, the light emitted by the second light source 23 passes through the partial reflector 28 twice and its intensity is greatly reduced, so sufficient reflected light is generated only on the strong reflective surface. At the same time, since the image sensor 21 is not located on the main reflected light path of the first light source 22, the scattered light incident on the image sensor 21 on the strong reflective surface is very weak. The processor 25 (similar to Figure 1 The processor 15) can calculate the displacement amount based on the reflected light of the second light source 23 in the image frame IF on the strong reflective surface.

[0045] When the light-reflecting surface is weak, the second light source 23 does not generate enough reflected light, and the processor 25 calculates the displacement according to the scattered light of the first light source 22 in the image frame IF. Due to the above characteristics, the first light source 22 and the second light source 23 can be turned on at the same time in this embodiment, but it is not limited to this. The processor 25 can also compare the image features of the image frames obtained when different light sources are turned on to light only one of the two light sources in the normal mode. The comparison method is the same as the previous embodiment, so it will not be repeated here.

[0046] In addition, in this embodiment, in order to avoid forming a fixed light spot on the partial reflector 28 and generating fixed imaging noise in the image frame IF, the second light source 23 can be selected as a light emitting diode instead of a laser diode. In other embodiments, if the processor 25 can eliminate the fixed imaging noise in the image frame IF, the second light source 23 can also be selected as a laser diode.

[0047] Please refer to Figure 3 FIG. 1 is a schematic diagram of a trajectory tracking device 300 according to another embodiment of the present invention. Figure 1 The embodiment is the same as that of the present invention, both of which are used to detect the displacement relative to the working surface S. The difference is that in order to reduce the size of the opening 300H on the bottom surface of the shell 37, the present embodiment adopts a configuration of a single light source and two image sensors.

[0048] The trajectory tracking device 300 includes a first image sensor 311 , a second image sensor 312 , a first light source 32 , a lens 34 , and a processor 35 . The processor 35 is electrically connected to the first image sensor 311 , the second image sensor 312 , and the first light source 32 .

[0049] The first image sensor 311 and the second image sensor 312 include, for example, a CCD image sensor, a CMOS image sensor or other optical sensors, and are used to output the first image frame IF1 and the second image frame IF2 at a preset or sampling rate according to the incident light received by the pixel array. Similarly, the first image sensor 311 and the second image sensor 312 are preferably suitable for sensing invisible light. The first image sensor 311 and the second image sensor 312 are preferably two different image sensors and have their own pixel arrays; wherein the two pixel arrays may have the same or different sizes and resolutions.

[0050] The first light source 32 is used to shine light toward the working surface S through the opening 300H to generate reflected light Lr and scattered light Ls. The reflected light Lr passes through the opening 300H and is incident on the first image sensor 311 without passing through the lens 34 or any other lens. The scattered light Ls passes through the opening 300H and is incident on the second image sensor 312 through the lens 34.

[0051] Since the first image sensor 311 is used to detect the reflected light Lr, the first image sensor 311 is disposed on the main reflected light path of the first light source 32 (receiving the main reflected light beam, which is opposite to the main incident light beam of the first light source 32). When the trajectory tracking device 30 travels on a strong reflective surface, the first image sensor 311 can sense better image features.

[0052] Since the second image sensor 312 is used to detect the scattered light Ls, the second image sensor 312 is not disposed on the main reflected light path of the first light source 32 (does not receive the main reflected light beam). When the trajectory tracking device 300 travels on a weakly reflective surface, the first image sensor 311 senses poor image features, while the second image sensor 312 senses good image features. Therefore, the processor 35 (similar to Figure 1 The processor 15 controls the opening and closing of the first image sensor 311 or the second image sensor 312 according to the working surface S on which the trajectory tracking device 300 operates.

[0053] In this embodiment, the definitions of reflected light, scattered light and the main reflected light path have been described above and will not be repeated here.

[0054] For example, the processor 35 is used to calculate the image features of the first image frame IF1 and the second image frame IF2, and control the first image sensor 311 or the second image sensor 312 to turn off according to the image features; wherein the image features have been described above and will not be repeated here.

[0055] For example, when the trajectory tracking device 300 is turned on or ends sleep, the processor 35 is preset to directly turn on the first image sensor 311 (or the second image sensor 312) for operation. When the image feature of the first image frame IF1 (or the second image frame IF2) obtained by the processor 35 is lower than the preset value or the image feature change exceeds the preset change threshold, the switching mode is entered. In the switching mode, the processor 35 sequentially or simultaneously controls the first image sensor 311 to obtain the first image frame IF1 and controls the second image sensor 312 to obtain the second image frame IF2. Then, the processor 35 compares the first image frame IF1 and the second image frame IF2 to confirm the one with better image features. When the image feature of the first image frame IF1 is better, the processor 35 controls the first image sensor 311 to continue to operate and return to normal mode. When the image feature of the second image frame IF2 is better, the processor 35 controls the second image sensor 312 to continue to operate and return to normal mode. When the processor 35 detects in the normal mode that the image feature of the first image frame IF1 or the second image frame IF2 (depending on the image sensor in operation) is lower than the preset value or the image feature change exceeds the preset change threshold, it enters the switching mode again for judgment. As mentioned above, the trajectory tracking device 300 of the present invention can be applied to different working surfaces.

[0056] In this embodiment, the normal mode refers to a mode in which only one of the two image sensors operates and calculates the displacement, and the switching mode refers to a mode in which the displacement is not calculated.

[0057] In some implementations, the first image sensor 311 and the second image sensor 312 operate simultaneously, and the processor 35 selects the first image frame IF1 or the second image frame IF2 having a better image feature for tracking.

[0058] In some embodiments, the trajectory tracking device 300 may also include a point light source 36, which is preferably a point light source formed by a laser diode for height determination. The definition of the point light source is as described above. Preferably, when the point light source 36 is turned on, the first light source 32 is turned off, the second image sensor 312 is turned on, and the first image sensor 311 is turned off. The second image sensor 312 receives the light emitted by the point light source 36 and reflected by the working surface S through the lens 34. The processor 35 calculates the distance D relative to the working surface S based on the second image frame IF2 acquired by the second image sensor 312 when the point light source 36 is turned on. The way in which the processor 35 calculates the distance D is as described above, so it will not be repeated here.

[0059] The processor 35 also adjusts the magnification of the obtained displacement according to the distance D so as to output approximately the same counts per inch when the trajectory tracking device 300 moves at a fixed speed. For example, the trajectory tracking device 300 may also include a memory, such as a non-volatile memory, for storing magnifications corresponding to different distances D. When the distance D obtained by the processor 35 is farther, the displacement is multiplied by a higher magnification; and when the distance D obtained by the processor 35 is closer, the displacement is multiplied by a lower magnification, so that the processor 35 can output approximately the same counts per inch when the trajectory tracking device 300 moves at a fixed speed. Thereby, even if the processor 35 uses different image frames (such as the first image frame IF1 or the second image frame IF2) to calculate the displacement, the user can still feel the same speed to enhance the user experience.

[0060] It should be noted that the values ​​(such as the light emitting angle) in the above embodiments are only for illustration and are not intended to limit the present invention. In the present invention, the processor controls the light source to light up relative to the image acquisition of the image sensor. Figure 1-3 The spatial relationship and proportion of the components are for illustration only and are not intended to limit the present invention. In some embodiments, the housing may be provided with buttons or a touch panel for user operation, and may also have a light to indicate the operating status of the track tracking device. In some embodiments, the housing is provided with wheels for movement on a work surface.

[0061] In the present invention, the light source or image sensor is not turned on means that the light source or image sensor that is not turned on is always turned on before entering the switching mode next time. By calculating the displacement of continuous time, the trajectory of the trajectory tracking device can be tracked.

[0062] In summary, conventional optical track tracking devices cannot correctly calculate displacement on certain specific working surfaces and have low applicability. Figures 1 to 3 ), which calculates the image features of the image frames obtained under two different conditions to determine the appropriate image frame to calculate the displacement. The trajectory tracking device of the present invention can switch working states relative to different working surfaces to be applicable to various working surfaces.

[0063] Although the present invention has been disclosed through the above examples, they are not intended to limit the present invention. Any person skilled in the art with common knowledge in the technical field to which the present invention belongs can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the appended claims.

Claims

1. A sweeping robot for detecting displacement relative to a carpet surface or a tile surface, the sweeping robot comprising: A first image sensor, the first image sensor is used to output a first image frame; a second image sensor, the second image sensor being configured to output a second image frame; lens; a first light source, the first light source being used to shine light toward the carpet surface or the tile surface to generate reflected light and scattered light, wherein the scattered light passes through the lens and is incident on the second image sensor, and the reflected light does not pass through the lens or any lens and is incident on the first image sensor, so as to reduce the size of the opening on the bottom surface of the housing of the cleaning robot; as well as processor, which is used to calculating a first image feature of the first image frame and a second image feature of the second image frame, When the first image feature is superior to the second image feature, it indicates that the cleaning robot is moving on the tile surface, and the displacement is calculated according to the first image frame, and When the second image feature is superior to the first image feature, it indicates that the cleaning robot is moving on the carpet surface, and the displacement is calculated according to the second image frame.

2. The sweeping robot according to claim 1, wherein The first image sensor is configured to receive a main reflected light beam from the first light source, and The second image sensor is configured not to receive the main reflected light beam of the first light source.

3. The cleaning robot according to claim 1, further comprising a processor, wherein the processor is used to control the first image sensor to be turned off when the displacement amount is calculated according to the second image frame, or to control the second image sensor to be turned off when the displacement amount is calculated according to the first image frame. 4 . The cleaning robot according to claim 1 , further comprising a point light source, wherein when the point light source is turned on, the first light source is turned off and the second image sensor is turned on.

5. The sweeping robot according to claim 4, further comprising a processor, wherein the processor is used to calculate the distance relative to the carpet surface or the tile surface according to the second image frame acquired by the second image sensor when the point light source is turned on. 6 . The cleaning robot according to claim 5 , wherein the processor further adjusts a magnification of the displacement amount according to the distance to output the same counts per inch when the cleaning robot moves at a fixed speed.

7. The cleaning robot according to claim 6 further comprises a memory for storing the magnifications corresponding to different distances. 8 . The cleaning robot according to claim 4 , wherein the second image sensor receives, through the lens, light reflected by the carpet surface or the tile surface when the point light source is turned on.

9. The cleaning robot according to claim 1, the cleaning robot can operate in a normal mode or a switching mode, wherein: The normal mode refers to a mode in which one of the first image sensor and the second image sensor operates and calculates the displacement, and The switching mode refers to a mode in which the image features of the first image frame and the second image frame are compared to determine which image feature is better but the displacement is not calculated.

Images