Method and system for detecting objects on a vehicle surface

Abstract

The present invention provides a method and system for detecting an object on a vehicle surface. A processor receives one or more images from an image capture device. A predetermined background is subtracted from a predetermined region of interest of the one or more images. The region of interest of the one or more images is tracked to detect at least one object on the vehicle surface based on a difference in pixel intensity in the region of interest. The location of the detected object in the region of interest is mapped to a location on a representative image of the vehicle surface. In addition, the representative image of the vehicle surface is displayed together with an alert to a vehicle user. In addition, an alert symbol is overlaid at the mapped location on the representative image of the vehicle surface.

Description

Technical Field

[0001] The present disclosure relates to the detection of objects inadvertently left on vehicle surfaces. Background Art

[0002] It is common for vehicle users to inadvertently leave personal items (e.g., wallets, purses, phones, laptops, etc.) on top of the vehicle. If the user forgets that the personal item is on top of the vehicle, the user may enter the vehicle and drive away. This may cause the personal item to fall and become damaged. Therefore, in such situations, it is desirable to have a system to locate the personal item and alert the user that the personal item has been left on top of the vehicle before the user drives away.

[0003] US Patent Document US6163250A discloses a system comprising a sensor for detecting an object located outside a vehicle. The sensor may be formed from known and commercially available components, such as a force or pressure gauge attached to the inside of a vehicle roof. Many types of sensors may be used, including any one or more of capacitive and / or force / pressure sensors, and are known in the art.

[0004] However, the prior art fails to detect and display the location of personal items on the roof of the vehicle. Therefore, a system is needed to display the location of personal items on the roof of the vehicle and issue an alert to the vehicle user. Summary of the invention

[0005] Embodiments of the present invention are directed to methods and systems for detecting objects on a surface of a vehicle.

[0006] A method for detecting an object on a vehicle surface includes receiving, by a processor, one or more images from at least one image capture device. A predetermined region of interest is identified from the one or more images. The region of interest of the one or more images is tracked to detect at least one object on the vehicle surface based on differences in pixel intensities in the region of interest. A location of the detected object in the region of interest is mapped to a location on a representative image of the vehicle surface. In addition, the representative image of the vehicle surface is displayed along with an alert to a vehicle user. In addition, an alert symbol is overlaid at the mapped location on the representative image of the vehicle surface.

[0007] A system for detecting an object on a vehicle surface includes an image capture device, a display device, and a processor operably coupled to the image capture device and the display device. The processor is configured to receive one or more images from the image capture device and identify a predetermined region of interest from the one or more images. The processor is also configured to track the region of interest and detect at least one object on the vehicle surface based on differences in pixel intensity. The processor is also configured to map the position of the detected object detected in the region of interest to a position on a representative image of the vehicle surface. In addition, the processor is configured to display a representative image of the vehicle surface and overlay a symbol at a position on the representative image of the vehicle surface.

[0008] Thus, the method and system of the present invention provides an alert to a vehicle user on a display device of the vehicle, including the location of an object on the roof of the vehicle.

[0009] A computer program is also provided, comprising program instructions for causing the computer program to execute the above method, the program instructions being embodied on a recording medium, a carrier signal or a read-only memory. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The invention will be more clearly understood from the following description of embodiments given by way of example only, with reference to the accompanying drawings, in which:

[0011] Figure 1A illustratively showing a side view of a vehicle according to some embodiments of the present invention;

[0012] Figure 1B exemplarily showing a top view of a vehicle according to some embodiments of the present invention;

[0013] Figure 2 is a flow chart illustrating an embodiment of an inventive method or process for detecting an object on a vehicle surface according to some embodiments of the present invention;

[0014] Figure 3 is a functional block diagram illustrating the main components of a system / apparatus for detecting an object on a vehicle surface according to some embodiments of the present invention; and

[0015] Figure 4 Vehicle displays according to some embodiments of the present invention are exemplarily shown. DETAILED DESCRIPTION

[0016] Figure 1A A side view of a vehicle according to some embodiments of the present invention is exemplarily shown, showing a vehicle 101 , an image capture device 102 arranged on the roof of the vehicle, and an object 103 left on the roof of the vehicle 101 . Figure 1A The dashed line in shows the field of view of the image capture device which also captures the roof of the vehicle.

[0017] Figure 1B A top view of a vehicle according to some embodiments of the present invention is exemplarily shown, which shows a vehicle 101 , an image capture device 102 arranged on the roof of the vehicle, and an object 103 left on the roof of the vehicle 101 .

[0018] Figure 2 201 is a flowchart illustrating an embodiment of an inventive method or process for detecting an object on a vehicle surface according to some embodiments of the present invention. The method for detecting an object on a vehicle surface includes receiving 201 one or more images from at least one image capture device. Subtracting 202 a predetermined background from a predetermined region of interest of the one or more images. It should be understood that other methods may be used to identify objects that are different from the background. Tracking 203 a region of interest of one or more images to detect at least one object on the vehicle surface based on a difference in pixel intensity in the region of interest after subtracting the background. The location of the detected object in the region of interest is mapped 204 to a location on a representative image of the vehicle surface. Additionally, a representative image of the vehicle surface is displayed 205 along with an alert to a vehicle user. Additionally, an alert symbol is overlaid at the mapped location on the representative image of the vehicle surface. For example, Figure 4 The display 401 is exemplarily shown showing a representative image 402 of the roof or vehicle 101, and the warning symbol 403 is overlaid on the representative image. In addition, the warning symbol 403 generally indicates the position or location of the detected object on the roof of the vehicle (see Figure 1B ).

[0019] The predetermined region of interest is generated based at least on the position of the image capture device relative to the vehicle surface. For example, the position of the image capture device relative to the vehicle surface determines the region of interest. In other words, the region of interest is a portion of the image that captures the vehicle surface.

[0020] As further described above, generating the predetermined background is based on at least one image of the vehicle surface. For example, an image capture device may be used to capture an image of the vehicle surface when no objects are placed on the surface. In another embodiment, the image capture device may capture multiple images at different times while the vehicle is in motion. Portions of the image that remain substantially static and do not change with the image are considered to be the background of the image. Furthermore, the outline of the static portion of the image may define a region of interest as described above.

[0021] The method also includes extracting an image of the detected object and classifying the object using a machine learning algorithm or a neural network-based model, and providing an audio alert to a user of the vehicle based on the classification of the object. For example, if an apple is left on the surface of the vehicle, an alert "Alert! There is an apple on the roof of the vehicle" may be sounded.

[0022] The method also includes sending a signal to an electronic control unit (ECU) of the vehicle to prevent the vehicle from moving or to limit the vehicle's speed. For example, if an object is detected on the surface of the vehicle, a signal is sent to the ECU to limit the vehicle's speed or prevent the vehicle from moving so that an object accidentally left on the vehicle does not fall off.

[0023] This method ignores objects attached to the vehicle when tracking the region of interest, where the object is part of the vehicle, for example, a luggage rack mounted on the roof of the vehicle.

[0024] The above method can be applied to any surface of a vehicle, such as a roof of a vehicle, a hood of a vehicle, or a trunk lid of a vehicle.

[0025] Figure 3 3 is a functional block diagram showing the main components of a system / device 300 for detecting an object on a surface of a vehicle according to some embodiments of the present invention. The system includes a processor 301, a memory 302, a transceiver 303, an image capture device 304, and a display 305. The processor 301 is operably coupled to the memory 302, the transceiver 303, the image capture device 304, and the display 305. In addition, an electronic control unit (ECU) 311 communicates with the processor 301 through the transceiver 303. In addition, a speed governor 312 of the vehicle is operably coupled to the ECU 311. In a preferred embodiment, the image capture device 304 is a fisheye camera.

[0026] The processor 301 is configured to receive one or more images from the image capture device 304. In addition, the processor 301 subtracts a predetermined background from a predetermined region of interest in the one or more images. The processor 301 is also configured to track the region of interest and detect at least one object on the vehicle surface based on pixel intensity differences within the region of interest. The processor 301 is also configured to map the position of the detected object detected in the region of interest to a position on the representative image of the vehicle surface. In addition, the processor 301 causes the display 305 to display the representative image of the vehicle surface and overlay a symbol at the position on the representative image of the vehicle surface. For example, Figure 4 The display 401 is exemplarily shown showing a representative image 402 of the roof or vehicle 101, and the warning symbol 403 is overlaid on the representative image. In addition, the warning symbol 403 generally indicates the position or location of the detected object on the roof of the vehicle (see Figure 1B ).

[0027] The processor 301 generates a predetermined region of interest based at least on the position of the image capture device 304 relative to the vehicle surface. For example, the position of the image capture device 304 relative to the vehicle surface determines the region of interest. In other words, the region of interest is a portion of the image capturing the vehicle surface.

[0028] In addition, the processor is configured to generate a predetermined background based on at least one image of the vehicle surface. For example, an image of the vehicle surface can be captured using the image capture device 304 while no objects are placed on the surface, and the image is stored in the memory 302 as a background image. In another embodiment, the processor 301 can cause the image capture device 304 to capture multiple images at different times when the vehicle is in motion. In addition, the processor can determine the image portion that remains substantially static / unchanged from the captured multiple images, wherein the determined image portion is stored in the memory 302 as the background image. In addition, the outline of the static portion of the image can define the region of interest as described above, and the configuration of the outline stored in the memory 302 is defined as the region of interest.

[0029] The processor 301 is also configured to extract an image of the detected object and classify the object using a machine learning algorithm or a neural network-based model. The system also includes an audio output device that is operably coupled to the processor 301, and the processor 301 is configured to provide an audio alert to a user of the vehicle based on the classification of the object. For example, if an apple is left on the surface of the vehicle, an alert "Alert! There is an apple on the roof of the vehicle" may be issued.

[0030] The processor 301 is also configured to send a signal to the vehicle's electronic control unit (ECU) 311 to prevent the vehicle from moving or limit the vehicle's speed. For example, if an object is detected on the surface of the vehicle, a signal is sent to the ECU to limit the vehicle's speed or to prevent the vehicle from moving using a speed governor 312 so that an object accidentally left on the vehicle does not fall off.

[0031] In addition, it will be appreciated by those of ordinary skill in the art that the various illustrative logic / functional blocks, modules, circuits, techniques / algorithms, and processing steps described in conjunction with the embodiments disclosed herein may be implemented as electronic hardware or a combination of hardware and software. In order to clearly illustrate the interchangeability of hardware and a combination of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been generally described above according to their functionality. Whether such functions are implemented as hardware or a combination of hardware and software depends on the design choices of those of ordinary skill in the art. Such skilled technicians may implement the described functions in different ways for each specific application, but such obvious design choices should not be interpreted as causing deviations from the scope of the present invention.

[0032] The processes described in the present disclosure can be implemented using various means. For example, the processes described in the present disclosure can be implemented in hardware, firmware, software, or any combination thereof. For hardware implementation, the processing unit or processor can be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, or combinations thereof.

[0033] For firmware and / or software implementation, the software codes can be stored in a memory and executed by a processor. The memory can be implemented inside the processor unit or outside the processor unit. The term "memory" as used herein refers to any type of volatile memory or non-volatile memory.

[0034] Furthermore, although elements of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.

[0035] The embodiments of the invention described with reference to the accompanying drawings include computer devices and / or processes executed in computer devices. However, the invention also extends to computer programs, in particular computer programs stored on or in a carrier suitable for putting the invention into practice. The program may be in the form of source code, object code or a code between source code and object code, such as a partially compiled form or any other form suitable for implementing the method according to the invention. The carrier may include a storage medium, such as a ROM, such as a memory stick or a hard disk. The carrier may be an electrical signal or an optical signal, which may be transmitted by an electrical or optical cable or by radio or other means.

[0036] In the specification, the term "comprises" or any variations thereof and the term "comprising" or any variations thereof are considered to be fully interchangeable and they should both be given the broadest possible interpretation and vice versa.

[0037] The invention is not limited to the embodiments described above but may be modified in structure and details.

Claims

1. A method for detecting an object on a surface of a vehicle, comprising: receiving, by a processor, one or more images from at least one image capture device; identifying a predetermined region of interest from the one or more images; subtracting a predetermined background from a predetermined region of interest in the one or more images; tracking a region of interest and detecting at least one object on a surface of the vehicle based on a difference in pixel intensity in the predetermined region of interest after subtracting a predetermined background; mapping the positions of detected objects detected in the region of interest to positions on a representative image of the vehicle surface; as well as The representative image of the vehicle surface is displayed, and the symbol is overlaid at a location on the representative image of the vehicle surface. 2 . The method of claim 1 , further comprising generating a predetermined region of interest based at least on a position of the image capture device relative to a surface of the vehicle. The method of claim 1 , further comprising generating a predetermined background based on at least one image of a vehicle surface.

4. The method according to claim 1 also includes extracting images of the detected objects and classifying the objects using a machine learning algorithm or a neural network-based model. 5 . The method of claim 4 , further comprising providing an audio alert to a user of the vehicle based on the classification of the object.

6. The method of claim 1 further comprising sending a signal to an electronic control unit of the vehicle to prevent vehicle movement or limit vehicle speed.

7. The method according to claim 1, wherein: Tracking the region of interest includes ignoring objects attached to the vehicle, wherein the objects attached to the vehicle are part of the vehicle.

8. A method according to any one of the preceding claims, wherein: The surface of the vehicle is one or more of a roof of the vehicle, a hood of the vehicle, or a trunk lid of the vehicle.

9. A system for detecting an object on a surface of a vehicle, comprising: at least one image capture device; Display devices; A processor operably coupled to at least one image capture device and a display device, wherein the processor is configured to: receiving one or more images from at least one image capture device; identifying a predetermined region of interest from the one or more images; subtracting a predetermined background from a predetermined region of interest in the one or more images; tracking a region of interest and detecting at least one object on a surface of the vehicle based on a difference in pixel intensity in the predetermined region of interest after subtracting a predetermined background; mapping the positions of detected objects detected in the region of interest to positions on a representative image of the vehicle surface; and The representative image of the vehicle surface is displayed using a display device, and a symbol is overlaid at a location on the representative image of the vehicle surface.

10. The system according to claim 9, wherein: The processor is further configured to generate a predetermined region of interest based at least on a position of the image capture device relative to a surface of the vehicle.

11. The system according to claim 9, wherein: The processor is further configured to generate a predetermined background based on at least one image of the vehicle surface when one or more objects are not positioned on the vehicle surface.

12. The system according to claim 9, wherein: The processor is also configured to extract images of detected objects and classify the objects using a machine learning algorithm or a neural network-based model.

13. The system of claim 12, further comprising an audio output device operably coupled to the processor, wherein: The processor provides an audio alert to a user of the vehicle based on the classification of the object using an audio output device.

14. The system of claim 9, further comprising an electronic control unit of the vehicle, wherein: The processor is configured to send a signal to an electronic control unit of the vehicle to prevent vehicle movement or limit vehicle speed.

15. The system according to claim 9, wherein: The image capture device is a fisheye camera.

16. A computer readable medium comprising instructions which, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 8.

Images