Apparatus and method for camera control of vehicle

Abstract

A vehicle camera control device according to one embodiment of the present invention may include a camera module comprising a plurality of image acquisition means and a control unit that controls the use of the plurality of image acquisition means selectively based on the driving situation of the vehicle and the operating state of at least one of the plurality of image acquisition means.

Description

Technology Field

[0001] The present invention relates to a vehicle camera control device and method. Background Technology

[0002] Built-in camera sensors installed in the vehicle play a fundamental and core role in ADAS (Advanced Driver Assistance System) and autonomous driving functions.

[0003] Conventional ADAS devices concentrate the role of ADAS functions on a single camera (a stereo camera for forward recognition), so there are cases where ADAS functions do not operate normally if the camera's field of view is obstructed.

[0004] In addition, when using a single camera, visibility may be obstructed due to foreign matter on the lens or the influence of weather conditions such as fog; furthermore, in areas with a high risk of unexpected situations, such as child protection zones, a camera with a wider field of view than a standard camera is required. The problem to be solved

[0005] One objective of the present invention is to provide a vehicle camera control device and method that enables the stable performance of Advanced Driver Assistance System (ADAS) functions and autonomous driving functions, which rely heavily on camera sensors, by utilizing an auxiliary camera as a substitute to acquire a vehicle front image in situations where a general camera is not operating normally.

[0006] The technical problems of the present invention are not limited to those mentioned above, and other unmentioned technical problems will be clearly understood by those skilled in the art from the description below. means of solving the problem

[0007] A vehicle camera control device according to one embodiment of the present invention may include a camera module comprising a plurality of image acquisition means and a control unit that controls the use of the plurality of image acquisition means selectively based on the driving situation of the vehicle and the operating state of at least one of the plurality of image acquisition means.

[0008] In one embodiment, the image acquisition means may include a first camera and a second camera that acquire an image with a predetermined angle of view, a third camera that acquires an image with an angle of view wider than the predetermined angle of view, and a fourth camera that acquires an image via infrared.

[0009] In one embodiment, the control unit may control the use of the second camera when normal image acquisition is impossible through the first camera.

[0010] In one embodiment, the control unit may control the use of the third camera when the vehicle decelerates to a speed below a predetermined speed while acquiring an image through the first camera.

[0011] In one embodiment, the control unit may control the use of the third camera when the vehicle rotates by more than a predetermined angle while acquiring an image through the first camera.

[0012] In one embodiment, the control unit can determine whether to recognize an object within a predetermined distance if normal image acquisition is impossible within a predetermined distance for more than a predetermined time through the first camera.

[0013] In one embodiment, the control unit may control the use of the fourth camera when an object is recognized within the predetermined distance.

[0014] In one embodiment, the control unit may control the use of the second camera when an object is not recognized within the predetermined distance.

[0015] A vehicle camera control method according to another embodiment of the present invention may include the steps of: operating a camera module comprising a plurality of image acquisition means; determining a driving situation of a vehicle and the operating state of at least one of the plurality of image acquisition means; and controlling the plurality of image acquisition means to be used selectively based on the operating state of any one of the image acquisition means.

[0016] In one embodiment, the step of determining the driving condition of the vehicle and the operating state of at least one of the plurality of image acquisition means may include the step of determining the operating state of a first camera that acquires an image at a predetermined angle of view.

[0017] In one embodiment, the step of controlling the multiple image acquisition means to be selectively used based on the operating state of any one of the image acquisition means may include the step of controlling the use of a second camera that acquires an image with a predetermined angle of view when normal image acquisition through the first camera is impossible.

[0018] In one embodiment, the step of controlling the multiple image acquisition means to be selectively used based on the operating state of any one of the image acquisition means may include the step of controlling the use of a third camera that acquires an image with a wider field of view than the predetermined field of view when the vehicle decelerates to a speed below a predetermined speed while acquiring an image through the first camera.

[0019] In one embodiment, the step of controlling the multiple image acquisition means to be selectively used based on the operating state of any one of the image acquisition means may include the step of controlling the third camera to be used when the vehicle rotates by more than a predetermined angle while acquiring an image through the first camera.

[0020] In one embodiment, the step of controlling the multiple image acquisition means to be used selectively based on the operating state of any one of the image acquisition means may include the step of determining whether an object is recognized within a predetermined distance when normal image acquisition is impossible within a predetermined distance for more than a predetermined time through the first camera.

[0021] In one embodiment, the step of controlling the plurality of image acquisition means to be selectively used based on the operating state of any one of the image acquisition means may include the step of controlling the use of a fourth camera that acquires an image via infrared when an object is recognized within the predetermined distance.

[0022] In one embodiment, the step of controlling the multiple image acquisition means to be used selectively based on the operating state of any one of the image acquisition means may include the step of controlling the second camera to be used when an object is not recognized within the predetermined distance. Effects of the invention

[0023] This technology has the effect of enabling the stable performance of ADAS (Advanced Driver Assistance System) and autonomous driving functions, which rely heavily on camera sensors, by utilizing an auxiliary camera as a substitute to acquire front-facing images of the vehicle in situations where the standard camera is not functioning properly.

[0024] In addition, various effects identified directly or indirectly through this document may be provided. Brief explanation of the drawing

[0025] FIG. 1 is a block diagram showing a vehicle camera control device according to one embodiment of the present invention, and FIG. 2 is an exemplary diagram showing a camera module constituting a vehicle camera control device according to one embodiment of the present invention, and FIG. 3 is a drawing showing an example of a captured image through a camera module constituting a vehicle camera control device according to an embodiment of the present invention, and FIG. 4 is a flowchart illustrating a vehicle camera control method according to an embodiment of the present invention. Specific details for implementing the invention

[0026] Hereinafter, various embodiments of the present invention are described with reference to the accompanying drawings. However, this is not intended to limit the present invention to specific embodiments and should be understood to include various modifications, equivalents, and / or alternatives of the embodiments of the present invention.

[0027] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments.

[0028] In relation to the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of the noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise.

[0029] In this document, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "at least one of A, B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof.

[0030] Terms such as “first,” “second,” or “first” or “second” may be used simply to distinguish a component from another component and do not limit the components in any other aspect (e.g., importance or order). Where any (e.g., first) component is referred to as “coupled” or “connected” to another (e.g., second) component, with or without the terms “functionally” or “communicationally,” it means that said component may be connected to said other component directly (e.g., wired), wirelessly, or through a third component.

[0031] Various embodiments of this document may be implemented as software (e.g., a program) comprising one or more instructions stored in a storage medium (e.g., internal memory or external memory) readable by a machine. For example, the machine may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter.

[0032] A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0033] According to one embodiment, the method according to the various embodiments disclosed herein may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store or directly between two user devices. In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0034] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separately disposed of in other components.

[0035] According to various embodiments, one or more of the aforementioned components or operations may be omitted, or one or more other components or operations may be added.

[0036] Generally or additionally, multiple components (e.g., modules or programs) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to the integration.

[0037] According to various embodiments, operations performed by a module, program or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0038] FIG. 1 is a block diagram showing a vehicle camera control device according to one embodiment of the present invention, and FIG. 2 is an exemplary diagram showing a camera module constituting a vehicle camera control device according to one embodiment of the present invention.

[0039] Referring to FIG. 1, a vehicle camera control device according to one embodiment of the present invention may be configured to include an image processing unit (110), a sensor unit (120), a camera module (130), and a control unit (140).

[0040] The image processing unit (110) can collect an image received from the camera module (130) and convert the collected image so that it can be digitally processed.

[0041] The sensor unit (120) senses signals related to the driving of the vehicle, and may include a speed sensor (121) that provides vehicle speed information, a GPS (122) that provides vehicle location information, a steering angle sensor (123) that provides vehicle rotation information due to steering wheel rotation, and a radar sensor (124) that provides object detection information.

[0042] Referring to FIG. 2, the camera module (130) may include a primary camera (131), a first auxiliary camera (132), a second auxiliary camera (133), and a third auxiliary camera (134), and may be installed at the upper front of the vehicle interior to capture images of the front of the vehicle, such as lanes, vehicles ahead, or pedestrians.

[0043] For example, the camera module (130) can be configured to be included in a vehicle black box.

[0044] Images captured by the main camera (131), the first auxiliary camera (132), the second auxiliary camera (133), and the third auxiliary camera (134) can be transmitted to the image processing unit (110) and converted so that digital processing is possible.

[0045] The primary camera (131) and the first auxiliary camera (132) can each be configured as general optical cameras having the same field of view, for example, a CCD (charge coupled device) camera or a CMOS (complementary metal oxide semiconductor) camera can be used.

[0046] The main camera (131) and the first auxiliary camera (132) can each have a field of view of 45 degrees.

[0047] When the primary camera (131) and the first auxiliary camera (132) operate simultaneously, they can be used as a stereo camera, and distance information about a forward object can be obtained based on the stereo image captured by the stereo camera.

[0048] For reference, optical cameras have the advantage of being able to detect colors and accurately classify objects, but they can be significantly affected by specific environments such as night, snow, rain, and fog.

[0049] The second auxiliary camera (133) is a camera having a wider field of view than the main camera (131) or the first auxiliary camera (132), for example, it may be a camera using a wide-angle lens with a field of view of 120 to 160 degrees.

[0050] The second auxiliary camera (133) can capture the front of the vehicle more widely and in greater detail than the main camera (131) and the first auxiliary camera (132).

[0051] The third auxiliary camera (134) can be an infrared camera or a thermal imaging camera capable of acquiring a front image of the vehicle using infrared light, and can increase the reliability of the image recording by capturing parts that could not be captured by the primary camera (131), the first auxiliary camera (132), or the second auxiliary camera (133) through a night shooting function.

[0052] In addition, the third auxiliary camera (134) can detect objects from a long distance and distinguish between living things and objects without being affected by specific environments such as snow, rain, or fog.

[0053] The control unit (140) can control the overall operation of the vehicle camera control device and may include an Electronic Control Unit (ECU). It determines whether to use the first auxiliary camera (132), the second auxiliary camera (133), or the third auxiliary camera (134) based on the operating state of the main camera (131), and accordingly, can control the operation of the first auxiliary camera (132), the second auxiliary camera (133), or the third auxiliary camera (134).

[0054] In addition, the control unit (140) can receive vehicle speed information, vehicle position information, vehicle rotation information, and front object detection information from the sensor unit (120).

[0055] FIG. 3 is a drawing showing an example of a captured image through a camera module constituting a vehicle camera control device according to one embodiment of the present invention.

[0056] The control unit (140) normally receives an image captured through the basic camera (131) from the image processing unit (110) and can perform object detection such as lane detection, vehicle detection, or pedestrian detection based on the received image (see FIG. 3 a, b).

[0057] If the control unit (140) does not receive the image captured through the primary camera (131) normally, it controls the operation of the first auxiliary camera (132) and then receives the image captured through the first auxiliary camera (132) in place of the primary camera (131).

[0058] For example, when it is recognized that a lane exists at the current location through the signal of the GPS (122), if foreign matter is attached to the lens of the basic camera (131), the lane may not be detected in the image captured through the basic camera (131).

[0059] In this case, the control unit (140) determines that normal video recording is impossible through the primary camera (131), operates the first auxiliary camera (132), and can receive the video recorded through the first auxiliary camera (132).

[0060] When the control unit (140) receives a video captured through the primary camera (131) and determines that a sudden situation has occurred, such as receiving a signal that the vehicle speed has been decelerated to a speed below a predetermined speed through the speed sensor (121) or receiving a signal that the vehicle has been rotated to a speed above a predetermined angle through the steering angle sensor (124), it can operate the second auxiliary camera (133) and receive a video captured through the second auxiliary camera (133).

[0061] For example, when the vehicle speed decreases as it enters a child protection zone with a speed limit of 30 km / h or less, or when the vehicle speed decreases due to many people in an urban area, the control unit (140) determines that a sudden situation has occurred and activates the second auxiliary camera (133), and can receive the video captured through the second auxiliary camera (133).

[0062] Even when the vehicle enters a curved road with severe curvature and rotates beyond a predetermined angle, the second auxiliary camera (133) can be operated and the image captured through the second auxiliary camera (133) can be received.

[0063] Therefore, when driving a vehicle in a child protection zone, urban area, or on a road with severe curvature, the area in front of the vehicle can be captured widely and in detail using a wide-angle lens, thereby increasing the ability to respond to sudden situations (see Fig. 3c).

[0064] The control unit (140) can determine whether to recognize an object within a predetermined distance through the radar sensor (124) if, at some point while receiving a front image within a predetermined distance through the basic camera (131), the image is not received normally and is not received normally until a predetermined time is exceeded.

[0065] Next, when the control unit (140) detects an object within a predetermined distance, it determines that normal video recording is impossible through the basic camera (131) due to a specific environmental situation, and activates the third auxiliary camera (134), and can receive the video recorded through the third auxiliary camera (132).

[0066] For example, when receiving a video of a lane 100m ahead through the basic camera (131) and entering a foggy area, a video of a lane shorter than 100m can be received due to the fog, and this time may exceed 5 minutes.

[0067] At this time, if an object is detected to be present within 100m through the radar sensor (124), the control unit (140) determines that normal video recording is impossible through the basic camera (131) and activates the third auxiliary camera (134), and can receive the video recorded through the third auxiliary camera (133).

[0068] Accordingly, the control unit (140) can receive images capable of long-distance detection and distinguishing between living things and objects without being affected by specific environments such as snow, rain, or fog through a third auxiliary camera (134) that uses infrared light (see FIG. 3d).

[0069] In addition, the control unit (140) can effectively respond to unexpected situations during night driving by receiving images captured through the third auxiliary camera (133) even when driving at night, when the lane recognition distance may be reduced compared to driving during the day.

[0070] Meanwhile, while receiving a video of a lane 100m ahead through the basic camera (131), the vehicle enters a foggy area and receives a video of a lane shorter than 100m after more than 5 minutes, but the radar sensor (124) can detect that no object exists within 100m.

[0071] In this case, the control unit (140) determines that normal image capture is impossible through the basic camera (131) because foreign matter is attached to the basic camera (131), and operates the first auxiliary camera (132), and can receive the image captured through the first auxiliary camera (132).

[0072] In addition, while receiving video of a lane 100m ahead through the basic camera (131), even if you enter a foggy area and receive video of a lane shorter than 100m away, there may be cases where this time is shorter than 5 minutes.

[0073] In this case, it is determined that the video of the lane was not received temporarily, such as when driving on a road without a lane, and the video captured through the basic camera (131) can be continuously received.

[0074] Although not shown in the drawings, according to the embodiments, the vehicle camera control device may further include a storage unit.

[0075] The storage unit may store commands for controlling a vehicle camera control device, control command codes, control data, or user data. For example, the storage unit may include at least one of an application program, an operating system (OS), middleware, or a device driver.

[0076] The storage unit may include one or more of volatile memory or non-volatile memory.

[0077] Volatile memory may include DRAM (dynamic random access memory), SRAM (static RAM), SDRAM (synchronous DRAM), PRAM (phase-change RAM), MRAM (magnetic RAM), RRAM (resistive RAM), FeRAM (ferroelectric RAM), etc.

[0078] Non-volatile memory may include ROM (read only memory), PROM (programmable ROM), EPROM (electrically programmable ROM), EEPROM (electrically erasable programmable ROM), flash memory, etc.

[0079] The storage unit may further include non-volatile media such as hard disk drives (HDD), solid state disks (SSD), embedded multi-media cards (eMMC), and universal flash storage (UFS).

[0080] Hereinafter, a vehicle camera control method according to another embodiment of the present invention will be described in detail with reference to FIG. 4.

[0081] FIG. 4 is a flowchart illustrating a vehicle camera control method according to an embodiment of the present invention.

[0082] In the following, it is assumed that the vehicle camera control device of FIG. 1 performs the process of FIG. 4.

[0083] First, a camera module (130) equipped with a main camera (131), a first auxiliary camera (132), a second auxiliary camera (133), and a third auxiliary camera (134) is operated, and the front of the vehicle can be photographed through the main camera (131) (S101).

[0084] Next, when the vehicle enters a speed-restricted area such as a child protection zone or a crowded urban area and slows down, or when the vehicle turns more than a predetermined angle on a road with severe curvature (S102), the vehicle can be controlled to use a second auxiliary camera (133) that acquires images with a wider field of view than the field of view of the primary camera (131) (S103).

[0085] Next, when receiving a forward image within a predetermined distance through the basic camera (131), if the image is not received normally from a certain point in time (S104) and the image is not received normally until a predetermined time is exceeded (S105), it can be determined whether an object within a predetermined distance is recognized through the radar sensor (124).

[0086] Next, when an object is detected within a predetermined distance, it is determined that normal video recording is impossible through the basic camera (131) due to a specific environmental condition (S106), and a third auxiliary camera (134) using infrared is activated, and the video captured through the third auxiliary camera (132) can be received (S107).

[0087] Meanwhile, a forward image of a distance shorter than a predetermined distance is received through the basic camera (131) until a predetermined time is exceeded, but it can be detected through the radar sensor (124) that no object exists within a predetermined distance.

[0088] In this case, the control unit (140) determines that normal video recording is impossible through the basic camera (131) because foreign matter is attached to the basic camera (131), and operates the first auxiliary camera (132) which has the same field of view as the basic camera (131), and receives the video recorded through the first auxiliary camera (132) (S108).

[0089] As explained above, this technology acquires front-facing vehicle images by utilizing an auxiliary camera as a substitute in situations where a standard camera is not functioning properly, thereby enabling the stable performance of ADAS (Advanced Driver Assistance System) and autonomous driving functions that rely heavily on camera sensors.

[0090] The above description is merely an illustrative explanation of the technical concept of the present invention, and those skilled in the art to which the present invention pertains will be able to make various modifications and variations within the scope of the essential characteristics of the present invention.

[0091] Accordingly, the embodiments disclosed in this invention are intended to illustrate, not limit, the technical concept of the invention, and the scope of the technical concept of the invention is not limited by these embodiments. The scope of protection of this invention shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of this invention.

Claims

Claim 1 A vehicle camera control device comprising: a camera module including a plurality of image acquisition means; and a control unit that controls the selective use of the plurality of image acquisition means based on the driving conditions of the vehicle and the operating state of at least one of the plurality of image acquisition means, wherein the image acquisition means includes a first camera that acquires an image with a predetermined angle of view, and the control unit determines whether to recognize an object within a predetermined distance when normal image acquisition is impossible within a predetermined distance for more than a predetermined time through the first camera. Claim 2 A vehicle camera control device according to claim 1, wherein the image acquisition means comprises a second camera that acquires an image with a predetermined angle of view, a third camera that acquires an image with an angle of view wider than the predetermined angle of view, and a fourth camera that acquires an image via infrared rays. Claim 3 A vehicle camera control device according to claim 2, wherein the control unit controls the use of the second camera when normal image acquisition through the first camera is impossible. Claim 4 A vehicle camera control device according to claim 2, wherein the control unit controls the use of the third camera when the vehicle decelerates to a speed below a predetermined speed while acquiring an image through the first camera. Claim 5 A vehicle camera control device according to claim 2, wherein the control unit controls the use of the third camera when the vehicle rotates by more than a predetermined angle while acquiring an image through the first camera. Claim 6 delete Claim 7 A vehicle camera control device according to claim 2, wherein the control unit controls the use of the fourth camera when an object is recognized within the predetermined distance. Claim 8 A vehicle camera control device according to claim 2, wherein the control unit controls the use of the second camera when an object is not recognized within the predetermined distance. Claim 9 A vehicle camera control method comprising: a step of operating a camera module including a plurality of image acquisition means; a step of determining a driving situation of a vehicle and an operating state of at least one of the plurality of image acquisition means; and a step of controlling the plurality of image acquisition means to be used selectively based on the operating state of any one of the image acquisition means, wherein the step of determining the driving situation of the vehicle and an operating state of at least one of the plurality of image acquisition means includes a step of determining the operating state of a first camera that acquires an image with a predetermined angle of view, and the step of controlling the plurality of image acquisition means to be used selectively based on the operating state of any one of the image acquisition means includes a step of determining whether an object is recognized within a predetermined distance when normal image acquisition is impossible within a predetermined time within a predetermined distance through the first camera. Claim 10 delete Claim 11 A vehicle camera control method according to claim 9, wherein the step of controlling the multiple image acquisition means to be selectively used based on the operating state of any one of the image acquisition means comprises the step of controlling the use of a second camera that acquires an image with a predetermined angle of view when normal image acquisition through the first camera is impossible. Claim 12 A vehicle camera control method according to claim 9, wherein the step of controlling the use of the plurality of image acquisition means selectively based on the operating state of any one of the image acquisition means comprises the step of controlling the use of a third camera that acquires an image with a wider field of view than the predetermined field of view when the vehicle decelerates to a speed below a predetermined speed while acquiring an image through the first camera. Claim 13 A vehicle camera control method according to claim 12, wherein the step of controlling the use of the plurality of image acquisition means selectively based on the operating state of any one of the image acquisition means includes the step of controlling the use of the third camera when the vehicle rotates by more than a predetermined angle while acquiring an image through the first camera. Claim 14 delete Claim 15 A vehicle camera control method according to claim 9, wherein the step of controlling the plurality of image acquisition means to selectively use the plurality of image acquisition means based on the operating state of any one of the image acquisition means includes the step of controlling the use of a fourth camera that acquires an image via infrared when an object is recognized within the predetermined distance. Claim 16 A vehicle camera control method according to claim 9, wherein the step of controlling the plurality of image acquisition means to be selectively used based on the operating state of any one of the image acquisition means includes the step of controlling the use of a second camera when an object is not recognized within the predetermined distance.

Images