Electronic device for providing blind spot image through front pillar of vehicle and operating method for the same

Abstract

To provide an electronic device for providing a blind spot video through a front pillar of a vehicle and an operating method for the same.SOLUTION: In the present disclosure, an electronic device is configured to include pillar display modules respectively disposed on front pillars, to detect at least one blind spot video obscured by the front pillars of a vehicle using at least one external video of the vehicle, and to display at least one of the blind spot videos through at least one of the pillar display modules. In one embodiment, the electronic device may acquire the external video from a front camera module and side camera modules mounted on the vehicle to provide a surround view video.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to an electronic device and method of operation for providing blind spot viewing through the front pillar (also called A-pillar) of a vehicle. [Background technology]

[0002] Typically, vehicles have pillars between the doors and the roof to provide additional strength. These pillars include the front pillars between the windshield and the side windows of the front doors, the side pillars (also called B pillars) between the side windows of the front doors and the side windows of the rear doors, and the rear pillars (also called C pillars) between the side windows of the rear doors and the rear window. Some vehicles do not have at least one of the side pillars or the rear pillars.

[0003] The front pillars of a vehicle create blind spots that obstruct the driver's view, as the driver must keep an eye on the front and sides of the vehicle. These blind spots impede the driver's driving, especially when the vehicle is turning. This poses a threat to the safety of the driver and other vehicle occupants, as well as pedestrians. Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides an electronic device and method of operation for providing blind spot viewing through a vehicle's front pillars. [Means for solving the problem]

[0005] In the present disclosure, an electronic device mounted on a vehicle may include at least two camera modules configured to each capture an external image of the vehicle, pillar display modules disposed inside the vehicle on each of the front pillars of the vehicle, and a processor connected to the camera modules and the pillar display module and configured to utilize at least one of the external images to display at least one of the blind spot images obstructed by the front pillar on at least one of the pillar display modules.

[0006] In the present disclosure, an operating method of an electronic device mounted on a vehicle may include a step of detecting at least one of blind spot images through a front pillar of the vehicle using at least one external image of the vehicle, and a step of displaying at least one of the blind spot images through at least one of pillar display modules respectively arranged on the front pillars.

[0007] In the present disclosure, an electronic device mounted on a vehicle may include a front camera module configured to capture a front image of the vehicle, a side camera module configured to capture a side image of the vehicle, and a rear camera module configured to capture a rear image of the vehicle, pillar display modules disposed inside the vehicle on front pillars of the vehicle, a main display module mounted inside the vehicle separately from the pillar display module, and a processor connected to the front camera module, the side camera module, the rear camera module, the main display module, and the pillar display module and configured to display a surround view image on the main display module using the front image, the side image, and the rear image, and to display at least one of the blind spot images blocked by the front pillar on at least one of the pillar display modules using at least one of the front image and the side image.

[0008] In the present disclosure, an operating method of an electronic device mounted on a vehicle may include the steps of capturing front, side, and rear images of the vehicle, displaying a surround view image on a main display module using the front, side, and rear images in response to a user input, and activating at least one of the pillar display modules respectively disposed on the front pillars in response to the user input, and displaying at least one of the blind spot images from the front pillars on at least one of the pillar display modules using at least one of the front and side images. Effect of the Invention

[0009] According to the present disclosure, it is possible to remove blind spots blocked by the front pillars of a vehicle. That is, the electronic device can display at least one of the blind spot images blocked by the front pillars of the vehicle on at least one of the pillar display modules disposed on the front pillars. This allows the driver to focus on the front and sides of the vehicle without the blind spots blocked by the front pillars. In particular, when the vehicle is turning, the driver can easily grasp the surrounding situation of the vehicle without the blind spots. This can ensure the safety of not only the driver and other vehicle passengers but also pedestrians. According to the first embodiment, the blind spot image is generated using an external image from a camera module generally attached to the vehicle for the surround view monitoring function, so that there is no need to newly add another camera module. This can prevent an increase in the number of work steps, an increase in installation costs, an increase in vehicle weight, etc., due to the addition of another camera module. According to the second embodiment, the blind spot image is additionally generated through the pillar display module using an external image from a camera module attached to the vehicle for the blind spot monitoring function, so that the operating load of the processor in the electronic device can be reduced. This is because the processor no longer needs to combine multiple external images to create a single blind spot image. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram illustrating an electronic device according to various embodiments. [Diagram 2] 1A to 1C are exemplary diagrams illustrating characteristic functions of an electronic device according to various embodiments. [Diagram 3] Fig. 3a is an exemplary diagram for explaining the configuration of a camera module in the electronic device according to the first embodiment, and Fig. 3b is an exemplary diagram for explaining the operational characteristics of the electronic device according to the first embodiment. [Figure 4] Fig. 4a is an exemplary diagram for explaining the configuration of a camera module in an electronic device according to the second embodiment, and Fig. 4b is an exemplary diagram for explaining the operational characteristics of the electronic device according to the second embodiment. [Diagram 5] 4 is a flow chart illustrating a method of operation of an electronic device in accordance with one of the first embodiments. [Figure 6] 10 is a flowchart illustrating an operation method of the electronic device according to another embodiment of the first embodiment. [Figure 7] 10 is a flow chart illustrating a method of operation of an electronic device in accordance with one of the second embodiments. [Figure 8] 10 is a flowchart illustrating an operation method of the electronic device according to another embodiment of the second embodiment. [Figure 9] 1 is a block diagram illustrating a vehicle equipped with an electronic device according to various embodiments; [Figure 10] FIG. 10 is a block diagram showing a control device for the vehicle shown in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Various embodiments of the present invention will now be described with reference to the accompanying drawings.

[0012] Fig. 1 is a block diagram of an electronic device 100 according to various embodiments. Fig. 2 is an exemplary diagram illustrating characteristic functions of the electronic device 100 according to various embodiments.

[0013] 1 and 2, an electronic device 100 may be installed in a vehicle. The electronic device 100 may include at least one of an input module 110, a sensor module 120, a communication module 130, an interface module 140, an audio output module 150, a main display module 160, a pillar display module 163, 165, a camera module 170, a memory 180, or a processor 190. In some embodiments, at least one of the components of the electronic device 100 may be omitted and at least one other component may be added. In some embodiments, at least two of the components of the electronic device 100 may be realized as one integrated circuit.

[0014] The input module 110 may input signals for use in at least one component of the electronic device 100. For example, the input module 110 may include at least one of a key, a button, a keyboard, a keypad, a mouse, a joystick, or a microphone. In some embodiments, the input module 110 may include at least one of touch circuitry configured to sense a touch or a sensor circuitry configured to measure the strength of a force generated by a touch.

[0015] The sensor module 120 may generate an electrical signal or data value corresponding to an internal operating condition (e.g., power or temperature) or an external environmental condition of the electronic device 100. For example, the sensor module 120 may include at least one of a global positioning system (GPS) sensor, a radar sensor, a LIDAR sensor, a motion sensor, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0016] The communication module 130 may perform communication with an external device. The communication module 130 may establish a communication channel between the electronic device 100 and the external device and perform communication with the external device through the communication channel. Here, the external device may include at least one of a satellite, a base station, a server, or another electronic device. The communication module 130 may include at least one of a wired communication module or a wireless communication module. The wired communication module may be wiredly connected to the external device via a connection terminal and perform wired communication. The wireless communication module may include at least one of a short-range communication module or a long-range communication module. The short-range communication module may communicate with the external device through a short-range communication method. For example, the short-range communication method may include at least one of Bluetooth, Wi-Fi direct, Near Field Communication (NFC), or infrared data association (IrDA). The long-range communication module may communicate with the external device through a long-range communication method. Here, the remote communication module may communicate with external devices via a network, which may include at least one of a cellular network, the Internet, or a computer network such as a local area network (LAN) or a wide area network (WAN), for example.

[0017] The interface module 140 may be provided for connection with an external device. Specifically, the interface module 140 may support a specified protocol for wired or wireless connection with the external device. Here, the external device may include a component related to driving the vehicle. For example, such a component may include a lever 200 for operating turn signals, i.e., a left turn signal and a right turn signal, as shown in FIG. 2. For example, the interface module 140 may include at least one of an HDMI (registered trademark), a USB interface, an SD card interface, or an audio interface.

[0018] According to various embodiments, at least one of the input module 110, the sensor module 120, the communication module 130, or the interface module 140 may generate a user input. In one embodiment, any sensor of the input module 110 or the sensor module 120 may generate a user input based on a signal directly input from a user. For example, the input module 110 may generate a user input based on at least one of a key (e.g., hard key or soft key) input or a voice input. For example, at least one of a motion sensor, a gesture sensor, a proximity sensor, a temperature sensor, or an illuminance sensor of the sensor module 120 may generate a user input based on a change in a state of an external environment. In another embodiment, the communication module 130 may generate a user input based on a signal input from another electronic device utilized by the user. In still another embodiment, the interface module 140 may generate a user input based on a signal generated in and input from the vehicle in response to the operation of the vehicle by the user. For example, the interface module 140 may sense the operation of the lever 200 for operating a turn signal light, and generate a user input in response thereto.

[0019] The audio output module 150 may output an audio signal. For example, the audio output module 150 may include at least one of a speaker or a receiver. In one embodiment, the speaker may include at least one voice coil that provides vibrations through a diaphragm, and a magnet that forms a magnetic field. When a current flows through the voice coil, the magnetic field formed by the voice coil may interact with the magnetic field formed by the magnet to vibrate the voice coil. The diaphragm connected to the voice coil may vibrate based on the vibration of the voice coil. The speaker may output an audio signal based on the vibration of the diaphragm.

[0020] The main display module 160 may display information. The main display module 160 may output visualized information to the driver. In this case, the main display module 160 may be arranged around the driver's seat, for example, on the dashboard of the vehicle, so that the driver of the vehicle can check the information at the driver's seat. For example, the main display module 160 may include at least one of a display, a hologram device, or a projector. As an example, the main display module 160 may be combined with at least one of a touch circuit or a sensor circuit of the input module 110 and realized as a touch screen. The main display module 160 may be controlled by a processor 190 including a circuit such as a GPU (graphic processing unit) to output visualized information to a user.

[0021] The pillar display modules 163, 165 may display a blind spot image blocked by a front pillar of the vehicle. In this case, the pillar display modules 163, 165 may be disposed on the front pillars of the vehicle inside the vehicle so that the driver of the vehicle can view the blind spot image from the driver's seat. The pillar display modules 163, 165 may include a first pillar display module 163 disposed on a front pillar on the left side inside the vehicle, and a second pillar display module 165 disposed on a front pillar on the right side inside the vehicle. Here, the dimensions (particularly, the area) of the pillar display modules 163, 165 may be substantially similar to the dimensions of the front pillar. For example, the pillar display modules 163, 165 may include at least one of a display, a hologram device, or a projector. As an example, the pillar display modules 163, 165 may be assembled with at least one of a touch circuit or a sensor circuit of the input module 110 and realized as a touch screen.

[0022] The camera module 170 may capture an external image of the vehicle. In this case, the camera module 170 may be disposed at different positions on the vehicle and capture the external image of the vehicle from different directions. For example, the camera module 170 may include at least one of one or more lenses, an image sensor, a flash, and an image signal processor (ISP). For example, some of the one or more lenses in the camera module 170 may have the same lens characteristics (e.g., angle of view, focal length, autofocus, f-number, or optical zoom). The one or more lenses may include a light source lens or a telephoto lens. For example, the image sensor may obtain an image corresponding to the object by converting light emitted or reflected from the object and transmitted through the one or more lenses into an electrical signal. According to an embodiment, the image sensor may include one image sensor selected from image sensors with different attributes, such as an RGB sensor, a BW (black and white) sensor, an IF sensor, or a UV sensor, a plurality of image sensors having the same attribute, or a plurality of image sensors having different attributes. Each of the image sensors included in the image sensor may be implemented using, for example, a charged coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor. For example, the flash may include one or more light emitting diodes (e.g., red-green-blue (RGB) LEDs, white LEDs, infrared LEDs, or ultraviolet LEDs) or a xenon lamp. In some embodiments, each of the camera modules 170 may include a serializer. The serializer may convert each external image from parallel data to serial data.

[0023] The memory 180 may store various data used by at least one component of the electronic device 100. For example, the memory 180 may include at least one of a volatile memory or a non-volatile memory. The data may include at least one program and input or output data associated therewith. The program may be stored in the memory 180 as software including at least one instruction and may include at least one of an operating system, middleware, or application, for example.

[0024] The processor 190 may control at least one component of the electronic device 100. In this way, the processor 190 can perform data processing or calculation. For example, hardware for processing data may include an arithmetic and logic unit (ALU), a floating point unit (FPU), a field programmable gate array (FPGA), a central processing unit (CPU), and / or an application processor (AP). The processor 190 may have a single-core processor structure, or may have a multi-core processor structure such as a dual core, a quad core, a hexa core, or an octa core. According to various embodiments, the processor 190 may execute instructions recorded in the memory 180. The processor 190 may obtain an external image from the camera module 170. In some embodiments, the processor 190 may include a deserializer. The deserializer may convert each external image from serial data to parallel data.

[0025] According to various embodiments, the processor 190 may process an external image. In particular, the processor 190 may perform a characteristic function of generating at least one of the blind spot images using at least one of the external images and displaying the blind spot image on at least one of the pillar display modules 163 and 165 as shown in FIG. 2. FIG. 2 shows an example in which the blind spot image is displayed on the first pillar display module 163, and similarly, the blind spot image may be displayed on the second pillar display module 165. In one embodiment, the processor 190 may include a central processing unit (CPU). In such a case, the central processing unit may perform the corresponding function. In another embodiment, the processor 190 may include not only the central processing unit but also a display controller for the pillar display modules 163 and 165 (e.g., an LCD controller if the pillar display modules 163 and 165 include an LCD (liquid crystal display)). In such a case, at least one of the central processing unit or the display controller may perform the corresponding function.

[0026] To this end, the processor 190 may activate at least one of the pillar display modules 163, 165 in response to a first user input. Meanwhile, the processor 190 may deactivate at least one of the pillar display modules 163, 165 in response to a second user input. For example, the first user input may be for turning on the vehicle's turn signals, i.e., one of the left or right turn signals, and the second user input may be for turning off the turn signals, i.e., one of the left or right turn signals. In one embodiment, the processor 190 may activate both pillar display modules 163, 165 in response to the first user input and deactivate both pillar display modules 163, 165 in response to the second user input. For example, when one of the turn indicator lights, i.e., the left or right turn indicator lights, is turned on, the processor 190 may activate both pillar display modules 163, 165, and when one of the turn indicator lights, i.e., the left or right turn indicator lights, is turned off again, the processor 190 may deactivate both pillar display modules 163, 165. In other embodiments, the processor 190 may activate one of the pillar display modules 163, 165 in response to a first user input and deactivate one of the pillar display modules 170 in response to a second user input. As an example, when the left turn indicator light is turned on, the processor 190 may activate the first pillar display module 163, and when the left turn indicator light is turned off again, the processor 190 may deactivate the first pillar display module 163. As another example, when the right turn signal light is turned on, the processor 190 may activate the second pillar display module 165, and when the right turn signal light is turned off again, the processor 190 may deactivate the second pillar display module 165.

[0027] Additionally, the processor 190 may analyze at least one of the external images to detect reference information and display the reference information together with the blind spot image on at least one of the pillar display modules 163, 165. The reference information may be information in at least one of the external images, such as a road sign, or information derived from at least one of the external images, such as a distance to an adjacent vehicle or a speed of the adjacent vehicle. Here, the reference information may be displayed in various graphic representations, such as text, symbols, images, etc.

[0028] Additionally, the processor 190 may use a sensing input input via the sensor module 120 to adjust the pillar display modules 163, 165. For example, the processor 190 may sense a sensing input indicative of ambient brightness or the like input via an illuminance sensor or the like. In such a case, the processor 190 may use the sensing input to change at least one of the brightness or reflectance of the pillar display modules 163, 165.

[0029] Fig. 3a is an exemplary diagram for explaining the configuration of the camera module 170 in the electronic device 100 according to the first embodiment. Fig. 3b is an exemplary diagram for explaining the operational characteristics of the electronic device 100 according to the first embodiment.

[0030] 3a, the camera module 170 may be installed in a vehicle for a surround view monitoring (SVM) (also referred to as around view monitoring (AVM)) function. Specifically, the camera module 170 may include a front camera module 371 configured to capture a front image of the vehicle, side camera modules 373, 375 configured to capture side images of the vehicle, respectively, and a rear camera module 377 configured to capture a rear image of the vehicle. The side camera modules 373, 375 may include a first side camera module 373 configured to capture a first side image of the left side of the vehicle, and a second side camera module 375 configured to capture a second side image of the right side of the vehicle.

[0031] In this case, the processor 190 may generate a surround view image using the front image, the side image, and the rear image. Specifically, the processor 190 may activate a surround view image monitoring function in response to a user input requesting a surround view image. In this function, the processor 190 may generate a surround view image by combining the front image, the side image, and the rear image. In one embodiment, the processor 190 may image-process the front image, the side image, and the rear image toward a floor, for example, the ground, and then combine the image-processed images. In another embodiment, the processor 190 may image-process the front image, the side image, and the rear image toward a floor, for example, the ground, and then combine the image-processed images. In this way, the processor 190 can display the surround view image on the main display module 160.

[0032] Meanwhile, the processor 190 may generate at least one blind spot image by using at least one of the front image and the side image. Specifically, the processor 190 may detect a first blind spot image to be displayed on the first pillar display module 163 by synthesizing the front image and the first side image as shown in FIG. 3b. In this way, the processor 190 may display the first blind spot image on the first pillar display module 163. Meanwhile, the processor 190 may detect a second blind spot image to be displayed on the second pillar display module 165 by synthesizing the front image and the second side image as shown in FIG. 3b. In this way, the processor 190 may display the second blind spot image on the second pillar display module 165.

[0033] Fig. 4a is an exemplary diagram for explaining the configuration of the camera module 170 in the electronic device 100 in the second embodiment. Fig. 4b is an exemplary diagram for explaining the operational characteristics of the electronic device 100 in the second embodiment.

[0034] 4a, the camera module 170 may be installed in the vehicle for the blind spot monitoring function by the pillar display modules 163 and 165. Specifically, the camera module 170 may be disposed on the front pillars on the outside of the vehicle. The camera module 170 may include a first camera module 473 disposed on the front pillar on the left side of the vehicle and configured to capture a first image, and a second camera module 475 disposed on the front pillar on the right side of the vehicle and configured to capture a second image.

[0035] In this case, the processor 190 may generate at least one blind spot image using at least one of the first image or the second image. Specifically, the processor 190 may detect a first blind spot image to be displayed on the first pillar display module 163 from the first image, as shown in FIG. 4b. In this way, the processor 190 may display the first blind spot image on the first pillar display module 163. Meanwhile, the processor 190 may detect a second blind spot image to be displayed on the second pillar display module 165 from the second image, as shown in FIG. 4b. In this way, the processor 190 may display the second blind spot image on the second pillar display module 165.

[0036] As described above, in the electronic device 100 according to the first embodiment, the camera module 170 may include a front camera module 371 configured to capture a front image of the vehicle, side camera modules 373 and 375 configured to capture side images of the vehicle, and a rear camera module 377 configured to capture a rear image of the vehicle. The side camera modules 373 and 375 may include a first side camera module 373 configured to capture a first side image of the left side of the vehicle, and a second side camera module 375 configured to capture a second side image of the right side of the vehicle. An embodiment of a method of operating such an electronic device 100 will be described with reference to FIGS. 5 and 6.

[0037] FIG. 5 is a flow chart illustrating a method of operation of the electronic device 100 in one of the first embodiments.

[0038] 5, in operation 511, the electronic device 100 may sense a first user input for turning on a turn signal light. Specifically, when a user operates the lever 200 to turn on a turn signal light, i.e., any one of the left turn signal light or the right turn signal light, the interface module 140 may generate a first user input and transmit the first user input to the processor 190. Thus, the processor 190 may detect the first user input via the interface module 140.

[0039] In response, in step 513, the electronic device 100 may activate both pillar display modules 163, 165. Specifically, with the first pillar display module 163 and the second pillar display module 165 in a deactivated state, the processor 190 may activate the first pillar display module 163 and the second pillar display module 165. This allows the first pillar display module 163 and the second pillar display module 165 to remain in an activated state while one of the turn indicator lights, i.e., the left turn indicator light or the right turn indicator light, is turned on.

[0040] Next, in operation 515, the electronic device 100 may generate a blind spot image using the front image and the side image. Specifically, the processor 190 may obtain a front image, a first side image, and a second side image from the front camera module 371, the first side camera module 373, and the second side camera module 375, respectively. At this time, the processor 190 may correct the front image, the first side image, and the second side image, respectively, through camera calibration. In this way, the processor 190 may remove distortion from the front image, the first side image, and the second side image, respectively. Next, the processor 190 may generate a first blind spot image by combining the front image and the first side image, and generate a second blind spot image by combining the front image and the second side image. At this time, the memory 180 may store setting information including positions and dimensions of the pillar display modules 163 and 165 in the vehicle. More specifically, the processor 190 may synthesize a front image and a first side image into a single image and detect a first blind spot image by cropping the single image based on the setting information. Similarly, the processor 190 may synthesize a front image and a second side image into a single image and detect a second blind spot image by cropping the single image based on the setting information.

[0041] Next, in operation 517, the electronic device 100 may display the blind spot area image on each of the pillar display modules 163 and 165. Specifically, the processor 190 may display the first blind spot area image on the first pillar display module 163 and the second blind spot area image on the second pillar display module 165. More specifically, the processor 190 may resize the first blind spot area image in accordance with a display area of ​​the first pillar display module 163 and then display it on the first pillar display module 163. Similarly, the processor 190 may resize the second blind spot area image in accordance with a display area of ​​the second pillar display module 165 and then display it on the second pillar display module 165.

[0042] Next, the electronic device 100 may repeat steps 515 and 517 until a second user input for turning off the turn signal lights is detected in step 519. Specifically, the processor 190 may continuously generate blind spot images using the forward images and the side images and display the blind spot images on the pillar display modules 163 and 165, respectively.

[0043] Meanwhile, in step 519, the electronic device 100 may sense a second user input to turn off the turn signal lights. Specifically, when the user operates the lever 200 to turn off one of the turn signals, i.e., the left turn signal light or the right turn signal light, the interface module 140 may generate a second user input and transmit the second user input to the processor 190. Thus, the processor 190 may detect the second user input via the interface module 140.

[0044] In response, in step 521, the electronic device 100 may deactivate both pillar display modules 163, 165. Specifically, in a state in which the first pillar display module 163 and the second pillar display module 165 are activated, the processor 190 may deactivate the first pillar display module 163 and the second pillar display module 165. This allows the first pillar display module 163 and the second pillar display module 165 to maintain a deactivated state while the turn indicator lights, i.e., the left turn indicator light and the right turn indicator light, are turned off.

[0045] According to the present embodiment, while a turn signal light, i.e., one of a left turn signal light or a right turn signal light, is turned on, the electronic device 100 may display a blind spot image on the pillar display modules 163, 165. Specifically, while the vehicle is turning left, the processor 190 may display a first blind spot image on the first pillar display module 163 while displaying a second blind spot image on the second pillar display module 165. Alternatively, while the vehicle is turning right, the processor 190 may display a first blind spot image on the first pillar display module 163 while displaying a second blind spot image on the second pillar display module 165.

[0046] FIG. 6 is a flowchart illustrating an operation method of the electronic device 100 according to another embodiment of the first embodiment.

[0047] 6 , in operation 611, the electronic device 100 may sense a first user input for turning on the left turn signal. Specifically, when a user operates the lever 200 to turn on the left turn signal, the interface module 140 may generate a first user input and transmit the first user input to the processor 190. Thus, the processor 190 may sense the first user input via the interface module 140.

[0048] In response to this, in operation 613, the electronic device 100 may activate the first pillar display module 163. Specifically, in a state in which the first pillar display module 163 and the second pillar display module 165 are inactivated, the processor 190 may activate the first pillar display module 163. In this way, while the left turn signal light is on, the first pillar display module 163 can be maintained in an activated state and the second pillar display module 165 can be maintained in a deactivated state.

[0049] Next, in operation 615, the electronic device 100 may generate a first blind spot image using the front image and the first side image. Specifically, the processor 190 may obtain the front image and the first side image from the front camera module 371 and the first side camera module 373, respectively. At this time, the processor 190 may correct the front image and the first side image, respectively, by camera calibration. As a result, the processor 190 may remove distortion from the front image and the first side image, respectively. Next, the processor 190 may generate the first blind spot image by synthesizing the front image and the first side image. At this time, the memory 180 may store setting information including positions and dimensions of the pillar display modules 163 and 165 in the vehicle. More specifically, the processor 190 may synthesize the front image and the first side image into a single image, and may detect the first blind spot image by cropping the single image based on the setting information.

[0050] Next, in operation 617, the electronic device 100 may display the first blind spot image on the first pillar display module 163. Specifically, the processor 190 may resize the first blind spot image in accordance with a display area of ​​the first pillar display module 163, and then display the first blind spot image on the first pillar display module 163.

[0051] Next, the electronic device 100 may repeat steps 615 and 617 until a second user input for turning off the left turn signal light is detected in step 619. Specifically, the processor 190 may continuously generate a first blind spot image from the forward image and the first side image and display the first blind spot image on the first pillar display module 163.

[0052] Meanwhile, in operation 619, the electronic device 100 may sense a second user input for turning off the left turn signal light. Specifically, when the user operates the lever 200 to turn off the left turn signal light, the interface module 140 may generate the second user input and transmit the second user input to the processor 190. Thus, the processor 190 may sense the second user input via the interface module 140.

[0053] In response to this, in operation 621, the electronic device 100 may deactivate the first pillar display module 163. Specifically, in a state in which the first pillar display module 163 is activated and the second pillar display module 165 is deactivated, the processor 190 may deactivate the first pillar display module 163. In this way, the first pillar display module 163 and the second pillar display module 165 can be maintained in a deactivated state while the left and right turn signals are turned off.

[0054] Meanwhile, in operation 631, the electronic device 100 may sense a first user input for turning on the right turn signal. Specifically, when a user operates the lever 200 to turn on the right turn signal, the interface module 140 may generate a first user input and transmit the first user input to the processor 190. Thus, the processor 190 may sense the first user input via the interface module 140.

[0055] In response to this, in operation 633, the electronic device 100 may activate the second pillar display module 165. Specifically, in a state in which the first pillar display module 163 and the second pillar display module 165 are inactivated, the processor 190 may activate the second pillar display module 165. In this way, while the right turn signal is on, the first pillar display module 163 can maintain a deactivated state and the second pillar display module 165 can maintain an activated state.

[0056] Next, in step 635, the electronic device 100 may generate a second blind spot image using the front image and the second side image. Specifically, the processor 190 may obtain the front image and the second side image from the front camera module 371 and the second side camera module 375, respectively. At this time, the processor 190 may correct the front image and the second side image, respectively, by camera calibration. In this way, the processor 190 may remove distortion from the front image and the second side image, respectively. Next, the processor 190 may generate the second blind spot image by synthesizing the front image and the second side image. At this time, the memory 180 may store setting information including positions and dimensions of the pillar display modules 163 and 165 in the vehicle. More specifically, the processor 190 may synthesize the front image and the second side image into a single image and detect the second blind spot image by cropping the single image based on the setting information.

[0057] Next, in operation 637, the electronic device 100 may display the second blind spot image on the second pillar display module 165. Specifically, the processor 190 may resize the second blind spot image in accordance with a display area of ​​the second pillar display module 165, and then display the second blind spot image on the second pillar display module 165.

[0058] Next, the electronic device 100 may repeat steps 635 and 637 until a second user input for turning off the right turn signal light is detected in step 639. Specifically, the processor 190 may continuously generate a second blind spot image from the forward image and the second side image and display the second blind spot image on the second pillar display module 165.

[0059] Meanwhile, in operation 639, the electronic device 100 may sense a second user input for turning off the right turn signal light. Specifically, when the user operates the lever 200 to turn off the right turn signal light, the interface module 140 may generate the second user input and transmit the second user input to the processor 190. Thus, the processor 190 may sense the second user input via the interface module 140.

[0060] In response to this, in operation 641, the electronic device 100 may deactivate the second pillar display module 165. Specifically, in a state in which the first pillar display module 163 is deactivated and the second pillar display module 165 is activated, the processor 190 may deactivate the second pillar display module 165. This allows the first pillar display module 163 and the second pillar display module 165 to maintain a deactivated state while the left and right turn signals are turned off.

[0061] According to the present embodiment, while a turn signal light, i.e., one of a left turn signal light or a right turn signal light, is on, the electronic device 100 may display a blind spot image on one of the pillar display modules 163, 165. Specifically, while the vehicle is turning left, the processor 190 may display a first blind spot image on the first pillar display module 163. Alternatively, while the vehicle is turning right, the processor 190 may display a second blind spot image on the second pillar display module 165.

[0062] As described above, in the electronic device 100 according to the second embodiment, the camera modules 170 may be disposed on the front pillars outside the vehicle. The camera modules 170 may include a first camera module 473 disposed on the front pillar on the left side of the vehicle and configured to capture a first image, and a second camera module 475 disposed on the front pillar on the right side of the vehicle and configured to capture a second image. An embodiment of a method for operating the electronic device 100 will be described with reference to FIGS. 7 and 8.

[0063] FIG. 7 is a flow chart illustrating a method of operating the electronic device 100 according to one of the second embodiments.

[0064] 7 , in step 711, the electronic device 100 may sense a first user input to turn on a turn signal. Specifically, when a user operates the lever 200 to turn on one of the turn signals, i.e., the left turn signal or the right turn signal, the interface module 140 may generate a first user input and transmit the first user input to the processor 190. Thus, the processor 190 may sense the first user input via the interface module 140.

[0065] In response to this, in step 713, the electronic device 100 may activate both pillar display modules 163, 165. Specifically, in a state in which the first pillar display module 163 and the second pillar display module 165 are deactivated, the processor 190 may activate the first pillar display module 163 and the second pillar display module 165. This allows the first pillar display module 163 and the second pillar display module 165 to maintain an activated state while a turn signal light, i.e., one of the left turn signal light or the right turn signal light, is turned on. At this time, in a state in which the first camera module 473 and the second camera module 475 are deactivated, the processor 190 may also activate the first camera module 473 and the second camera module 475. This allows the first camera module 473 to maintain an activated state while a turn signal light, i.e., one of the left turn signal light or the right turn signal light, is turned on.

[0066] Next, in operation 715, the electronic device 100 may generate a blind spot image using images from the first camera module 473 and the second camera module 475. Specifically, the processor 190 may obtain a first image from the first camera module 473 and a second image from the second camera module 473. At this time, setting information including positions and dimensions of the pillar display modules 163 and 165 in the vehicle may be recorded in the memory 180. More specifically, the processor 190 may detect the first blind spot image by cropping the first image based on the setting information. Similarly, the processor 190 may detect the second blind spot image by cropping the second image based on the setting information.

[0067] Next, in operation 717, the electronic device 100 may display the blind spot area image on each of the pillar display modules 163 and 165. Specifically, the processor 190 may display the first blind spot area image on the first pillar display module 163 and the second blind spot area image on the second pillar display module 165. More specifically, the processor 190 may resize the first blind spot area image in accordance with the display area of ​​the first pillar display module 163 and then display it on the first pillar display module 163. Similarly, the processor 190 may resize the second blind spot area image in accordance with the display area of ​​the second pillar display module 165 and then display it on the second pillar display module 165.

[0068] Next, in step 719, the electronic device 100 may repeat steps 715 and 717 until a second user input for turning off the turn signal lights is detected. Specifically, the processor 190 may generate blind spot images using images from the first camera module 473 and the second camera module 475, and display the blind spot images on the pillar display modules 163 and 165, respectively.

[0069] Meanwhile, in step 719, the electronic device 100 may sense a second user input to turn off the turn signals. Specifically, when the user operates the lever 200 to turn off one of the turn signals, i.e., the left turn signal or the right turn signal, the interface module 140 may generate a second user input and transmit the second user input to the processor 190. Thus, the processor 190 may sense the second user input via the interface module 140.

[0070] In response to this, in operation 721, the electronic device 100 may deactivate both pillar display modules 163, 165. Specifically, in a state in which the first pillar display module 163 and the second pillar display module 165 are activated, the processor 190 may deactivate the first pillar display module 163 and the second pillar display module 165. As a result, the first pillar display module 163 and the second pillar display module 165 can maintain a deactivated state while the turn indicator lights, i.e., the left turn indicator light and the right turn indicator light, are turned off. At this time, in a state in which the first camera module 473 and the second camera module 475 are activated, the processor 190 may deactivate the first camera module 473 and the second camera module 475. As a result, the first camera module 473 and the second camera module 475 can also maintain a deactivated state while the turn indicator lights, i.e., the left turn indicator light and the right turn indicator light, are turned off.

[0071] According to the present embodiment, while a turn signal light, i.e., one of a left turn signal light or a right turn signal light, is turned on, the electronic device 100 may display a blind spot image on the pillar display modules 163, 165. Specifically, while the vehicle is turning left, the processor 190 may display a first blind spot image on the first pillar display module 163 while displaying a second blind spot image on the second pillar display module 165. Alternatively, while the vehicle is turning right, the processor 190 may display a first blind spot image on the first pillar display module 163 while displaying a second blind spot image on the second pillar display module 165.

[0072] FIG. 8 is a flowchart illustrating an operation method of the electronic device 100 according to another embodiment of the second embodiment.

[0073] 8 , in operation 811, the electronic device 100 may sense a first user input for turning on the left turn signal. Specifically, when a user operates the lever 200 to turn on the left turn signal, the interface module 140 may generate a first user input and transmit the first user input to the processor 190. Thus, the processor 190 may sense the first user input via the interface module 140.

[0074] In response to this, in operation 813, the electronic device 100 may activate the first pillar display module 163. Specifically, in a state in which the first pillar display module 163 and the second pillar display module 165 are inactivated, the processor 190 may activate the first pillar display module 163. As a result, while the left turn signal light is on, the first pillar display module 163 may maintain an activated state, and the second pillar display module 165 may maintain a deactivated state. At this time, in a state in which the first camera module 473 and the second camera module 475 are inactivated, the processor 190 may also activate the first camera module 473. As a result, while the left turn signal light is on, the first camera module 473 may maintain an activated state, and the second camera module 475 may maintain a deactivated state.

[0075] Next, in operation 815, the electronic device 100 may generate a first blind spot image from the first image from the first camera module 473. Specifically, the processor 190 may acquire the first image from the first camera module 473. At this time, setting information including positions and dimensions of the pillar display modules 163 and 165 in the vehicle may be recorded in the memory 180. More specifically, the processor 190 may detect the first blind spot image by cropping the first image based on the setting information.

[0076] Next, in operation 817, the electronic device 100 may display the first blind spot image on the first pillar display module 163. Specifically, the processor 190 may resize the first blind spot image in accordance with a display area of ​​the first pillar display module 163, and then display the first blind spot image on the first pillar display module 163.

[0077] Next, the electronic device 100 may repeat steps 815 and 817 until a second user input for turning off the left turn signal light is detected in step 819. Specifically, the processor 190 may generate a first blind spot image from the first image from the first camera module 473 and display the first blind spot image on the first pillar display module 163.

[0078] Meanwhile, in operation 819, the electronic device 100 may sense a second user input for turning off the left turn signal light. Specifically, when the user operates the lever 200 to turn off the left turn signal light, the interface module 140 may generate the second user input and transmit the second user input to the processor 190. Thus, the processor 190 may sense the second user input via the interface module 140.

[0079] In response to this, in operation 821, the electronic device 100 may deactivate the first pillar display module 163. Specifically, in a state in which the first pillar display module 163 is activated and the second pillar display module 165 is deactivated, the processor 190 may deactivate the first pillar display module 163. As a result, the first pillar display module 163 and the second pillar display module 165 can maintain a deactivated state while the left and right turn indicator lights are turned off. At this time, in a state in which the first camera module 473 is activated and the second camera module 475 is deactivated, the processor 190 may also deactivate the first camera module 473. As a result, the first camera module 473 and the second camera module 475 can also maintain a deactivated state while the left and right turn indicator lights are turned off.

[0080] Meanwhile, in operation 831, the electronic device 100 may sense a first user input for turning on the right turn signal. Specifically, when a user operates the lever 200 to turn on the right turn signal, the interface module 140 may generate a first user input and transmit the first user input to the processor 190. Thus, the processor 190 may sense the first user input via the interface module 140.

[0081] In response to this, in operation 833, the electronic device 100 may activate the second pillar display module 165. Specifically, in a state in which the first pillar display module 163 and the second pillar display module 165 are inactivated, the processor 190 may activate the second pillar display module 165. As a result, while the right turn signal is on, the first pillar display module 163 may maintain a deactivated state, and the second pillar display module 165 may maintain an activated state. At this time, in a state in which the first camera module 473 and the second camera module 475 are inactivated, the processor 190 may also activate the second camera module 475. As a result, while the left turn signal is on, the first camera module 473 may maintain a deactivated state, and the second camera module 475 may maintain an activated state.

[0082] Next, in operation 835, the electronic device 100 may generate a second blind spot image using the second image from the second camera module 475. Specifically, the processor 190 may obtain the second image from the second camera module 475. At this time, setting information including positions and dimensions of the pillar display modules 163 and 165 in the vehicle may be recorded in the memory 180. More specifically, the processor 190 may detect the second blind spot image by cropping the second image based on the setting information.

[0083] Next, in operation 837, the electronic device 100 may display the second blind spot image on the second pillar display module 165. Specifically, the processor 190 may resize the second blind spot image in accordance with a display area of ​​the second pillar display module 165, and then display the second blind spot image on the second pillar display module 165.

[0084] Next, the electronic device 100 may repeat steps 835 and 837 until a second user input for turning off the right turn signal is detected in step 839. Specifically, the processor 190 may generate a second blind spot image from the second image from the second camera module 475 and display the second blind spot image on the second pillar display module 165.

[0085] Meanwhile, in operation 839, the electronic device 100 may sense a second user input for turning off the right turn signal light. Specifically, when the user operates the lever 200 to turn off the right turn signal light, the interface module 140 may generate the second user input and transmit the second user input to the processor 190. Thus, the processor 190 may sense the second user input via the interface module 140.

[0086] In response to this, in operation 841, the electronic device 100 may deactivate the second pillar display module 165. Specifically, in a state in which the first pillar display module 163 is deactivated and the second pillar display module 165 is activated, the processor 190 may deactivate the second pillar display module 165. As a result, the first pillar display module 163 and the second pillar display module 165 can maintain a deactivated state while the left and right turn indicator lights are turned off. At this time, in a state in which the first camera module 473 is deactivated and the second camera module 475 is activated, the processor 190 may also deactivate the second camera module 475. As a result, the first camera module 473 and the second camera module 475 can also maintain a deactivated state while the left and right turn indicator lights are turned off.

[0087] According to the present embodiment, while a turn signal light, i.e., one of a left turn signal light or a right turn signal light, is on, the electronic device 100 may display a blind spot image on one of the pillar display modules 163, 165. Specifically, while the vehicle is turning left, the processor 190 may display a first blind spot image on the first pillar display module 163. Alternatively, while the vehicle is turning right, the processor 190 may display a second blind spot image on the second pillar display module 165.

[0088] According to various embodiments, it is possible to remove a blind spot blocked by a front pillar of a vehicle. That is, the electronic device 100 can display at least one of the images of the blind spot blocked by the front pillar of the vehicle on at least one of the pillar display modules 163, 165 arranged on the front pillar. This allows the driver to focus on the front and sides of the vehicle without the blind spot blocked by the front pillar. In particular, when the vehicle is turning, the driver can easily grasp the surrounding situation of the vehicle without the blind spot. This can ensure the safety of not only the driver and other vehicle passengers, but also pedestrians. According to the first embodiment, since the blind spot image is generated using an external image from the camera module 170 that is generally installed in the vehicle for the surround view monitoring function, it is not necessary to newly add another camera module. This can prevent an increase in the number of work steps, an increase in installation costs, an increase in the vehicle weight, etc., due to the addition of another camera module. According to the second embodiment, the blind spot image is generated using an external image from a camera module mounted on the vehicle for the blind spot monitoring function by the pillar display modules 163 and 165, thereby reducing the operating load of the processor in the electronic device, since the processor does not need to synthesize multiple external images to generate one blind spot image.

[0089] In summary, the present disclosure provides an electronic device 100 and method of operation for providing blind spot video through a front pillar of a vehicle.

[0090] In the present disclosure, the electronic device 100 mounted on a vehicle may include at least two camera modules 170 configured to capture external images of the vehicle, pillar display modules 163, 165 arranged inside the vehicle on the front pillars of the vehicle, and a processor 190 connected to the camera module 170 and the pillar display modules 163, 165 and configured to display at least one of the blind spot images from the front pillars on at least one of the pillar display modules 163, 165 using at least one of the external images.

[0091] According to various embodiments, the processor 190 may be configured to activate at least one of the pillar display modules 163, 165 in response to a first user input, and to deactivate at least one of the pillar display modules 163, 165 in response to a second user input.

[0092] According to various embodiments, the first user input may be for turning on one of the left or right turn signals of the vehicle, and the second user input may be for turning off one of the left or right turn signals.

[0093] According to various embodiments, the pillar display modules 163, 165 may include a first pillar display module 163 arranged on a front pillar on the left side within the vehicle interior, and a second pillar display module 165 arranged on a front pillar on the right side within the vehicle interior.

[0094] According to the first embodiment, the camera module 170 may include a front camera module 371 configured to capture a front image of the vehicle, a first side camera module 373 configured to capture a first side image of the left side of the vehicle, and a second side camera module 375 configured to capture a second side image of the right side of the vehicle.

[0095] According to the first embodiment, the processor 190 may be configured to detect a first blind spot image for synthesizing a forward image and a first side image to display on the first pillar display module 163, and to detect a second blind spot image for synthesizing a forward image and a second side image to display on the second pillar display module 165.

[0096] According to a second embodiment, the camera module 170 may include a first camera module 473 and a second camera module 475 disposed on each of the front pillars on the exterior of the vehicle.

[0097] According to the second embodiment, the processor 190 may be configured to detect a first blind spot image from the image from the first camera module 473 for display on the first pillar display module 163, and to detect a second blind spot image from the image from the second camera module 475 for display on the second pillar display module 165.

[0098] In some embodiments, the processor 190 may be configured to activate the first pillar display module 163 in response to a first user input for turning on the vehicle's left turn indicator light, deactivate the first pillar display module 163 in response to a second user input for turning off the left turn indicator light, activate the second pillar display module 165 in response to a first user input for turning on the vehicle's right turn indicator light, and deactivate the second pillar display module 165 in response to a second user input for turning off the right turn indicator light.

[0099] According to various embodiments, the processor 190 may be configured to detect blind spot images from the external image based on configuration information including the positions and dimensions of the pillar display modules 163, 165 in the vehicle.

[0100] In the present disclosure, an operating method of the electronic device 100 mounted on a vehicle may include steps of detecting at least one of the blind spot area images through a front pillar of the vehicle using at least one external image of the vehicle (steps 515, 615, 635, 715, 815, and 835), and displaying at least one of the blind spot area images through at least one of the pillar display modules 163, 165 respectively arranged on the front pillars (steps 517, 617, 637, 717, 817, and 837).

[0101] According to various embodiments, the method of operating the electronic device 100 may further include a step of activating at least one of the pillar display modules 163, 165 in response to a first user input (step 513, step 613, step 633, step 713, step 813, step 833).

[0102] According to various embodiments, the steps of detecting at least one of the blind spot images (steps 515, 615, 635, 715, 815, 835) and the steps of displaying at least one of the blind spot images (steps 513, 613, 633, 713, 813, 833) may be repeated while at least one of the pillar display modules 163, 165 is activated.

[0103] According to various embodiments, the operating method of the electronic device 100 may further include steps of deactivating at least one of the pillar display modules 163, 165 in response to a second user input (steps 521, 621, 641, 721, 821, 841).

[0104] According to various embodiments, the first user input may be for turning on one of the left or right turn signals of the vehicle, and the second user input may be for turning off one of the left or right turn signals.

[0105] According to various embodiments, the pillar display modules 163, 165 may include a first pillar display module 163 arranged on a front pillar on the left side within the vehicle interior, and a second pillar display module 165 arranged on a front pillar on the right side within the vehicle interior.

[0106] According to the first embodiment, the electronic device 100 may include a front camera module 371 configured to capture a front image of the vehicle, a first side camera module 373 configured to capture a first side image of the left side of the vehicle, and a second side camera module 375 configured to capture a second side image of the right side of the vehicle.

[0107] According to the first embodiment, the step of detecting at least one of the blind spot area images (step 515, step 615, step 635) may include at least one of a step of detecting a first blind spot area image for synthesizing a forward image and a first side image to display on the first pillar display module 163, or a step of detecting a second blind spot area image for synthesizing a forward image and a second side image to display on the second pillar display module 165.

[0108] According to a second embodiment, the electronic device 100 may include a first camera module 473 and a second camera module 475 arranged on each of the front pillars on the exterior of the vehicle.

[0109] According to the second embodiment, the steps of detecting at least one of the blind spot area images (steps 715, 815, and 835) may include at least one of a step of detecting a first blind spot area image for display on the first pillar display module 163 from an image from the first camera module 473, and a step of detecting a second blind spot area image for display on the second pillar display module 165 from an image from the second camera module 475.

[0110] In some embodiments, the step of activating at least one of the pillar display modules 163, 165 (step 613, step 633, step 813, step 833) may include a step of activating the first pillar display module 163 (step 613, step 813) in response to a first user input for turning on the left turn signal of the vehicle, or a step of activating the second pillar display module 165 (step 633, step 833) in response to a first user input for turning on the right turn signal of the vehicle.

[0111] In some embodiments, the step of deactivating at least one of the pillar display modules 163, 165 (step 621, step 621, step 641) may include a step of deactivating the first pillar display module 163 in response to a second user input for turning off the left turn signal (step 621, step 821), or a step of deactivating the second pillar display module 165 in response to a second user input for turning off the right turn signal (step 621, step 821).

[0112] According to various embodiments, the steps of detecting at least one of the blind spot images (steps 515, 615, 635, 715, 815, and 835) may detect at least one of the blind spot images from the external image based on setting information including the positions and dimensions of the pillar display modules 163 and 165 in the vehicle.

[0113] In the present disclosure, the electronic device 100 mounted on the vehicle may include a front camera module 371 configured to capture a front image of the vehicle, side camera modules 373, 375 configured to capture a side image of the vehicle, and a rear camera module 377 configured to capture a rear image of the vehicle, pillar display modules 163, 165 disposed on the front pillars of the vehicle, respectively, inside the vehicle, a main display module 160 mounted inside the vehicle separately from the pillar display modules 163, 165, and a processor 190 connected to the front camera module 371, the side camera modules 373, 375, the rear camera module 377, the main display module 160, and the pillar display modules 163, 165, and configured to display a surround view image on the main display module 160 using the front image, the side image, and the rear image, and to display at least one of the blind spot images caused by the front pillars on at least one of the pillar display modules 163, 165 using at least one of the front image and the side image.

[0114] According to various embodiments, the processor 190 may be configured to synthesize a surround view image by synthesizing the front image, the side image, and the rear image in response to a user input requesting a surround view image.

[0115] In the present disclosure, an operating method of the electronic device 100 mounted on a vehicle may include the steps of capturing a front image, a side image, and a rear image of the vehicle, displaying a surround view image on the main display module 160 using the front image, the side image, and the rear image in response to a user input, and activating at least one of the pillar display modules 163, 165 disposed on the front pillars, respectively, in response to the user input, and displaying at least one of the blind spot images from the front pillars on at least one of the pillar display modules 163, 165 using at least one of the front image and the side image.

[0116] 9 is a block diagram showing a vehicle 2000 equipped with an electronic device 100 according to various embodiments. FIG 10 is a block diagram showing a control device 2100 of the vehicle of FIG 9.

[0117] 9 and 10, the electronic device 100 according to various embodiments may be mounted on a vehicle 2000, and the vehicle 2000 may include a control device 2100. In this case, the vehicle 2000 may be an autonomous vehicle. In some embodiments, at least one component of the electronic device 100 may be integrated into at least one component of the control device 2100.

[0118] The control device 2100 may include a controller 2120 including a memory 2122 and a processor 2124 , a sensor 2110 , a wireless communication device 2130 , a LIDAR 2140 , and a camera module 2150 .

[0119] The controller 2120 may be configured at the time of manufacture by the vehicle manufacturer, may be additionally configured after manufacture to perform autonomous driving functions, or may include configuration to perform additional ongoing functions by upgrading the controller 2120 configured at manufacture.

[0120] The controller 2120 may transmit control signals to other components included in the vehicle, such as the sensor 2110, the engine 2006, the user interface 2008, the wireless communication device 2130, the LIDAR 2140, and the camera module 2150. The controller 2120 may also transmit control signals to an acceleration device, a braking system, a steering device, or a navigation device associated with the running of the vehicle, which are not shown in the figure.

[0121] The controller 2120 may control the engine 2006, for example, by detecting the speed limit of the road on which the vehicle 2000 is traveling and controlling the engine 2006 so that the traveling speed does not exceed the speed limit, or by controlling the engine 2006 so that the traveling speed of the vehicle 2000 accelerates within a range not exceeding the speed limit. Additionally, when the sensing modules 2004a, 2004b, 2004c, and 2004d detect the environment outside the vehicle and transmit it to the sensor 2110, the controller 2120 may receive it and generate a signal to control the engine 2006 or a steering device (not shown) to control the traveling of the vehicle.

[0122] The controller 2120 may control the engine 2006 or the brake system to decelerate the traveling vehicle when there is another vehicle or an obstacle ahead of the vehicle, and may control the trajectory, driving path, steering angle, and other aspects of the speed. Alternatively, the controller 2120 may generate necessary control signals to control the traveling of the vehicle according to other external environment recognition information such as the lane the vehicle is traveling in, driving signals, and the like.

[0123] In addition to generating its own control signals, the controller 2120 can also control the running of the vehicle by communicating with surrounding vehicles or a central server and sending instructions to control peripheral devices based on the information received.

[0124] In addition, when the position or angle of view of the camera module 2150 is changed, the vehicle or lane may not be accurately recognized. To prevent this, the controller 2120 may generate a control signal to control the camera module 2150 to perform calibration. Therefore, the controller 2120 can continue to maintain the normal mounting position, direction, angle of view, etc. of the camera module 2150 by generating a calibration control signal through the camera module 2150, even if the mounting position of the camera module 2150 is changed due to vibration or impact caused by the movement of the autonomous vehicle 2000. The controller 2120 may generate a control signal to perform calibration of the camera module 2150 when the initial mounting position, direction, angle of view information of the camera module 2150 stored in advance and the initial mounting position, direction, angle of view information, etc. of the camera module 2150 measured during the traveling of the autonomous vehicle 2000 differ by a threshold value or more.

[0125] The controller 2120 may include a memory 2122 and a processor 2124. The processor 2124 may execute software stored in the memory 2122 in accordance with control signals from the controller 2120. Specifically, the controller 2120 may store data and instructions in the memory 2122 for detecting a field of view image from a rear image of the vehicle 2000, and the instructions may be executed by the processor 2124 to implement one or more methods disclosed herein.

[0126] In this case, the memory 2122 may be stored in a non-volatile recording medium executable by the processor 2124. The memory 2122 may store software and data through an appropriate internal or external device. The memory 2122 may be configured with a memory 2122 device connected to a RAM (random access memory), a ROM (read only memory), a hard disk, or a dongle.

[0127] The memory 2122 may store at least an operating system (OS), user applications, and executable instructions, and may also store application data and array data structures.

[0128] The processor 2124 may be a microprocessor or any suitable electronic processor, such as a controller, microcontroller, or state machine.

[0129] The processor 2124 may be implemented by a combination of computing devices, which may be digital signal processors, microprocessors, or any suitable combination thereof.

[0130] The controller 2100 may also monitor interior and exterior characteristics of the vehicle 2000 via at least one or more sensors 2110 to sense conditions.

[0131] The sensor 2110 may be configured with at least one or more sensing modules 2004, which may be implemented in a specific location of the vehicle 2000 depending on the purpose of sensing. The sensing modules 2004 may be located at the bottom, rear end, front end, top or side end of the vehicle 2000, may be located in an internal part or tire of the vehicle, etc.

[0132] Thus, the sensing module 2004 can sense information related to driving, such as the engine 2006, tires, steering angle, speed, vehicle weight, etc., as internal information of the vehicle. In addition, at least one of the sensing modules 2004 may be configured with an acceleration sensor 2110, a gyroscope, an image sensor 2110, a RADAR, an ultrasonic sensor, a LiDAR sensor, etc., and may sense motion information of the vehicle 2000.

[0133] The sensing module 2004 can receive specific data on external environmental conditions such as road condition information on the road where the vehicle 2000 is located, surrounding vehicle information, weather, etc. as external information, and can sense vehicle parameters based on the data. The sensed information can be temporarily or long-term stored in the memory 2122 depending on the purpose.

[0134] The sensor 2110 may integrate and collect information from the sensing module 2004 to collect information occurring inside and outside the vehicle 2000.

[0135] The control device 2100 may further include a wireless communication device 2130 .

[0136] The wireless communication device 2130 is configured to realize wireless communication between the vehicles 2000. For example, it allows the vehicle 2000 to communicate with a user's mobile phone, or other wireless communication devices 2130, other vehicles, a central device (traffic control device), a server, etc. The wireless communication device 2130 may transmit and receive wireless signals according to a connection wireless protocol. The wireless communication protocol may be Wi-Fi, Bluetooth, Long-Term Evolution (LTE), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Global Systems for Mobile Communications (GSM), but the communication protocol should not be limited thereto.

[0137] The vehicle 2000 may also realize inter-vehicle communication through the wireless communication device 2130. That is, the wireless communication device 2130 may perform communication with other vehicles on the road and other vehicles through vehicle-to-vehicle (V2V) communication. The vehicle 2000 may transmit and receive information such as driving warnings and traffic information through the inter-vehicle communication, and may also request information from and receive requests from other vehicles. For example, the wireless communication device 2130 may perform V2V communication as a dedicated short-range communication (DSRC) device or a C-V2V (Celluar-V2V) device. In addition to inter-vehicle communication, communication between vehicles and other things (e.g., electronic devices carried by pedestrians) (V2X, Vehicle to Everything communication) may also be realized through the wireless communication device 2130.

[0138] The control device 2100 may also include a LIDAR device 2140. The LIDAR device 2140 may detect objects around the vehicle 2000 during operation using data sensed through the LIDAR sensor. The LIDAR device 2140 may transmit detected information to the controller 2120, and the controller 2120 may operate the vehicle 2000 based on the detected information. For example, when the detection information indicates that there is a vehicle traveling at a low speed ahead, the controller 2120 may command the engine 2006 to reduce the vehicle speed. Alternatively, the controller 2120 may command the vehicle to reduce the entry speed according to the curvature of the curve into which the vehicle is entering.

[0139] The control device 2100 may further include a camera module 2150. The controller 2120 may extract object information from an external image captured by the camera module 2150, and the controller 2120 may process the information related thereto.

[0140] The control device 2100 may further include an imaging device for recognizing the external environment. In addition to the LIDAR 2140, RADAR, GPS devices, odometry devices, and other computer vision devices may be used, which may be selected or operated simultaneously as needed to enable more precise sensing.

[0141] The vehicle 2000 may further include a user interface 2008 for user input to the control device 2100 described above. The user interface 2008 may allow the user to input information through appropriate interaction. For example, the user interface 2008 may be realized by a touch screen, a keypad, an operation button, etc. The user interface 2008 may transmit the input or command to the controller 2120, and the controller 2120 may execute a control operation of the vehicle in response to the input or command.

[0142] The user interface 2008 may be a device external to the vehicle 2000 and may communicate with the vehicle 2000 through the wireless communication device 2130. For example, the user interface 2008 may be capable of linking with a mobile phone, tablet, or other computer device.

[0143] Additionally, while the vehicle 2000 has been described as including the engine 2006, it may include other types of propulsion systems. For example, the vehicle may be powered by electric energy, hydrogen energy, or a hybrid system that combines these. Thus, the controller 2120 may include propulsion mechanisms according to the propulsion system of the vehicle 2000 and provide corresponding control signals to the configuration of each propulsion mechanism.

[0144] Hereinafter, the detailed configuration of the control device 2100 for providing a blind spot image through the front pillar of the vehicle 2000 will be described in more detail with reference to FIG.

[0145] The controller 2100 includes a processor 2124. The processor 2124 may be a general purpose single or multi-chip microprocessor, a special purpose microprocessor, a microcontroller, a programmable gate array, etc. A processor is also referred to as a central processing unit (CPU). The processor 2124 may also be used with a combination of multiple processors.

[0146] The controller 2100 includes a memory 2122. The memory 2122 may be any electronic component capable of storing electronic information. The memory 2122 may also include a single memory as well as a combination of memories 2122.

[0147] According to various embodiments, data and instructions 2122a for providing blind spot images through the front pillars of the vehicle 2000 may be stored in the memory 2122. When the processor 2124 executes the instructions 2122a, all or a portion of the instructions 2122a and data 2122b required to execute the instructions may be loaded 2124a, 2124b onto the processor 2124.

[0148] The controller 2100 may include a transmitter 2130a, a receiver 2130b, or a transceiver 2130c to allow transmission and reception of signals. One or more antennas 2132a, 2132b may be electrically connected to the transmitter 2130a, the receiver 2130b, or each transceiver 2130c, and may additionally include an antenna.

[0149] The controller 2100 may include a digital signal processor (DSP) 2170. The DSP 2170 may enable the vehicle to quickly process digital signals.

[0150] The controller 2100 may include a communication interface 2180. The communication interface 2180 may include one or more ports and / or communication modules for connecting other devices to the controller 2100. The communication interface 2180 may allow a user and the controller 2100 to interact.

[0151] The various components of the controller 2100 may be connected via one or more buses 2190, which may include a power bus, a control signal bus, a status signal bus, a data bus, etc. Under the control of the processor 2124, the components may communicate information with each other via the bus 2190 to perform their intended functions.

[0152] The above-mentioned devices may be realized by hardware components, software components, and / or a combination of hardware and software components. For example, the devices and components described in the embodiments may be realized using one or more general-purpose or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or various devices capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications that run on the OS. The processing device may also respond to the execution of the software to access, record, manipulate, process, and generate data. For convenience of understanding, a single processing device may be described as being used, but one skilled in the art will understand that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, the processing device may include multiple processors or one processor and one controller. Other processing configurations, such as parallel processors, are also possible.

[0153] The software may include computer programs, codes, instructions, or a combination of one or more of these, and may configure a processing device to operate as desired or instruct the processing device, either independently or collectively. The software and / or data may be embodied in any type of machine, component, physical device, computer storage medium, or device to be interpreted by the processing device or to provide instructions or data to the processing device. The software may be distributed and stored or executed in a distributed manner on computer systems connected by a network. The software and data may be stored on one or more computer-readable storage media.

[0154] The method according to the embodiment may be realized in the form of program instructions executable by various computer means and recorded on a computer-readable medium. In this case, the medium may be one that continuously records a program executable by a computer, or one that temporarily records the program for execution or download. The medium may be one of various recording means or storage means in the form of a single or multiple hardware devices combined, and may be a medium directly connected to a certain computer system, or may be one that is distributed on a network. Examples of the medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and media configured to record program instructions such as ROMs, RAMs, flash memories, etc. Examples of other media include recording media and storage media managed by app stores that distribute applications, sites that supply or distribute various other software, and servers.

[0155] The various embodiments and terms used herein should not be understood to limit the technology described herein to a particular embodiment, but should be understood to include various modifications, equivalents, and / or substitutes of the embodiment. In connection with the description of the drawings, similar reference numerals have been used for similar components. A singular expression may include a plural expression unless the context clearly indicates otherwise. In this specification, expressions such as "A or B," "at least one of A and / or B," "A, B, or C," or "at least one of A, B, and / or C" may include all possible combinations of the listed items. Expressions such as "first," "second," "first," or "second" modify the corresponding component regardless of order or importance, and are used only to distinguish a certain component from other components, and do not limit the component. When a (e.g., first) component is described as being "(functionally or communicatively) coupled" or "connected" to another (e.g., second) component, the one component may be directly connected to the other component or may be connected through another component (e.g., a third component).

[0156] The term "module" as used herein includes a unit configured of hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integrally configured component, or a minimum unit or part thereof that performs one or more functions. For example, a module may be configured as an application-specific integrated circuit (ASIC).

[0157] According to various embodiments, each component (e.g., a module or program) of the described components may include one or more objects. According to various embodiments, one or more components or steps of the above-mentioned components may be omitted, or one or more different components or steps may be added. Alternatively or additionally, multiple components (e.g., modules or programs) may be integrated into one component. In such a case, the integrated component may perform one or more functions of each of the multiple components in a manner that is the same or similar to that performed by the corresponding component of the multiple components before integration. According to various embodiments, the steps performed by the modules, programs, or other components may be performed sequentially, in parallel, iteratively, or heuristically, one or more of the steps may be performed in a different order, omitted, or one or more different steps may be added. [Explanation of symbols]

[0158] 100:Electronic equipment 110: Input module 120: Sensor module 130: Communication module 140: Interface module 150: Audio output module 160: Main display module 163: First pillar display module 165: Second pillar display module 170: Camera module 180:Memory 190: Processor

Claims

1. An electronic device mounted on a vehicle, at least two camera modules each configured to capture an external image of the vehicle; Pillar display modules are respectively arranged on front pillars of the vehicle inside the vehicle; and a processor connected to the camera module and the pillar display module and configured to display at least one of the blind spot images blocked by the front pillar on at least one of the pillar display modules by utilizing at least one of the external images. Including, electronic equipment.

2. The processor, activating at least one of the pillar display modules in response to a first user input; configured to deactivate at least one of the pillar display modules in response to a second user input; 2. The electronic device of claim 1.

3. the first user input is for turning on one of a left turn signal light or a right turn signal light of the vehicle; the second user input is to turn off one of the left turn signal light or the right turn signal light.

3. The electronic device of claim 2.

4. The pillar display module includes: A first pillar display module disposed on a front pillar on the left side inside the vehicle; and a second pillar display module disposed on a front pillar on the right side inside the vehicle; Including, 3. The electronic device of claim 2.

5. The camera module includes: a front camera module configured to capture an image in front of the vehicle; a first side camera module configured to capture a first side image of a left side of the vehicle; and a second side camera module configured to capture a second side image of a right side of the vehicle; Including, 5. The electronic device of claim 4.

6. The processor, detecting a first blind spot image to be displayed on the first pillar display module by synthesizing the front image and the first side image; and detecting a second blind spot image for displaying the second blind spot image on the second pillar display module by synthesizing the front image and the second side image.

6. The electronic device of claim 5.

7. The camera module includes: a first camera module and a second camera module disposed on each of the front pillars outside the vehicle; 5. The electronic device of claim 4.

8. The processor, detecting a first blind spot image to be displayed on the first pillar display module from the image from the first camera module; configured to detect a second blind spot image from the image from the second camera module for display on the second pillar display module; 8. The electronic device of claim 7.

9. The processor, activating the first pillar indication module in response to a first user input for turning on a left turn signal light of the vehicle and deactivating the first pillar indication module in response to a second user input for turning off the left turn signal light of the vehicle; configured to activate the second pillar indication module in response to a first user input for turning on a right turn signal light of the vehicle, and to deactivate the second pillar indication module in response to a second user input for turning off the right turn signal light.

5. The electronic device of claim 4.

10. The processor, The blind spot image is detected from the external image based on setting information including a position and a size of the pillar display module in the vehicle.

2. The electronic device of claim 1.

11. A method for operating an electronic device mounted on a vehicle, comprising: detecting at least one of blind spot images obstructed by a front pillar of the vehicle using at least one external image of the vehicle; and displaying at least one of the blind spot images through at least one of the pillar display modules disposed on the front pillars, respectively; Including, A method for operating an electronic device.

12. activating at least one of the pillar display modules in response to the first user input. Further comprising: The steps of detecting at least one of the blind spot images and displaying at least one of the blind spot images include: repeated while at least one of the pillar display modules is active. A method for operating an electronic device according to claim 11.

13. deactivating at least one of the pillar display modules in response to the second user input. Further comprising: A method for operating an electronic device according to claim 12.

14. the first user input is for turning on one of a left turn signal light or a right turn signal light of the vehicle; the second user input is to turn off one of the left turn signal light or the right turn signal light. A method for operating an electronic device according to claim 13.

15. The pillar display module includes: a first pillar display module disposed on the front pillar on the left side inside the vehicle; and a second pillar display module disposed on the front pillar on the right side inside the vehicle; Including, A method for operating an electronic device according to claim 13.

16. The electronic device comprises: a front camera module configured to capture an image in front of the vehicle; a first side camera module configured to capture a first side image of a left side of the vehicle; and a second side camera module configured to capture a second side image of a right side of the vehicle; Including, A method for operating an electronic device according to claim 15.

17. The step of detecting at least one of the blind spot images includes: detecting a first blind spot image to be displayed on the first pillar display module by synthesizing the front image and the first side image; or detecting a second blind spot image to be displayed on the second pillar display module by synthesizing the front image and the second side image; At least one of:

20. A method for operating an electronic device according to claim 16.

18. The electronic device comprises: a first camera module and a second camera module disposed on each of the front pillars outside the vehicle; A method for operating an electronic device according to claim 15.

19. The step of detecting at least one of the blind spot images includes: detecting a first blind spot image to be displayed on the first pillar display module from an image from the first camera module; and detecting a second blind spot image to be displayed on the second pillar display module from the image from the second camera module; At least one of:

20. A method for operating an electronic device according to claim 18.

20. activating at least one of the pillar display modules includes: activating the first pillar indication module in response to a first user input for turning on a left turn signal light of the vehicle; or enabling the second pillar indicator module in response to a first user input for turning on a right turn signal light of the vehicle. Including, A method for operating an electronic device according to claim 15.

21. The step of deactivating at least one of the pillar display modules includes: deactivating the first pillar indication module in response to a second user input for turning off the left turn signal light; or deactivating the second pillar indication module in response to a second user input for turning off the right turn signal light. Including, 21. A method for operating an electronic device according to claim 20.

22. The step of detecting at least one of the blind spot images includes: Detecting at least one of the blind spot images from the external image based on setting information including a position and a size of the pillar display module in the vehicle. A method for operating an electronic device according to claim 11.

23. An electronic device mounted on a vehicle, a front camera module configured to capture a front image of the vehicle, a side camera module configured to capture a side image of the vehicle, and a rear camera module configured to capture a rear image of the vehicle; a pillar display module disposed on each of the front pillars of the vehicle inside the vehicle; a main display module mounted inside the vehicle separately from the pillar display module; and a processor connected to the front camera module, the side camera module, the rear camera module, the main display module, and the pillar display module, and configured to display a surround view image on the main display module using the front image, the side image, and the rear image, and to display at least one of the blind spot images blocked by the front pillar on at least one of the pillar display modules using at least one of the front image and the side image. Including, electronic equipment.

24. The processor, activating at least one of the pillar display modules in response to a first user input; configured to deactivate at least one of the pillar display modules in response to a second user input; 24. The electronic device of claim 23.

25. the first user input is for turning on one of a left turn signal light or a right turn signal light of the vehicle; the second user input is to turn off one of the left turn signal light or the right turn signal light.

25. The electronic device of claim 24.

26. The pillar display module includes: A first pillar display module disposed on a front pillar on the left side inside the vehicle; and a second pillar display module disposed on a front pillar on the right side inside the vehicle; Including, 25. The electronic device of claim 24.

27. The processor, activating the first pillar indication module in response to a first user input for turning on a left turn signal light of the vehicle and deactivating the first pillar indication module in response to a second user input for turning off the left turn signal light of the vehicle; configured to activate the second pillar indication module in response to a first user input for turning on a right turn signal light of the vehicle, and to deactivate the second pillar indication module in response to a second user input for turning off the right turn signal light.

26. The electronic device of claim 25.

28. The processor, The blind spot image is detected based on setting information including a position and a size of the pillar display module in the vehicle.

24. The electronic device of claim 23.

29. The processor, and configured to synthesize the surround view image by synthesizing the front image, the side image, and the rear image in response to a user input requesting the surround view image.

24. The electronic device of claim 23.

30. A method for operating an electronic device mounted on a vehicle, comprising: capturing a front image, a side image, and a rear image of the vehicle; displaying a surround view image on a main display module using the front image, the side image, and the rear image in response to a user input; and activating at least one of pillar display modules disposed on the front pillars in response to a user's input, and displaying at least one of the blind spot images blocked by the front pillars on at least one of the pillar display modules using at least one of the front image and the side view image; Including, A method for operating an electronic device.

Images