Electronic device for displaying surrounding images of vehicle in split screen and operating method of the same

Abstract

To provide an electronic device for displaying surrounding videos of a vehicle in a split screen and an operating method of the same.SOLUTION: In the present disclosure, the electronic device is configured to perform a first mode of displaying surrounding videos of the vehicle in a split screen through a digital rear mirror of the vehicle based on a motion related to stopping of the vehicle, and to perform a preset second mode through the digital rear mirror while driving the vehicle. According to some example embodiments, in the first mode, the electronic device may recognize a predetermined object in at least one of the surrounding videos, and may output information on the object while displaying the split screen through the digital rear mirror.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to an electronic device for displaying a peripheral image of a vehicle on a split screen and an operation method thereof.

Background Art

[0002] Generally, before starting a parked or stopped vehicle, a driver checks the surroundings of the vehicle. This is to prevent a collision between the starting vehicle and an object due to any sudden movement of the object in the surroundings of the vehicle. Specifically, the driver checks the front of the vehicle through the windshield, and checks the rear of the vehicle through the rear mirror and the side mirrors on both sides. That is, the driver checks the surroundings of the vehicle through the windshield, rear mirror, and side mirrors while moving the line of sight. However, the line of sight movement path for checking the surroundings of the vehicle is long, and there may be a sudden movement of an object in the surroundings of the vehicle while the driver is moving the line of sight. Therefore, even if the driver checks the surroundings of the vehicle, a collision may occur between the starting vehicle and the object.

Summary of the Invention

Problems to be Solved by the Invention

[0003] The present disclosure provides an electronic device for minimizing a driver's line of sight movement path for checking the surroundings of a parked or stopped vehicle and an operation method thereof.

[0004] The present disclosure provides an electronic device for displaying a peripheral image of a vehicle on a split screen and an operation method thereof.

Means for Solving the Problems

[0005] In the present disclosure, a method for operating an electronic device mounted on a vehicle may include a step of executing a first mode in which peripheral images of the vehicle are displayed in a split screen through a digital rear mirror (DRM) of the vehicle based on an operation related to the stop of the vehicle, and a step of executing a preset second mode through the digital rear mirror while the vehicle is running.

[0006] According to some embodiments, the step of executing the first mode may include a step of recognizing a predetermined object from at least one of the peripheral images, and a step of outputting information about the object while displaying a split screen on the digital rear mirror.

[0007] In the present disclosure, an electronic device mounted on a vehicle includes a memory and a processor configured to connect to the memory and execute at least one instruction stored in the memory. The processor is configured to execute a first mode in which peripheral images of the vehicle are displayed in a split screen through the digital rear mirror of the vehicle based on an operation related to the stop of the vehicle, and to execute a preset second mode through the digital rear mirror while the vehicle is running.

[0008] According to some embodiments, the processor may be configured to recognize a predetermined object from at least one of the peripheral images in the first mode and output information about the object while displaying a split screen on the digital rear mirror.

[0009] In the present disclosure, a video processing system mounted on a vehicle includes a plurality of camera devices configured to capture peripheral images of the vehicle respectively, an electronic device communicably connected to the camera devices and configured to process the peripheral images, and a digital rear mirror communicably connected to the electronic device and configured to display video information from the electronic device. The electronic device is configured to execute a first mode in which peripheral images of the vehicle are displayed in a split screen through the digital rear mirror of the vehicle based on an operation related to the stop of the vehicle, and to execute a preset second mode through the digital rear mirror while the vehicle is running.

[0010] According to some embodiments, in the first mode, the electronic device may be configured to recognize a predetermined object from at least one of the surrounding videos and output information about the object while displaying a split screen on the digital rearview mirror.

Advantages of the Invention

[0011] According to the present disclosure, the electronic device can display the surrounding video of the vehicle on the digital rearview mirror in a split screen based on an operation related to the stop for parking or halting of the vehicle. Therefore, the driver can check all the surrounding videos of the vehicle through the digital rearview mirror. As a result, the line-of-sight movement path of the driver for checking the surroundings of the vehicle can be minimized, and thus, the driver can effectively sense and quickly avoid any sudden movement of an object around the vehicle. According to some embodiments, the electronic device can recognize an object from at least one of the surrounding videos and output information about the object while displaying the split screen. Therefore, the driver can more effectively sense and quickly avoid any sudden movement of an object based on the information about the object. In this way, the electronic device can ensure the safety of not only the vehicle and the driver but also passengers other than the driver.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, various embodiments of the present specification will be described with reference to the accompanying drawings.

[0014] FIG. 1 is a block diagram showing the configuration of a video processing system 100 in various embodiments.

[0015] Referring to FIG. 1, the video processing system 100 may include at least two camera devices 110, a digital rearview mirror 120, and an electronic device 130. The video processing system 100 may be mounted on a vehicle. In some embodiments, at least one other component may be added to the video processing system 100.

[0016] The camera device 110 may respectively acquire video information regarding the surrounding environment of the vehicle. In some embodiments, the camera device 110 may store the video information in an internal memory. Specifically, the camera device 110 may respectively capture the surrounding video of the vehicle. The surrounding video may include at least two of the front video, the rear video, the left rear video, or the right rear video of the vehicle. For this purpose, the camera device 110 may be respectively attached at different positions of the vehicle.

[0017] In some embodiments, each camera device 110 may include at least one of one or more lenses, an image sensor, a flash, and an image signal processor (ISP). Some of the lenses may have the same lens characteristics (e.g., angle of view, focal length, autofocus, f number, or optical zoom). The lenses may include a light source lens or a telephoto lens. The image sensor may acquire an image corresponding to the subject by converting the light emitted or reflected from the subject and transmitted through one or more lenses into an electrical signal. For example, the image sensor may include one image sensor selected from image sensors with different attributes such as an RGB sensor, a BW (black and white) sensor, an IF sensor, or a UV sensor, a plurality of image sensors having the same attribute, or a plurality of image sensors having different attributes. Each image sensor may be realized by using, for example, a CCD (charged coupled device) sensor or a CMOS (complementary metal oxide semiconductor) sensor. The flash may include one or more light emitting diodes (e.g., RGB (red-green-blue) LED, white LED, infrared LED, or ultraviolet LED), or a xenon lamp.

[0018] The digital rearview mirror may provide a view of the vehicle's surrounding environment to the vehicle driver. For this purpose, the digital rearview mirror 120 may be disposed within the forward field of view of the vehicle driver. The digital rearview mirror 120 may have a mirror state or a display state. When in the mirror state, the digital rearview mirror 120 may directly project the rear view of the vehicle. When in the display state, the digital rearview mirror 120 may display a screen from the electronic device 130 to provide a view of at least a part of the vehicle's surrounding environment.

[0019] The electronic device 130 may generate a screen using at least a part of the surrounding video of the vehicle acquired through the camera device 110 and transmit the screen to the digital rearview mirror 120. At this time, the electronic device 130 may switch between a first mode and a second mode based on the state of the vehicle. For this purpose, the electronic device 130 may be respectively connected to the vehicle, the camera device 110, and the digital rearview mirror 120 in a wired or wireless communicable manner. Here, the electronic device 130 may be connected to the vehicle via the vehicle's internal network communication (for example, CAN communication).

[0020] Specifically, the electronic device 130 may execute the first mode based on operations related to the stop of the vehicle. For example, the electronic device 130 may sense operations related to the stop of the vehicle in the following cases: (i) immediately after the vehicle stops, (ii) when a preset time has elapsed since the vehicle stopped, (iii) when the vehicle starts moving after stopping, (iv) when the speed of the vehicle drops below a preset threshold (e.g., 10 km / h), or (v) when the speed of the vehicle rises above the threshold after dropping below the threshold. In response, the electronic device 130 may execute the first mode. In the first mode, the electronic device 130 may generate a split screen using the surrounding video of the vehicle. Here, the surrounding video may be arranged in a preset array on the split screen. In some embodiments, the electronic device 130 may recognize an object from at least one of the surrounding videos and cause information about the object to be output when the digital rearview mirror 120 displays the split screen. For example, the object may include at least one of a human, an object, an object approaching the vehicle, or an object moving away from the vehicle. Here, the information about the object may include at least one of the type, size, speed, or distance from the vehicle of the object.

[0021] On the other hand, the electronic device 130 may execute the second mode while the vehicle is in motion. For example, when the vehicle starts moving, when it reaches a preset speed (e.g., 10 km / h) after starting, or when a preset time has elapsed after the vehicle starts moving, the electronic device 130 may sense this as an operation related to the start of the vehicle's travel and, in response, execute the second mode. The second mode is preset and may be set, for example, by the user or by default. In the second mode, the electronic device 130 may cause the digital rearview mirror 120 to be driven in a mirror state or generate a screen using at least one surrounding video of the vehicle.

[0022] In some embodiments, each camera device 110 and the electronic device 130 may be connected via a communication cable. In one embodiment, the camera device 110 and the electronic device 130 may communicate in an analog manner. The analog manner may include, for example, AHD (analog high definition). In other embodiments, the camera device 110 and the electronic device 130 may communicate in a digital manner. For example, the digital manner may include a serial transmission manner. In such a case, the camera device 110 may include a serializer, and the electronic device 130 may include a deserializer. However, it should not be limited thereto, and the camera device 110 and the electronic device 130 may also be connected via an in-vehicle network communication (e.g., CAN communication). The camera device 110 and the electronic device 130 may include various communication chips.

[0023] In one embodiment, the digital rearview mirror 120 and the electronic device 130 may be integrally configured. In such a case, the electronic device 130 may be disposed within the forward field of view of the vehicle driver together with the digital rearview mirror 120. In other embodiments, the digital rearview mirror 120 and the electronic device 130 may be separately configured. In such a case, the digital rearview mirror 120 may be disposed within the forward field of view of the vehicle driver, and the electronic device 130 may be disposed at another position of the vehicle.

[0024] FIG. 2 is a flowchart showing the signal flow in the video processing system 100 in one embodiment. FIGS. 3A and 3B are exemplary diagrams for explaining the characteristics of the operation of the electronic device 130 in one embodiment.

[0025] Referring to FIG. 2, at stage 210, the electronic device 130 may sense an operation related to the stop of the vehicle. For example, the electronic device 130 may sense an operation related to the stop of the vehicle in the following cases: (i) immediately after the vehicle stops, (ii) when a preset time has elapsed since the vehicle stopped, (iii) when the vehicle starts moving after stopping, (iv) when the speed of the vehicle decreases below a preset threshold (e.g., 10 km / h), or (v) when the speed of the vehicle rises above the threshold after decreasing below the threshold. In response, the electronic device 130 may execute the first mode. Here, when the digital mirror 120 is in the mirror state, the electronic device 130 may cause the digital mirror 120 to execute in the display state, and when the digital mirror 120 is in the display state, the electronic device 130 may maintain the digital mirror 120. Therefore, the electronic device 130 can execute the first mode based on the operation related to the stop of the vehicle.

[0026] Next, at stage 220, the camera device 110 may capture the surrounding images of the vehicle respectively. The surrounding images may include at least two of the front image, the rear image, the left rear image, or the right rear image of the vehicle. Next, at stage 230, the camera device 110 may transmit the surrounding images to the electronic device 130 respectively. Therefore, the electronic device 130 can acquire the surrounding images through the camera device 110 in the first mode.

[0027] Next, at step 240, the electronic device 130 may generate a split screen 310 with peripheral video. In the first mode, the electronic device 130 may utilize the peripheral video of the vehicle to generate a split screen 310 as shown in FIG. 3a. Here, the peripheral video may be arranged on the split screen 310 in a preset array. Next, at step 250, the electronic device 130 may transmit the split screen 310 to the digital rearview mirror 120. Thereby, at step 260, the digital rearview mirror 120 may display the split screen 310. Therefore, in the first mode, the electronic device 130 can display the split screen 310 on the digital rearview mirror 120 as shown in FIG. 3a. In FIG. 3a, an example is shown where the rear left video, the front video, the directly rear video, and the rear right video of the vehicle are arranged from left to right on the split screen 310, but it is not limited thereto, and the split screen 310 may be configured with various combinations and various arrangements of the peripheral video.

[0028] On the other hand, at step 270, the electronic device 130 may sense the running of the vehicle. For example, when the vehicle starts, when it reaches a preset speed (e.g., 10 km / h) after starting, or when a preset time has elapsed after starting, the electronic device 130 senses this as an operation related to the start of the vehicle's running and may execute the second mode in response. The second mode is preset and may be set, for example, by the user or by default. Therefore, the electronic device 130 can execute the second mode during the running of the vehicle.

[0029] Next, at step 280, the electronic device 130 may execute the set mode, that is, the second mode. In the second mode, the electronic device 130 may cause the digital mirror 120 to execute in the mirror state. Alternatively, in the second mode, as shown in FIG. 3b, the electronic device 130 may display a screen 320 for at least one surrounding video of the vehicle on the digital mirror 120. Depending on the setting, the second mode may be the same as or different from the first mode. As an example, in the same manner as steps 220 to 260, the electronic device 130 may acquire all surrounding videos through the camera device 110 and display the surrounding videos on the digital mirror 120 in a split screen. As another example, the electronic device 130 may acquire a single surrounding video through one of the camera devices 110 and display the single surrounding video on a single screen through the digital mirror 120. As still another example, the electronic device 130 may acquire at least two surrounding videos through a part of the camera devices 110 and display the surrounding videos on the digital mirror 120 in a split screen. In FIG. 3b, an example in which the screen 320 is composed of a rear video of the vehicle is shown, but it is not limited thereto, and the screen 320 may be composed of various combinations and various arrangements of surrounding videos.

[0030] FIG. 4 is a flowchart showing the signal flow in the video processing system 100 in another embodiment. FIGS. 5a and 5b are exemplary diagrams for explaining the operation characteristics of the electronic device 130 in another embodiment.

[0031] Referring to FIG. 4, at step 410, the electronic device 130 may sense an operation related to the stop of the vehicle. For example, the electronic device 130 may sense an operation related to the stop of the vehicle in the following cases: (i) immediately after the vehicle stops, (ii) when a preset time has elapsed since the vehicle stopped, (iii) when the vehicle starts moving after stopping, (iv) when the speed of the vehicle decreases below a preset threshold (e.g., 10 km / h), or (v) when the speed of the vehicle rises above the threshold after decreasing below the threshold. In response to this, the electronic device 130 may execute the first mode. Here, when the digital rearview mirror 120 is in the mirror state, the electronic device 130 may cause the digital rearview mirror 120 to be executed in the display state, and when the digital rearview mirror 120 is in the display state, the electronic device 130 may maintain the digital rearview mirror 120. Therefore, the electronic device 130 can execute the first mode based on an operation related to the stop of the vehicle.

[0032] Next, at step 420, the camera device 110 may capture the surrounding images of the vehicle, respectively. The surrounding images may include at least two of the front image, the rear image, the left rear image, or the right rear image of the vehicle. Next, at step 430, the camera device 110 may transmit the surrounding images to the electronic device 130, respectively. Therefore, the electronic device 130 can acquire the surrounding images through the camera device 110 in the first mode.

[0033] Next, at step 441, the electronic device 130 may recognize an object from at least one of the surrounding images. For example, the object may include at least one of a human, an object, an object approaching a vehicle, or an object leaving a vehicle. At this time, the electronic device 130 may check information regarding the object. Here, the information regarding the object may include at least one of the type, size, speed, or distance from the vehicle of the object. Next, at step 443, the electronic device 130 may generate a split screen 510 with the surrounding images. In the first mode, the electronic device 130 may use the surrounding images of the vehicle to generate a split screen 510 as shown in FIG. 5a. Here, the surrounding images may be arranged in a preset array on the split screen 510. As an example, the electronic device 130 may represent an indicator indicating information regarding the object within the split screen 510. Next, at step 450, the electronic device 130 may transmit the split screen 510 to the digital rearview mirror 120. Thereby, at step 460, the digital rearview mirror 120 may display the split screen 510. Therefore, in the first mode, as shown in FIG. 5a, the electronic device 130 can display the split screen 510 on the digital rearview mirror 120. As an example, by displaying the indicator within the split screen 510, the electronic device 130 may display information regarding the object on the digital rearview mirror 120. As another example, the electronic device 130 may output information regarding the object as an audio signal by itself or through the digital rearview mirror 120. As still another example, while displaying information regarding the object on the digital rearview mirror 120, the electronic device 130 may output information regarding the object as an audio signal by itself or through the digital rearview mirror 120.

[0034] On the one hand, at stage 470, the electronic device 130 may sense the running of the vehicle. For example, when the vehicle starts, when it reaches a preset speed (e.g., 10 km / h) after starting, or when a preset time has elapsed after the vehicle starts, the electronic device 130 senses this as an operation related to the start of the vehicle's running and may react to execute the second mode accordingly. The second mode is preset and may be set, for example, by the user or by default. Therefore, the electronic device 130 can execute the second mode during the running of the vehicle.

[0035] Next, at stage 480, the electronic device 130 may execute the set mode, that is, the second mode. In the second mode, the electronic device 130 may cause the digital rearview mirror 120 to execute in a mirror state. Or, in the second mode, the electronic device 130 may display a screen 520 for at least one peripheral video of the vehicle on the digital rearview mirror 120 as shown in FIG. 5b. Depending on the setting, the second mode may be the same as or different from the first mode. As an example, in the same manner as in stages 220 - 260, the electronic device 130 may acquire all peripheral videos through the camera device 110 and display the peripheral videos on the digital rearview mirror 120 in a split screen. As another example, the electronic device 130 may acquire a single peripheral video through one of the camera devices 110 and display the single peripheral video on the digital rearview mirror 120 in a single screen. As yet another example, the electronic device 130 may acquire at least two peripheral videos through a part of the camera devices 110 and display the peripheral videos on the digital rearview mirror 120 in a split screen.

[0036] FIG. 6 is a block diagram showing the configuration of the electronic device 130 in various embodiments.

[0037] Referring to FIG. 6, the electronic device 130 may include at least one of a communication module 610, an input module 620, a sensor module 630, an interface module 640, an audio output module 650, a memory 660, or a processor 670. The electronic device 130 may be attached to a vehicle together with the camera device 110 and the digital rearview mirror 120, and may be communicably connected to the vehicle, the camera device 110, and the digital rearview mirror 120, either wired or wirelessly. In some embodiments, at least one of the components of the electronic device 130 (e.g., the communication module, the sensor module) may be omitted, and at least one other component (e.g., a display module) may be added. In some embodiments, at least two of the components of the electronic device 130 may be implemented as one integrated circuit. In one embodiment, the components of the electronic device 130 may be integrated and configured as a single unit. In such a case, the components of the electronic device 130 may be housed within a single housing. In other embodiments, the components of the electronic device 130 may be configured to be distributed among at least two units.

[0038] The communication module 610 may execute communication between the electronic device 130 and an external device. The communication module 610 may establish a communication channel with the external device and execute communication with the external device via the communication channel. Here, the external device may include at least one of a satellite, a base station, a server, or another electronic device (for example, used by a driver). The communication module 610 may include at least one of a wired communication module or a wireless communication module. The wired communication module may be connected to the external device by wire and communicate by wire. The wireless communication module may include at least one of a short-range communication module or a long-range communication module. The short-range communication module may communicate with the external device in a short-range communication manner. For example, the short-range communication manner may include at least one of Bluetooth (registered trademark), Wi-Fi direct, NFC communication (Near Field Communication), or infrared communication (IrDA: infrared data association). The long-range communication module may communicate with the external device in a long-range communication manner. Here, the long-range communication module may communicate with the external device via a network. For example, the network may include at least one of a cellular network, the Internet, or a computer network such as a LAN (local area network) or a WAN (wide area network).

[0039] The input module 620 may input signals used for at least one component of the electronic device 130. The input module 630 may include, for example, at least one of at least one button (also called a key), a keyboard, a keypad, a mouse, a joystick, or a microphone. Here, the button may include at least one of a physical button or a touch button. In some embodiments, the input module 620 may include at least one of a touch circuit configured to sense a touch or a sensor circuit configured to measure the intensity of a force generated by the touch. In some embodiments, when the electronic device 130 is integrally configured with the digital mirror 120, the touch circuit or the sensor may be combined with the digital mirror 120 to implement a touch screen.

[0040] The sensor module 630 may generate an electrical signal or a data value corresponding to an operating state inside the electronic device 130 (e.g., power or temperature) or an external environmental state. The sensor module 630 may include, for example, at least one of a GPS (global positioning system) sensor, a motion sensor (also called a gesture sensor), a proximity sensor, a touch sensor, a radar sensor, a LIDAR sensor, a movement sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor (e.g., a G sensor), an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0041] According to various embodiments, at least one of the communication module 610, the input module 620, or the sensor module 630 may generate a user input. In one embodiment, any sensor of the input module 620 or the sensor module 630 may generate a user input based on a signal directly input from a user. In other embodiments, the communication module 610 may generate a user input based on a signal input from an electronic device used by the user.

[0042] The interface module 640 may be provided for an interface between the electronic device 130 and an external device. Specifically, the interface module 640 may support a specified protocol for connecting to the external device either wired or wirelessly. Here, the external device may include at least one of a vehicle, a camera device 110, or a digital rearview mirror 120. In one embodiment, when communicating with each camera device 110 in an analog manner, the interface module 640 may receive video information from the camera device 110 and convert it from an analog signal to digital data. In other embodiments, when communicating with the camera device 110 in a digital manner, the interface module 640 may receive video information from the camera device 110 and convert it from serial data to parallel data. In such a case, the interface module 640 may be implemented as a deserialiser.

[0043] The audio output module 650 may output an audio signal to the outside of the electronic device 130. For example, the audio output module 650 may include at least one of a speaker or a receiver. In one embodiment, it may include at least one voice coil that provides vibration by a diaphragm in the speaker, and a magnet that forms a magnetic field. When a current flows through the voice coil, the magnetic field formed by the voice coil may interact with the magnetic field formed by the magnet to vibrate the voice coil. The diaphragm connected to the voice coil may vibrate based on the vibration of the voice coil. The speaker may output an audio signal based on the vibration of the diaphragm.

[0044] The memory 660 may store various data used by at least one component of the electronic device 130. For example, the memory 660 may include at least one of a volatile memory or a non-volatile memory. The data may include at least one program and related input data or output data. The program may be stored in the memory 660 as software including at least one instruction, and may include, for example, at least one of an operating system, middleware, or an application. The memory 660 may include at least one of a first memory built into the electronic device 130 or a second memory removable from the electronic device 130.

[0045] Processor 670 may control at least one component of the electronic device 130. Thereby, processor 670 may execute data processing or operations. For example, the hardware for processing data may include an ALU (arithmetic and logic unit), an FPU (floating point unit), an FPGA (field programmable gate array), a CPU (central processing unit), and / or an AP (application processor). Processor 670 may have a single-core processor structure or a multi-core processor structure such as a dual core, a quad core, a hexa core, or an octa core. Processor 670 may execute the instruction words stored in the memory 660. According to various embodiments, processor 670 may generate a screen using at least a part of the surrounding video of the vehicle acquired through the camera device 110 and transmit the screen to the digital rearview mirror 120. At this time, processor 670 may switch between the first mode and the second mode based on the state of the vehicle.

[0046] Specifically, the processor 670 may execute the first mode based on operations related to the vehicle's stop. For example, the processor 670 may sense operations related to the vehicle's stop in the following cases: (i) immediately after the vehicle stops, (ii) when a preset time has elapsed since the vehicle stopped, (iii) when the vehicle starts moving after stopping, (iv) when the vehicle's speed drops below a preset threshold (e.g., 10 km / h), or (v) when the vehicle's speed rises above the threshold after dropping below the threshold. In response, the processor 670 may execute the first mode. In the first mode, the processor 670 may generate a split screen using the vehicle's surrounding video. Here, the surrounding video may be arranged in a preset array on the split screen. In some embodiments, the processor 670 may recognize an object from at least one of the surrounding videos and output information about the object when the digital rearview mirror 120 displays the split screen. For example, the object may include at least one of a human, an object, an object approaching the vehicle, or an object moving away from the vehicle. Here, the information about the object may include at least one of the object's type, size, speed, or distance from the vehicle.

[0047] On the other hand, the processor 670 may execute the second mode while the vehicle is in motion. For example, when the vehicle starts moving, when it reaches a preset speed (e.g., 10 km / h) after starting, or when a preset time has elapsed after the vehicle starts moving, the processor 670 may sense this as an operation related to the start of the vehicle's travel and, in response, execute the second mode. The second mode is preset and may be set, for example, by the user or by default. In the second mode, the processor 670 may cause the digital rearview mirror 120 to operate in a mirror state or generate a screen using at least one of the vehicle's surrounding videos.

[0048] FIG. 7 is a flowchart showing a method of operating the electronic device 130 in various embodiments.

[0049] Referring to FIG. 7, at step 710, the electronic device 130 may sense that the vehicle engine is on. At this time, the interface module 640 may be communicably connected to the vehicle by wire or wirelessly. Here, the interface module 640 may be connected to the vehicle via the vehicle's internal network communication (e.g., CAN communication). Accordingly, the processor 670 can sense that the vehicle engine is on through the interface module 640. When the vehicle engine is on, the processor 670 may continuously receive the vehicle's status information through the interface module 640. Here, the status information may include at least one of the vehicle's speed information, brake pedal operation information, accelerator pedal operation information, or vehicle position information.

[0050] Next, at step 720, the electronic device 130 may sense an operation related to the vehicle's stop. Specifically, the processor 670 may use the vehicle's status information to sense an operation related to the vehicle's stop. Here, the operation related to the vehicle's stop may indicate the vehicle's substantial stop or may not indicate the vehicle's substantial stop. For example, the processor 670 may sense an operation related to the vehicle's stop in the following cases: (i) immediately after the vehicle stops, (ii) when a preset time has elapsed since the vehicle stopped, (iii) when the vehicle starts after stopping, (iv) when the vehicle's speed drops below a preset threshold (e.g., 10 km / h), or (v) when the vehicle's speed rises above the threshold after dropping below the threshold. In response to this, at step 730, the electronic device 130 may execute the first mode. Specifically, in the first mode, the processor 670 may display the vehicle's surrounding video on the split screens 310 and 510 of the digital rearview mirror 120. This will be described in more detail with reference to FIGS. 8 and 9. The processor 670 may repeatedly execute the first mode until the vehicle engine stop is sensed at step 740 or until the vehicle's driving is sensed at step 750.

[0051] FIG. 8 is a flowchart showing the stage (stage 730) of executing the first mode of the electronic device 130 in one embodiment.

[0052] Referring to FIG. 8, at step 810, the electronic device 130 may determine whether the digital mirror 120 is in the mirror state. Specifically, the processor 130 may determine whether the digital mirror 120 is in the mirror state or the display state. Next, if it is determined that the digital mirror 120 is in the mirror state, at step 820, the electronic device 130 may switch the digital mirror 120 to the display state. At this time, the interface module 640 may be respectively connected to the digital mirror 120 in a wired or wirelessly communicable manner. Specifically, when the digital mirror 120 is in the mirror state, the processor 670 may cause the digital mirror 120 to execute in the display state through the interface module 640, and when the digital mirror 120 is in the display state, the processor 670 may maintain the digital mirror 120.

[0053] Next, at step 830, the electronic device 130 may acquire the surrounding video of the vehicle. At this time, the interface module 640 may be respectively connected to the camera device 110 in a wired or wirelessly communicable manner. The camera device 110 may respectively capture the surrounding video of the vehicle. The surrounding video may include at least two of the front video, the rear video, the left rear video, or the right rear video of the vehicle. After that, the camera device 110 may respectively transmit the surrounding video to the electronic device 130. Therefore, the processor 670 can respectively receive the surrounding video from the camera device 110 through the interface module 640.

[0054] Next, at step 840, the electronic device 130 may generate a split screen 310 with surrounding video. Specifically, the processor 670 may utilize the surrounding video of the vehicle to generate a split screen 310 as shown in FIG. 3a. Here, the surrounding video may be arranged in a preset array on the split screen 310. In FIG. 3a, an example is shown where the rear left video, front video, rear video, and rear right video of the vehicle are arranged from left to right on the split screen 310, but it is not limited thereto, and the split screen 310 may be configured with various combinations and various arrays of the surrounding video.

[0055] Next, at step 850, the electronic device 130 may display the split screen 310 on the digital rearview mirror 120. Specifically, the processor 670 may transmit the split screen 310 to the digital rearview mirror 120 through the interface module 640. Thereby, the digital rearview mirror 120 can display the split screen 310 as shown in FIG. 3a in the display state. Optionally or additionally, during the display of the split screen 310, the processor 670 may partially enlarge the split screen 310. As an example, the processor 670 may select one of the surrounding videos on the split screen 310 based on a user input and enlarge the selected surrounding video. As another example, the processor 670 may select a partial area on the split screen 310 based on a user input and enlarge the selected area. Here, the processor 670 may sense the user input through at least one of the communication module 610, the input module 620, or the sensor module 630.

[0056] FIG. 9 is a flowchart showing the steps (step 730) of executing the first mode of the electronic device 130 in another embodiment.

[0057] Referring to FIG. 9, at step 910, the electronic device 130 may determine whether the digital mirror 120 is in the mirror state. Specifically, the processor 130 may determine whether the digital mirror 120 is in the mirror state or the display state. Next, if it is determined that the digital mirror 120 is in the mirror state, at step 920, the electronic device 130 may switch the digital mirror 120 to the display state. At this time, the interface module 640 may be respectively connected to the digital mirror 120 in a wired or wirelessly communicable manner. Specifically, when the digital mirror 120 is in the mirror state, the processor 670 may cause the digital mirror 120 to execute in the display state through the interface module 640, and when the digital mirror 120 is in the display state, the processor 670 may maintain the digital mirror 120.

[0058] Next, at step 930, the electronic device 130 may acquire the surrounding video of the vehicle. At this time, the interface module 640 may be respectively connected to the camera device 110 in a wired or wirelessly communicable manner. The camera device 110 may respectively capture the surrounding video of the vehicle. The surrounding video may include at least two of the front video of the vehicle, the rear video, the left rear video, or the right rear video. After that, the camera device 110 may respectively transmit the surrounding video to the electronic device 130. Therefore, the processor 670 can respectively receive the surrounding video from the camera device 110 through the interface module 640.

[0059] Next, at step 941, the electronic device 130 may recognize an object from at least one of the surrounding videos. Specifically, the processor 670 may be equipped with an AI (artificial intelligence) algorithm for object recognition, and may use such an AI algorithm to recognize an object from at least one of the surrounding videos. For example, the object may include at least one of a human, an object, an object approaching a vehicle, or an object leaving a vehicle. At this time, the processor 670 may check information about the object. Here, the information about the object may include at least one of the type, size, speed, or distance from the vehicle of the object.

[0060] Next, at step 943, the electronic device 130 may generate a split screen 510 with the surrounding videos. Specifically, the processor 670 may use the surrounding videos of the vehicle to generate a split screen 510 as shown in FIG. 5a. Here, the surrounding videos may be arranged in a preset array on the split screen 510. FIG. 5a shows an example in which the rear left video, the front video, the rear video, and the rear right video of the vehicle are arranged from left to right on the split screen 510, but it is not limited thereto, and the split screen 510 may be composed of various combinations and various arrays of the surrounding videos. As an example, the processor 670 may represent an indicator indicating information about the object within the split screen 510. Here, the indicator may be represented by at least one of text, a frame, a color, a shape, or a symbol.

[0061] Next, at step 950, the electronic device 130 may display the split screen 510 on the digital micromirror 120. Specifically, the processor 670 may transmit the split screen 510 to the digital micromirror 120 through the interface module 640. Thereby, the digital micromirror 120 can display the split screen 510 in the display state as shown in FIG. 5a. As an example, by displaying an indicator within the split screen 510, the processor 670 may display information about the object on the digital micromirror 120. As another example, the processor 670 may output information about the object as an audio signal through the audio output module 650 or the digital micromirror 120. As still another example, the processor 670 may output information about the object as an audio signal through the audio output module 650 or the digital micromirror 120 while displaying information about the object through the digital micromirror 120. Optionally or additionally, during the display of the split screen 510, the processor 670 may partially enlarge the split screen 510. As an example, the processor 670 may select one of the peripheral videos from the split screen 510 based on user input and enlarge the selected peripheral video. As another example, the processor 670 may select a partial area of the split screen 510 based on user input and enlarge the selected area. Here, the processor 670 may sense user input through at least one of the communication module 610, the input module 620, or the sensor module 630.

[0062] Referring to FIG. 7 again, at step 740, the electronic device 130 may sense the stop of the vehicle engine. Specifically, while executing the first mode at step 730, at step 740, the processor 670 may sense the stop of the vehicle engine. The processor 670 may sense the stop of the vehicle engine through the interface module 640. In such a case, the electronic device 130 may end the first mode. On the other hand, if the stop of the vehicle engine is not sensed at step 740, the electronic device 130 may return to step 720.

[0063] On the other hand, at step 750, the electronic device 130 may sense the running of the vehicle. That is, when the engine of the vehicle is started and no operation related to the stop of the vehicle is sensed at step 720, the running of the vehicle may be sensed at step 730. Specifically, the processor 670 may sense the running of the vehicle by using the state information of the vehicle. Here, the running of the vehicle may indicate the substantial, i.e., continuous running of the vehicle, or may also indicate the discontinuous running including the temporary stop of the vehicle. For example, when the vehicle starts, when the vehicle reaches a preset speed (e.g., 10 km / h) after starting, or when a preset time elapses after the vehicle starts, the processor 670 may sense this as an operation related to the start of the running of the vehicle and may execute the second mode in response thereto.

[0064] In response thereto, at step 760, the electronic device 130 may execute the second mode. The second mode is preset and may be set, for example, by the user or by default. Specifically, in the second mode, the processor 670 may cause the digital rearview mirror 120 to execute in the mirror state. Or, in the second mode, the processor 670 may display a screen 520 of at least one peripheral video of the vehicle on the digital rearview mirror 120 as shown in FIG. 5b. Depending on the setting, the second mode may be the same as or different from the first mode. This will be described in more detail with reference to FIG. 10. The processor 670 may repeatedly execute the second mode until an operation related to the stop of the vehicle is sensed at step 720.

[0065] FIG. 10 is a flowchart showing the steps (step 760) of executing the second mode of the electronic device 130 in various embodiments.

[0066] Referring to FIG. 10, at step 1010, the electronic device 130 may check the second mode. Specifically, the processor 670 may check the content set as the second mode.

[0067] Next, at step 1020, the electronic device 130 may determine whether the second mode is set to the mirror mode. Specifically, the processor 670 may check whether the second mode is set to the mirror mode or a specific display mode. Next, if it is determined that the second mode is set to the mirror mode, at step 1030, the electronic device 130 may switch the digital rearview mirror 120 to the mirror state. Specifically, if the second mode is set to the mirror mode, the processor 670 may cause the digital rearview mirror 120 to operate in the mirror state through the interface module 640. Therefore, when the second mode is set to the mirror mode, the digital rearview mirror 120 can directly display the rear view of the vehicle in the mirror state.

[0068] On the other hand, if it is determined that the second mode is not set to the mirror mode, at step 1040, the electronic device 130 may acquire at least one peripheral video of the vehicle. Specifically, if the second mode is set to the display mode, the processor 670 may maintain the digital rearview mirror 120. After that, the processor 670 may acquire at least one peripheral video through the interface module 640. At this time, the processor 670 may select at least one of the camera devices 110 as set in the second mode and acquire at least one peripheral video through at least one camera device 110. The at least one peripheral video may be at least one of a front video, a directly rear video, a left rear video, or a right rear video of the vehicle. After that, the at least one camera device 110 may transmit the at least one peripheral video to the electronic device 130. Therefore, the processor 670 can receive the at least one peripheral video from the at least one camera device 110 through the interface module 640.

[0069] Next, at stage 1050, the electronic device 130 may display at least one peripheral video on the digital micromirror 120. Specifically, the processor 670 may generate screens 320 and 520 as shown in FIGS. 3b and 5b using at least one peripheral video. Next, the processor 670 may transmit the screens 320 and 520 to the digital micromirror 120 through the interface module 640. As a result, the digital micromirror 120 can display the screens 320 and 520 as shown in FIGS. 3b and 5b in the display state. FIGS. 3b and 5b show an example in which the screens 320 and 520 are composed of one peripheral video, but it should not be limited thereto, and the screens 320 and 520 may be composed of various combinations and various arrangements of peripheral videos. Optionally or additionally, during the display of the screens 320 and 520, the processor 670 may partially enlarge the screens 320 and 520. As an example, the processor 670 may select one of the peripheral videos from the screens 320 and 520 based on a user input and enlarge the selected peripheral video. As another example, the processor 670 may select a partial area of the screens 320 and 520 based on a user input and enlarge the selected area. Here, the processor 670 may sense a user input through at least one of the communication module 610, the input module 620, or the sensor module 630.

[0070] As an example, when the second mode is the same as the first mode, the processor 670 may acquire all peripheral videos through the camera device 110 and display the peripheral videos on the digital micromirror 120 in a split screen. As another example, the processor 670 may acquire a single peripheral video through one of the camera devices 110 and display the single peripheral video on a single screen through the digital micromirror 120. As still another example, the processor 670 may acquire at least two peripheral videos through a part of the camera device 110 and display the peripheral videos on the digital micromirror 120 in a split screen.

[0071] According to the present disclosure, the electronic device 130 can display the surrounding video of the vehicle on the split screens 310 and 510 of the digital rearview mirror 120 based on an operation related to stopping the vehicle for parking or halting. Accordingly, the driver can check all the surrounding videos of the vehicle through the digital rearview mirror 120. This can minimize the driver's line-of-sight movement path for checking the surroundings of the vehicle, and as a result, the driver can effectively sense a sudden movement of any object around the vehicle and quickly avoid it. According to some embodiments, the electronic device 130 can recognize an object from at least one of the surrounding videos and output information about the object while displaying the split screens 310 and 510. Accordingly, the driver can more effectively sense a sudden movement of any object and quickly avoid it based on the information about the object. In this way, the electronic device 130 can ensure the safety of not only the vehicle and the driver but also passengers other than the driver.

[0072] In short, the present disclosure provides an electronic device 100 and an operating method thereof for displaying the surrounding video of a vehicle on split screens 310 and 510.

[0073] In the present disclosure, an operating method of an electronic device 130 mounted on a vehicle may include, based on an operation related to stopping the vehicle (step 720), executing a first mode of displaying the surrounding video of the vehicle on split screens 310 and 510 through the digital rearview mirror 120 of the vehicle (step 730), and executing a preset second mode through the digital rearview mirror 120 while the vehicle is running (step 750) (step 760).

[0074] According to various embodiments, the surrounding video may include at least two of a front video of the vehicle, a rear video, a left rear video, or a right rear video.

[0075] According to various embodiments, the step of executing the first mode (step 730) may be executed when, immediately after the vehicle stops, a preset time has elapsed since the vehicle stopped, when the vehicle starts after stopping, when the speed of the vehicle decreases below a preset threshold, or when the speed of the vehicle rises above the threshold after decreasing below the threshold.

[0076] According to various embodiments, the step of executing the first mode (step 730) may include the steps of respectively acquiring surrounding images through a plurality of camera devices 110 (steps 830, 930), arranging the surrounding images in a preset array to generate split screens 310, 510 (steps 840, 943), and displaying the split screens 310, 510 on the digital rearview mirror 120 (steps 850, 950).

[0077] According to various embodiments, the step of executing the first mode (step 730) may be executed after switching the digital rearview mirror 120 to the display state (steps 820, 920) if the digital rearview mirror 120 is in the mirror state (steps 810, 910).

[0078] According to various embodiments, the step of executing the second mode (step 760) may be executed when the vehicle starts, when the vehicle reaches a preset speed after starting, or when a preset time has elapsed after the vehicle starts.

[0079] According to various embodiments, the step of executing the second mode (step 760) may include the step of causing the digital rearview mirror 120 to execute in the mirror state (step 1030).

[0080] According to various embodiments, the step of executing the second mode (step 760) may include the steps of displaying at least one surrounding image of the vehicle on the digital rearview mirror 120 (steps 1040 and 1050).

[0081] According to various embodiments, the method of operating the electronic device 130 may further include determining a stop of the vehicle and a running of the vehicle by using state information of the vehicle, and the state information may include at least one of speed information, operation information of a brake pedal, operation information of an accelerator pedal, or position information.

[0082] According to some embodiments, the step of executing the first mode (step 730) may include recognizing a predetermined object from at least one of surrounding images (step 941), and outputting information about the object while displaying the split screen 510 on the digital mirror 120 (step 950).

[0083] According to some embodiments, the step of outputting information about the object (step 950) may include at least one of displaying an indicator indicating information about the object within the split screen 510, or outputting an audio signal indicating information about the object.

[0084] According to some embodiments, the object may include at least one of a human, an object, an object approaching the vehicle, or an object leaving the vehicle.

[0085] According to some embodiments, the information about the object may include at least one of the type, size, speed, or distance from the vehicle of the object.

[0086] In the present disclosure, an electronic device 130 mounted on a vehicle includes a memory 660 and a processor 670 connected to the memory 660 and configured to execute at least one instruction stored in the memory 660. The processor 670 may execute a first mode of displaying surrounding images of the vehicle on split screens 310 and 510 through a digital mirror 120 of the vehicle based on an operation related to a stop of the vehicle, and may be configured to execute a preset second mode through the digital mirror 120 while the vehicle is running.

[0087] According to various embodiments, the surrounding video may include at least two of the front video, rear video, left rear video, or right rear video of the vehicle.

[0088] According to various embodiments, the processor 670 may be configured to execute the first mode when the vehicle has just stopped, when a preset time has elapsed since the vehicle stopped, when the vehicle starts after stopping, when the speed of the vehicle decreases below a preset threshold, or when the speed of the vehicle rises above the threshold after decreasing below the threshold.

[0089] According to various embodiments, in the first mode, the processor 670 may be configured to respectively obtain surrounding videos through a plurality of camera devices 110, arrange the surrounding videos in a preset array to generate split screens 310 and 510, and display the split screens 310 and 510 on the digital rearview mirror 120.

[0090] According to various embodiments, when the digital rearview mirror 120 is in the mirror state, the processor 670 may be configured to execute the first mode after switching the digital rearview mirror 120 to the display state.

[0091] According to various embodiments, when the vehicle starts, when the vehicle reaches a preset speed after starting, or when a preset time has elapsed after the vehicle starts, the processor 670 may be configured to execute the second mode.

[0092] According to various embodiments, in the second mode, the processor 670 may be configured to execute the digital rearview mirror 120 in the mirror state.

[0093] According to various embodiments, in the second mode, the processor 670 may be configured to display at least one surrounding video of the vehicle on the digital rearview mirror 120.

[0094] According to various embodiments, the processor 670 is configured to utilize the state information of the vehicle to determine the stop and driving of the vehicle, and the state information may include at least one of speed information, brake pedal operation information, accelerator pedal operation information, or position information.

[0095] According to some embodiments, in the first mode, the processor 670 may be configured to recognize a predetermined object from at least one of the surrounding videos and output information about the object while displaying the split screen 510 on the digital mirror 120.

[0096] According to some embodiments, the processor 670 may be configured to perform at least one of displaying an indicator indicating information about the object within the split screen 510 or outputting an audio signal indicating information about the object.

[0097] According to some embodiments, the object may include at least one of a human, an object, an object approaching the vehicle, or an object leaving the vehicle.

[0098] According to some embodiments, the information about the object may include at least one of the type, size, speed, or distance from the vehicle of the object.

[0099] In the present disclosure, a video processing system 100 mounted on a vehicle includes a plurality of camera devices 110 configured to respectively capture the surrounding videos of the vehicle, an electronic device 130 communicably connected to the camera devices 110 and configured to process the surrounding videos, and a digital mirror 120 communicably connected to the electronic device 130 and configured to display the video information from the electronic device 130. The electronic device 130 is configured to execute a first mode of displaying the surrounding videos of the vehicle on the split screens 310 and 510 through the digital mirror 120 of the vehicle based on an operation related to the stop of the vehicle, and may be configured to execute a preset second mode through the digital mirror 120 during the driving of the vehicle.

[0100] According to some embodiments, in the first mode, the electronic device 130 may be configured to recognize a predetermined object from at least one of the surrounding videos and output information about the object while displaying the split screen 510 on the digital mirror 120.

[0101] In one embodiment, the electronic device 130 and the digital mirror 120 may be integrally configured and disposed within the forward field of view of the vehicle driver.

[0102] In other embodiments, the electronic device 130 and the digital mirror 120 may be separately configured, and the digital mirror 120 may be disposed within the forward field of view of the vehicle driver.

[0103] FIG. 11 is a block diagram showing a vehicle 2000 equipped with the video processing system 100 in various embodiments. FIG. 12 is a block diagram showing the control device 2100 of the vehicle of FIG. 11.

[0104] Referring to FIGS. 11 and 12, the video processing system 100 according to various embodiments may be mounted on the vehicle 2000, and the vehicle 2000 may include a control device 2100. At this time, the vehicle 2000 may be an autonomous vehicle. In some embodiments, at least one component of the video processing system 100 may be integrated with at least one component of the control device 2100.

[0105] The control device 2100 may include a controller 2120 including a memory 2122 and a processor 2124, a sensor 2110, a wireless communication device 2130, a LIDAR 2140, and a camera module 2150.

[0106] The controller 2120 may be configured when manufactured by the vehicle manufacturer, or may be additionally configured to perform the autonomous driving function after manufacture. Or, a configuration for performing continuous additional functions may be included by upgrading the controller 2120 configured at the time of manufacture.

[0107] The controller 2120 may transmit control signals to the sensor 2110, the engine 2006, the user interface (UI) 2008, the wireless communication device 2130, the LIDAR 2140, and the camera module 2150, which are included as other components within the vehicle. Also, although not shown in the figure, control signals may also be transmitted to an acceleration device, a braking system, a steering device, or a navigation device related to the running of the vehicle.

[0108] The controller 2120 may control the engine 2006. For example, when the vehicle 2000 senses the speed limit of the road on which it is running, the controller 2120 may control the engine 2006 so that the running speed does not exceed the speed limit, or may control the engine 2006 so that the running speed of the vehicle 2000 accelerates within a range not exceeding the speed limit. Additionally, when the environmental sensing modules 2004a, 2004b, 2004c, 2004d sense the environment outside the vehicle and transmit it to the sensor 2110, the controller 2120 may receive this and generate a signal for controlling the engine 2006 or a steering device (not shown) to control the running of the vehicle.

[0109] When there is another vehicle or an obstacle in front of the vehicle, the controller 2120 may control the engine 2006 or the braking system so that the running vehicle decelerates. In addition to the speed, the trajectory, the running route, and the steering angle may also be controlled. Alternatively, the controller 2120 may generate necessary control signals according to recognition information of other external environments such as the running lane or the running signal of the vehicle to control the running of the vehicle.

[0110] In addition to generating its own control signals, the controller 2120 can also control the running of the vehicle by executing communication with surrounding vehicles or a central server and transmitting instructions for controlling peripheral devices based on the received information.

[0111] In addition, when the position of the camera module 2150 is changed or the viewing angle is changed, the controller 2120 may not be able to accurately recognize the vehicle or the lane. Therefore, in order to prevent this, a control signal may be generated to control the execution of calibration of the camera module 2150. Accordingly, the controller 2120 causes the camera module 2150 to generate a calibration control signal, so that even if the mounting position of the camera module 2150 is changed due to vibrations or impacts generated by the movement of the autonomous vehicle 2000, the normal mounting position, direction, viewing angle, etc. of the camera module 2150 can be continuously maintained. The controller 2120 may generate a control signal to execute calibration of the camera module 2120 when the initial mounting position, direction, viewing angle information of the camera module 2120 stored in advance and the initial mounting position, direction, viewing angle information of the camera module 2120 measured during the running of the autonomous vehicle 2000 differ by more than a threshold value.

[0112] The controller 2120 may include a memory 2122 and a processor 2124. The processor 2124 may execute the software stored in the memory 2122 according to the control signal of the controller 2120. Specifically, the controller 2120 stores data and instructions for detecting a field-of-view image from the rear image of the vehicle 2000 in the memory 2122, and the instructions may be executed by the processor 2124 to implement one or more methods disclosed herein.

[0113] At this time, the memory 2122 may be stored in a non-volatile recording medium executable by the processor 2124. The memory 2122 may store software and data through appropriate internal and external devices. The memory 2122 may be composed of a RAM (random access memory), a ROM (read only memory), a hard disk, and a memory 2122 device connected to a dongle.

[0114] The memory 2122 may store at least an operating system (OS), user applications, and executable instructions. The memory 2122 may also store application data and array data structures.

[0115] The processor 2124 is a microprocessor or a suitable electronic processor, and may be a controller, a microcontroller, or a state machine.

[0116] The processor 2124 may be implemented by a combination of computing devices, and the computing devices may be a digital signal processor, a microprocessor, or a suitable combination thereof.

[0117] In addition, the control device 2100 may monitor the internal and external characteristics of the vehicle 2000 by at least one or more sensors 2110 to sense the state.

[0118] The sensor 2110 may be composed of at least one or more sensing modules 2004, and the sensing module 2004 may be realized at a specific position of the vehicle 2000 according to the purpose of sensing. It may be located at the lower part, rear end, front end, upper part, or side end of the vehicle 2000, or may be located on internal components or tires of the vehicle.

[0119] Thereby, the sensing module 2004 can sense information related to driving, such as the engine 2006, tires, steering angle, speed, vehicle weight, etc., as internal information of the vehicle. In addition, at least one or more sensing modules 2004 may be composed of an acceleration sensor 2110, a gyroscope, an image sensor 2110, RADAR, an ultrasonic sensor, a LiDAR sensor, etc., and may sense the movement information of the vehicle 2000.

[0120] The sensing module 2004 can also receive, as external information, specific data regarding the state of the road on which the vehicle 2000 is located, information on surrounding vehicles, the weather, and other external environmental conditions, and thereby sense vehicle parameters. The sensed information may be stored in the memory 2122 temporarily or long-term according to the purpose.

[0121] The sensor 2110 may integrate and collect the information of the sensing module 2004 for collecting the information generated inside and outside the vehicle 2000.

[0122] The control device 2100 may further include a wireless communication device 2130.

[0123] The wireless communication device 2130 is configured to realize wireless communication between vehicles 2000. For example, it enables the vehicle 2000 to communicate with the user's mobile phone, or other wireless communication devices 2130, other vehicles, a central device (traffic control device), a server, etc. The wireless communication device 2130 may transmit and receive wireless signals according to a connection wireless protocol. The wireless communication protocol may be Wi-Fi, Bluetooth, Long-Term Evolution (LTE), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Global Systems for Mobile Communications (GSM), but the communication protocol should not be limited to these.

[0124] In addition, the vehicle 2000 can also achieve communication between vehicles through the wireless communication device 2130. That is, the wireless communication device 2130 may execute communication with other vehicles on the road and other vehicles by vehicle-to-vehicle (V2V) communication. The vehicle 2000 may transmit and receive information such as driving warnings and traffic information through vehicle-to-vehicle communication, and it is also possible to request information from other vehicles or receive requests. For example, the wireless communication device 2130 may execute V2V communication with a dedicated short-range communication (DSRC) device or a C-V2V (Cellular-V2V) device. In addition to vehicle-to-vehicle communication, communication between the vehicle and other things (for example, electronic devices carried by pedestrians, etc.) (V2X, Vehicle to Everything communication) may also be realized through the wireless communication device 2130.

[0125] In addition, the control device 2100 may include a LIDAR device 2140. The LIDAR device 2140 may use data sensed through a LIDAR sensor to detect objects around the vehicle 2000 during operation. The LIDAR device 2140 transmits the detected information to the controller 2120, and the controller 2120 may operate the vehicle 2000 based on the detected information. For example, when there is a vehicle traveling at a low speed as detected information, the controller 2120 may instruct to reduce the speed of the vehicle through the engine 2006. Or, it may also instruct to reduce the entry speed according to the curvature of the curve that the vehicle enters.

[0126] The control device 2100 may further include a camera module 2150. The controller 2120 may extract object information from an external image captured by the camera module 2150, and the controller 2120 may process the related information.

[0127] In addition, the control device 2100 may further include an imaging device for recognizing the external environment. In addition to the LIDAR 2140, RADAR, GPS devices, an odometry device, and other computer vision devices may be used, and these devices may be selected as needed or operate simultaneously to enable more precise sensing.

[0128] The vehicle 2000 may further include a user interface 2008 for user input to the above-described control device 2100. The user interface 2008 may allow the user to input information through appropriate interaction. For example, it may be implemented by a touch screen, keypad, operation buttons, etc. The user interface 2008 may send an input or command to the controller 2120, and the controller 2120 may execute a control operation of the vehicle in response to the input or command.

[0129] In addition, the user interface 2008 may be a device external to the vehicle 2000 and may communicate with the vehicle 2000 through the wireless communication device 2130. For example, the user interface 2008 may be able to interact with a mobile phone, tablet, or other computer device.

[0130] Furthermore, although the vehicle 2000 has been described as including the engine 2006, it is also possible to include other types of propulsion systems. For example, the vehicle may be driven by electric energy or may be driven by a hydrogen energy or a hybrid system combining these. Therefore, the controller 2120 may include a propulsion mechanism corresponding to the propulsion system of the vehicle 2000 and may provide a control signal corresponding thereto to the configuration of each propulsion mechanism.

[0131] Hereinafter, with reference to FIG. 12, the detailed configuration of the control device 2100 of the vehicle 2000 will be described in more detail.

[0132] The control device 2100 includes a processor 2124. The processor 2124 may be a general-purpose single or multi-chip microprocessor, a dedicated microprocessor, a microcontroller, a programmable gate array, or the like. The processor is also referred to as a central processing unit (CPU). Also, the processor 2124 can be used by a combination of multiple processors.

[0133] The control device 2100 includes a memory 2122. The memory 2122 may be any electronic component capable of storing electronic information. The memory 2122 may also include a combination of memories in addition to a single memory.

[0134] According to various embodiments, the data and instruction words 2122a of the vehicle 2000 may be stored in the memory 2122. When the processor 2124 executes the instruction words 2122a, all or part of the instruction 2122a and the data 2122b required for the execution of the instruction may be loaded 2124a, 2124b onto the processor 2124.

[0135] The control device 2100 may include a transmitter 2130a, a receiver 2130b, or a transceiver 2130c for allowing transmission and reception of signals. One or more antennas 2132a, 2132b may be electrically connected to the transmitter 2130a, the receiver 2130b, or each transceiver 2130c, and may additionally include an antenna.

[0136] The control device 2100 may include a digital signal processor (DSP) 2170. Through the DSP 2170, the vehicle may be enabled to process digital signals quickly.

[0137] The control device 2100 may include a communication interface 2180. The communication interface 2180 may include one or more ports and / or communication modules for connecting other devices to the control device 2100. The communication interface 2180 may enable interaction between the user and the control device 2100.

[0138] The various components of the control device 2100 may be connected via one or more buses 2190, and the bus 2190 may include a power bus, a control signal bus, a status signal bus, a data bus, and the like. According to the control of the processor 2124, the components may transmit mutual information via the bus 2190 and execute the intended functions.

[0139] The devices described above may be implemented by hardware components, software components, and / or combinations of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as a processor, a controller, an ALU (arithmetic logic unit), a digital signal processor, a microcomputer, an FPGA (field programmable gate array), a PLU (programmable logic unit), a microprocessor, or various devices capable of executing instructions and responding. The processing device may execute an operating system (OS) and one or more software applications running on the OS. Further, the processing device may access data, record, manipulate, process, and generate data in response to the execution of the software. For the sake of convenience of understanding, although it may be described that one processing device is used, those skilled in the art will understand that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, the processing device may include multiple processors or one processor and one controller. Also, other processing configurations, such as parallel processors, are possible.

[0140] Software may include a computer program, code, instructions, or a combination of one or more of these, and may configure a processing device to operate as desired or independently or collectively instruct the processing device. Software and / or data may be embodied in any kind of machine, component, physical device, computer recording medium, or device for interpretation by a processing device or for providing instructions or data to the processing device. Software may be distributed on a computer system connected by a network and recorded or executed in a distributed state. Software and data may be recorded on one or more computer-readable recording media.

[0141] The method according to an embodiment may be realized in the form of program instructions executable by various computer means and recorded on a computer-readable medium. At this time, the medium may be one that continuously records a program executable by a computer, or may be one that temporarily records it for execution or download. Further, the medium may be various recording means or storage means in a form in which a single or a plurality of hardware are combined, and may be not only a medium directly connected to a certain computer system but also one that is distributed and exists on a network. Examples of the medium include magnetic media such as hard disks, floppy (registered trademark) disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to record program instructions such as ROMs, RAMs, and flash memories. Further, as examples of other media, it also includes an app store that distributes applications, and recording media and storage media managed by sites, servers, etc. that supply and distribute other various software.

[0142] The various embodiments of this specification and the terms used herein are not intended to limit the technology described herein to specific embodiments, but should be understood to include various modifications, equivalents, and / or alternatives of the corresponding embodiments. In connection with the description of the drawings, similar reference numerals are used for similar components. Singular expressions may include plural expressions unless the context clearly dictates otherwise. In this specification, expressions such as "A or B", "at least one of A and / or B", "A, B, or C", or "at least one of A, B, and / or C" may include all possible combinations of the listed items. Expressions such as "first", "second", "the first", or "the second" modify the corresponding components regardless of order or importance, and are used only to distinguish one component from another, not to limit the component. When a certain (e.g., first) component is described as being "(functionally or communicatively) coupled to" or "connected to" another (e.g., second) component, the certain component may be directly connected to the other component or may be connected via another component (e.g., a third component).

[0143] The term "module" as used herein includes units composed of hardware, software, or firmware, and may be used interchangeably with terms such as, for example, logic, logic blocks, components, or circuits. A module may be an integrally configured component or the smallest unit or a part thereof that executes one or more functions. For example, a module may be configured as an ASIC (application-specific integrated circuit).

[0144] According to various embodiments, each of the described components (e.g., modules or programs) may include one or more objects. According to various embodiments, one or more of the above-described components or steps may be omitted, or one or more different components or steps may be added. Alternatively or additionally, a plurality of components (e.g., modules or programs) may be integrated as one component. In such a case, the integrated component may perform one or more functions of each of the plurality of components in the same or similar manner as those performed by the corresponding component among the plurality of components before integration. According to various embodiments, the steps performed by a module, program, or other component may be performed sequentially, in parallel, iteratively, or heuristically, or one or more of the steps may be performed in a different order, omitted, or one or more different steps may be added.

Description of Reference Numerals

[0145] 110: Camera device 120: Digital mirror 130: Electronic device

Claims

1. A method for operating an electronic device mounted on a vehicle, comprising: executing a first mode of displaying a surrounding image of the vehicle in a split screen through a digital rearview mirror of the vehicle based on an operation associated with stopping the vehicle; and Executing a preset second mode through the digital rear-view mirror while the vehicle is running. Including, A method for operating an electronic device.

2. The surrounding image includes at least two of a front image, a direct rear image, a left rear image, or a right rear image of the vehicle. A method for operating the electronic device of claim 1.

3. The step of executing the first mode includes: Immediately after the vehicle has stopped, When a preset time has elapsed since the vehicle stopped, When the vehicle starts moving after stopping, when the speed of the vehicle drops below a preset threshold; or when the speed of the vehicle falls below the threshold and then rises above the threshold; To be executed, A method for operating the electronic device of claim 1.

4. The step of executing the first mode includes: acquiring the surrounding images through a plurality of camera devices; generating the split screen by arranging the peripheral images in a preset arrangement; and displaying the split screen on the digital rearview mirror. Including, A method for operating the electronic device of claim 1.

5. The step of executing the first mode includes: When the digital rearview mirror is in a mirror state, the digital rearview mirror is switched to a display state, and then the digital rearview mirror is executed. A method for operating the electronic device of claim 1.

6. The step of executing the second mode includes: When the vehicle starts moving, when the vehicle reaches a preset speed after starting, or When a preset time has elapsed after the vehicle has started moving, To be executed, A method for operating the electronic device of claim 1.

7. The step of executing the second mode includes: and executing the digital rearview mirror in a mirror state. Including, A method for operating the electronic device of claim 1.

8. The step of executing the second mode includes: displaying at least one image of the surroundings of the vehicle on the digital rearview mirror; Including, A method for operating the electronic device of claim 1.

9. Using the vehicle status information, determining whether the vehicle is stopped or running Further comprising: The state information includes at least one of speed information, brake pedal operation information, accelerator pedal operation information, and position information. A method for operating the electronic device of claim 1.

10. The step of executing the first mode includes: recognizing a predetermined object from at least one of the surrounding images; and outputting information about the object while displaying the split screen on the digital rearview mirror; Including, A method for operating the electronic device of claim 1.

11. The step of outputting the information about the object comprises: displaying an indicator within the split screen indicating the information regarding the object; or outputting an audio signal indicative of said information regarding said object. at least one of A method for operating an electronic device according to claim 10.

12. The object includes at least one of a human, an object, an object approaching the vehicle, or an object moving away from the vehicle. A method for operating an electronic device according to claim 10.

13. The information about the object includes at least one of the type, size, speed, or distance of the object to the vehicle. A method for operating an electronic device according to claim 10.

14. An electronic device mounted on a vehicle, Memory, and a processor coupled to the memory and configured to execute at least one instruction stored in the memory; Including, The processor, Executing a first mode in which a surrounding image of the vehicle is displayed in a split screen through a digital rearview mirror of the vehicle based on an operation associated with the stop of the vehicle; and executing a preset second mode through the digital rearview mirror while the vehicle is traveling. electronic equipment.

15. The surrounding image includes at least two of a front image, a direct rear image, a left rear image, or a right rear image of the vehicle.

15. The electronic device of claim 14.

16. The processor, Immediately after the vehicle has stopped, When a preset time has elapsed since the vehicle stopped, When the vehicle starts moving after stopping, when the speed of the vehicle drops below a preset threshold; or when the speed of the vehicle decreases below the threshold and then increases above the threshold; configured to execute the first mode; 15. The electronic device of claim 14.

17. In the first mode, the processor: Acquiring the surrounding images through a plurality of camera devices, generating the split screen by arranging the peripheral images in a preset arrangement; configured to display the split screen on the digital rearview mirror; 15. The electronic device of claim 14.

18. The processor, When the digital rear mirror is in a mirror state, the first mode is executed after the digital rear mirror is switched to a display state.

15. The electronic device of claim 14.

19. The processor, When the vehicle starts moving, when the vehicle reaches a preset speed after starting, or When a preset time has elapsed after the vehicle has started moving, configured to execute the second mode; 15. The electronic device of claim 14.

20. In the second mode, the processor: The digital rearview mirror is configured to perform in a mirror state.

15. The electronic device of claim 14.

21. In the second mode, the processor: The digital rearview mirror is configured to display at least one image of the surroundings of the vehicle.

15. The electronic device of claim 14.

22. The processor, The vehicle status information is used to determine whether the vehicle is stopped or running, The state information includes at least one of speed information, brake pedal operation information, accelerator pedal operation information, and position information.

15. The electronic device of claim 14.

23. In the first mode, the processor: Recognizing a predetermined object from at least one of the surrounding images; and outputting information about the object while displaying the split screen on the digital rearview mirror.

15. The electronic device of claim 14.

24. The processor, displaying an indicator within the split screen that indicates the information regarding the object; or outputting an audio signal indicative of the information regarding the object; configured to perform at least one of 24. The electronic device of claim 23.

25. The object includes at least one of a human, an object, an object approaching the vehicle, or an object moving away from the vehicle.

24. The electronic device of claim 23.

26. The information about the object includes at least one of the type, size, speed, or distance of the object to the vehicle.

24. The electronic device of claim 23.

27. An image processing system mounted on a vehicle, A plurality of camera devices each configured to capture an image of the surroundings of the vehicle; an electronic device communicatively coupled to the camera device and configured to process the peripheral image; and a digital rearview mirror communicatively coupled to the electronic device and configured to display video information from the electronic device; Including, The electronic device comprises: Executing a first mode in which a surrounding image of the vehicle is displayed in a split screen through a digital rearview mirror of the vehicle based on an operation associated with the stop of the vehicle; and executing a preset second mode through the digital rearview mirror while the vehicle is traveling. Video processing system.

28. The electronic device comprises: In the first mode, Recognizing a predetermined object from at least one of the surrounding images; and outputting information about the object while displaying the split screen on the digital rearview mirror.

28. The video processing system of claim 27.

29. The electronic device and the digital rear mirror are integrally configured and disposed within a forward field of vision of a driver in the vehicle.

28. The video processing system of claim 27.

30. The electronic device and the digital rear mirror are configured separately, The digital rear-view mirror is disposed within a forward field of vision of a driver of the vehicle.

28. The video processing system of claim 27.

Images