Electronic device of vehicle for recording operation intensity of pedal based on image and operating method of the same

Abstract

To provide an electronic device of a vehicle for recording the pedal operation intensity based on a video, and an operating method of the same.SOLUTION: In the present disclosure, the electronic device is configured to analyze a video for a pedal and detect a location of the pedal, to determine an operation intensity of the pedal on the basis of the location, and to store the operation intensity. The electronic device stores a database in which different locations of the pedal and different operation intensities of the pedal are mapped to each other, and determines the operation intensity mapped to the location using the database. The camera device is fixed adjacent to the pedal, captures a video for the pedal, and transmits the video to the electronic device. The pedal includes at least one of a brake pedal and an accelerator pedal.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to an electronic device for a vehicle and an operating method thereof for recording the operation intensity of a pedal based on an image.

Background Art

[0002] Generally, a vehicle is equipped with an event data recorder (EDR). The accident recording device records the situation of the vehicle before and after a collision for 15 items. Some of these items are related to the operation of the accelerator pedal and the brake pedal. Specifically, the accident recording device records the operation intensity of the accelerator pedal and the presence or absence of the operation of the brake pedal.

[0003] When an accident suspected of sudden acceleration of the vehicle occurs, even if the driver of the vehicle operates the brake pedal, it is difficult to prove this. Since the accident recording device only records whether there is an operation on the brake pedal, when the driver operates the brake pedal relatively weakly, the accident recording device records that there is no operation on the brake pedal. As a result, the vehicle manufacturer claims that the driver did not operate the brake pedal or did not operate it sufficiently, and tries to avoid the responsibility for the accident.

Summary of the Invention

Problems to be Solved by the Invention

[0004] The present disclosure provides an electronic device for a vehicle and an operating method thereof for recording the operation intensity of a pedal based on an image.

Means for Solving the Problems

[0005] In the present disclosure, an operating method of an electronic device mounted on a vehicle may include a step of analyzing an image of a pedal to detect the position of the pedal, a step of determining the operation intensity of the pedal based on the position, and a step of storing the operation intensity.

[0006] In the present disclosure, an electronic device mounted on a vehicle includes a memory and a processor configured to be connected to the memory and execute at least one instruction word stored in the memory. The processor may be configured to analyze an image of a pedal to detect the position of the pedal, determine the operation intensity of the pedal based on the position, and store the operation intensity in the memory.

[0007] In the present disclosure, a video processing system mounted on a vehicle may include a camera device fixed adjacent to a pedal and configured to capture an image of the pedal, and an electronic device configured to analyze the image to detect the position of the pedal, determine the operation intensity of the pedal based on the position, and store the operation intensity.

[0008] In the method of operating a video processing system mounted on a vehicle according to the present disclosure, the method may include a step in which a camera device fixed adjacent to a pedal captures an image of the pedal, a step in which an electronic device analyzes the image to detect the position of the pedal, a step in which the electronic device determines the operation intensity of the pedal based on the position, and a step in which the electronic device stores the operation intensity.

Effect of the Invention

[0009] According to the present disclosure, an electronic device records the operation intensity of a pedal based on an image of the pedal. At this time, the electronic device records at least one of the operation intensity of the brake pedal or the operation intensity of the accelerator pedal. This serves as data for proving not only the presence or absence of the operation of the pedal but also the operation intensity of the pedal. Therefore, in a situation where it is difficult to investigate the cause, such as an accident in which a sudden acceleration of the vehicle is suspected, a relatively accurate judgment can be made regarding the driver's fault. Here, when recording the operation intensity, the electronic device can enhance the reliability of the operation intensity through encryption and prevent indiscriminate sharing.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Embodiments for Carrying Out the Invention

[0011] Hereinafter, various embodiments of the present specification will be described with reference to the accompanying drawings.

[0012] FIG. 1 is a block diagram showing the configuration of a video processing system 100 in various embodiments.

[0013] Referring to FIG. 1, the video processing system 100 may include a camera device 110 and an electronic device 120. The video processing system 100 may be attached to a vehicle. In some embodiments, at least one other component may be added to the video processing system 100.

[0014] The camera device 110 may capture an image of the vehicle's pedal. For this purpose, the camera device 110 may be fixedly installed adjacent to the vehicle's pedal. Here, the pedal may include at least one of a brake pedal or an accelerator pedal. According to one embodiment, the camera device 110 may be installed in front of at least one of the brake pedal or the accelerator pedal. In such a case, the camera device 110 can capture an image of the front surface of at least one of the brake pedal or the accelerator pedal. According to other embodiments, the camera device 110 may be installed on one side of the brake pedal. In such a case, the camera device 110 can capture an image of one side surface of the brake pedal. According to still other embodiments, the camera device 110 may be installed on one side of the accelerator pedal. In such a case, the camera device 110 can capture an image of one side surface of the accelerator pedal.

[0015] In some embodiments, the camera device 110 may include at least one of one or more lenses, an image sensor, a flash, and an image signal processor (ISP). Some of the lenses may have the same lens characteristics (e.g., angle of view, focal length, autofocus, f number, or optical zoom). The lenses may include a light source lens or a telephoto lens. The image sensor may obtain an image corresponding to a subject by converting light emitted or reflected from the subject and transmitted through one or more lenses into an electrical signal. For example, the image sensor may include one image sensor selected from among image sensors with different attributes such as an RGB sensor, a BW (black and white) sensor, an IF sensor, or a UV sensor, a plurality of image sensors having the same attribute, or a plurality of image sensors having different attributes. Each image sensor may be realized using, for example, a CCD (charged coupled device) sensor or a CMOS (complementary metal oxide semiconductor) sensor. The flash may include one or more light emitting diodes (e.g., RGB (red - green - blue) LED, white LED, infrared LED, or ultraviolet LED), or a xenon lamp.

[0016] The electronic device 120 may record the operation intensity of the pedal based on the video acquired by the camera device 110. At this time, the electronic device 120 may record the operation intensity of the pedal periodically. Specifically, the electronic device 120 may analyze the video to detect the position of the pedal and record the operation intensity of the pedal corresponding to the position of the pedal. This is because the camera device 110 is fixed adjacent to the pedal, and the position of the pedal in the video varies depending on the operation intensity of the pedal by the driver. Here, the position of the pedal may indicate the displacement from the initial position, that is, the position when the pedal is not operated. The electronic device 120 may record the operation intensity of at least one of the brake pedal or the accelerator pedal. According to one embodiment, when the camera device 110 is installed in front of at least one of the brake pedal or the accelerator pedal, the electronic device 120 may record at least one of the operation intensity of the brake pedal or the operation intensity of the accelerator pedal from the position in front of at least one of the brake pedal or the accelerator pedal. According to another embodiment, when the camera device 110 is installed on one side of the brake pedal, the electronic device 120 may record the operation intensity of the brake pedal from the position on one side surface of the brake pedal. According to still another embodiment, when the camera device 110 is installed on one side of the accelerator pedal, the electronic device 120 may record the operation intensity of the accelerator pedal from the position on one side surface of the accelerator pedal.

[0017] The camera device 110 and the electronic device 120 may be communicably connected either wired or wirelessly. In some embodiments, the camera device 110 and the electronic device 120 may be connected via a communication cable. In one embodiment, the camera device 110 and the electronic device 120 may communicate in an analog manner. The analog manner may include, for example, AHD (analog high definition). In other embodiments, the camera device 110 and the electronic device 120 may communicate in a digital manner. For example, the digital manner may include a serial transmission manner. In such a case, the camera device 110 may include a serializer, and the electronic device 120 may include a deserializer. However, it should not be limited thereto, and the camera device 110 and the electronic device 120 may also be connected via an in-vehicle network communication (for example, CAN communication). The camera device 110 and the electronic device 120 may include various communication chips.

[0018] FIG. 2 is a flowchart showing an operation method of recording the operation intensities of the pedals 320, 330, and 420 of the video processing system 100 in various embodiments. FIGS. 3A and 3B are exemplary diagrams for explaining the features of the operation of detecting the positions of the pedals 320 and 330 of the electronic device 120 in one embodiment. FIG. 4 is an exemplary diagram for explaining the features of the operation of detecting the position of the pedal 420 of the electronic device 120 in another embodiment.

[0019] Referring to FIG. 2, at step 210, the camera device 110 may capture images 310, 410 of the pedals 320, 330, 420. For this purpose, the camera device 110 may be fixedly installed adjacent to the pedals 320, 330, 420 inside the vehicle. Here, the pedals 320, 330, 420 may include at least one of the brake pedals 320, 420 or the accelerator pedal 330. According to one embodiment, the camera device 110 may be installed in front of at least one of the brake pedal 320 or the accelerator pedal 330. In such a case, as shown in FIGS. 3a and 3b, the camera device 110 can capture the image 310 with respect to the front surface of at least one of the brake pedal 320 or the accelerator pedal 330. According to other embodiments, the camera device 110 may be installed on one side of the brake pedal 420. In such a case, as shown in FIG. 4, the camera device 110 can capture the image 410 with respect to one side surface of the brake pedal 420. According to still other embodiments, although not shown in the figure, the camera device 110 may be installed on one side of the accelerator pedal. In such a case, the camera device 110 can capture an image with respect to one side surface of the accelerator pedal. Next, at step 220, the camera device 110 may transmit the images 310, 410 to the electronic device 120. Thereby, the electronic device 120 can acquire the images 310, 410 from the camera device 110.

[0020] Next, at step 230, the electronic device 120 may analyze the images 310, 410 to detect the positions of the pedals 320, 330, 420. Specifically, after recognizing the preset reference axes 321, 331, 421 for the pedals 320, 330, 420 in the images 310, 410, the electronic device 120 may detect the positions of the reference axes 321, 331, 421. Here, the reference axes 321, 331, 421 may include at least one of the horizontal axis or the vertical axis. At this time, the electronic device 120 may detect the position of at least one of the brake pedals 320, 420 or the accelerator pedal 330.

[0021] According to one embodiment, when the camera device 110 is installed in front of at least one of the brake pedal 320 or the accelerator pedal 330, the electronic device 120 may detect the position of the front surface of at least one of the brake pedal 320 or the accelerator pedal 330 from the video 310, as shown in FIGS. 3a and 3b. Here, the reference axes 321 and 331 may be the horizontal axes as shown in the figure, but are not limited thereto, and may be, for example, the vertical axis, or a combination of the horizontal axis and the vertical axis. More specifically, when the camera device 110 is installed in front of the brake pedal 320 or both the brake pedal 320 and the accelerator pedal 330, the electronic device 120 may detect the position of the reference axis 321 after recognizing the reference axis 321 on the front surface of the brake pedal 320, as shown in FIG. 3a. On the other hand, when the camera device 110 is installed in front of the accelerator pedal 330 or both the brake pedal 320 and the accelerator pedal 330, the electronic device 120 may detect the position of the reference axis 331 after recognizing the reference axis 331 on the front surface of the accelerator pedal 330, as shown in FIG. 3b. Here, the electronic device 120 may recognize the reference axes 321 and 331 with respect to the lower bottom of the front surface as shown in the figure, but is not limited thereto, and it is also possible to recognize the reference axes 321 and 331 with respect to, for example, the center of the front surface or the upper bottom of the front surface.

[0022] According to another embodiment, when the camera device 110 is installed on one side of the brake pedal 420, the electronic device 120 may detect the position of one side surface of the brake pedal 420 from the video 410, as shown in FIG. 4. Here, the reference axis 421 may be the vertical axis, but is not limited thereto, and may be, for example, the horizontal axis, or a combination of the horizontal axis and the vertical axis. More specifically, the electronic device 120 may detect the position of the reference axis 421 after recognizing the reference axis 421 on the side surface of the brake pedal 420, as shown in the figure. Here, the electronic device 120 may recognize the reference axis 421 with respect to the center of the side surface as shown in the figure, but is not limited thereto, and it is also possible to recognize the reference axis 421 with respect to, for example, a point on the inner side (driver side) or the outer side (opposite side of the inner side) of the side surface.

[0023] According to another embodiment, when the camera device 110 is installed on one side of the accelerator pedal, the electronic device 120 may detect the position of one side surface of the accelerator pedal from the video, although not shown in the figure. Here, the reference axis may be the vertical axis, the horizontal axis, or a combination of the horizontal axis and the vertical axis. More specifically, although not shown in the figure, after recognizing the reference axis on the side surface of the accelerator pedal, the electronic device 120 may detect the position of the reference axis. Here, the electronic device 120 may recognize the reference axis with respect to the center of the side surface, or a point on the inner side (driver side) or the outer side (opposite side of the inner side) of the side surface.

[0024] Next, at step 240, the electronic device 120 may determine the operation intensity of the pedals 320, 330, and 420 based on the position. At this time, the electronic device 120 may store a database in which different positions of the pedals 320, 330, and 420 and different operation intensities of the pedals 320, 330, and 420 are mapped to each other. Using such a database, the electronic device 120 can determine the operation intensity mapped to the position. Specifically, in the database, a plurality of positions and a plurality of operation intensities may be defined in the same number and may be mapped to each other. The position may be defined by dividing the position range from the non-operated position of the pedals 320, 330, and 420 to the maximum-operated position of the pedals 320, 330, and 420 at predetermined intervals by a predetermined number. The operation intensity may be defined by dividing the range of the operation intensity defined for the pedals 320, 330, and 420 at predetermined intervals by the number of positions. According to one embodiment, the database may be constructed with the cooperation of the driver when the video processing system 100 or the electronic device 120 is installed in the vehicle, or when the video processing system 100 or the electronic device 120 is initialized in the vehicle. According to another embodiment, the database may be generated in a batch for the vehicle model by the vehicle or electronic device 120 manufacturer and then stored in the electronic device 120.

[0025] Next, at stage 250, the electronic device 120 may store the operation intensity. Specifically, the electronic device 120 may store the operation intensity while storing the videos 310, 410. Here, the electronic device 120 may store the operation intensity corresponding to the shooting time of the videos 310, 410. According to one embodiment, the electronic device 120 may store the operation intensity in the file in which the videos 310, 410 are recorded. According to other embodiments, the electronic device 120 may store the operation intensity in a file separate from the videos 310, 410. At this time, the operation intensity may be displayed only through a predetermined viewer program. The electronic device 120 may encrypt the operation intensity using a preset encryption key and then store it. Thereby, the operation intensity will be displayed after being decrypted using the encryption key.

[0026] FIG. 5 is a block diagram showing the configuration of the electronic device 120 in various embodiments.

[0027] Referring to FIG. 5, the electronic device 120 may be attached to a vehicle together with the camera device 110 and may be communicably connected to the camera device 110. Specifically, the electronic device 100 may include at least one of a communication module 510, an input module 520, a display module 530, an interface module 540, an audio output module 550, a memory 560, or a processor 570. In some embodiments, at least one of the components of the electronic device 120 (for example, the communication module, the display module, the audio output module) may be omitted, and at least one other component (for example, at least one sensor) may be added. In some embodiments, at least two of the components of the electronic device 120 may be implemented as one integrated circuit. In one embodiment, the components of the electronic device 120 may be integrated and configured as a single unit. In such a case, the components of the electronic device 120 may be housed in a single housing. In other embodiments, the components of the electronic device 120 may be distributed and configured in at least two units.

[0028] The communication module 510 may perform communication between the electronic device 120 and an external device. The communication module 510 may establish a communication channel with the external device and perform communication with the external device via the communication channel. Here, the external device may include at least one of a satellite, a base station, a server, or another electronic device (for example, used by a driver). The communication module 510 may include at least one of a wired communication module or a wireless communication module. The wired communication module may be connected to the external device by wire and communicate by wire. The wireless communication module may include at least one of a short-range communication module or a long-range communication module. The short-range communication module may communicate with the external device by a short-range communication method. For example, the short-range communication method may include at least one of Bluetooth (registered trademark), Wi-Fi direct, NFC communication (Near Field Communication), or infrared communication (IrDA: infrared data association). The long-range communication module may communicate with the external device by a long-range communication method. Here, the long-range communication module may communicate with the external device via a network. For example, the network may include at least one of a cellular network, the Internet, or a computer network such as a LAN (local area network) or a WAN (wide area network).

[0029] The input module 520 may input signals used for at least one component of the electronic device 120. The input module 520 may include, for example, at least one of at least one button (also called a key), a keyboard, a keypad, a mouse, a joystick, or a microphone. Here, the button may include at least one of a physical button or a touch button. In some embodiments, the input module 520 may include at least one of a touch circuity configured to sense a touch or a sensor circuit configured to measure the intensity of a force generated by the touch.

[0030] The display module 530 may visually output information outside the electronic device 120. For example, the display module 530 may include at least any one of a display, a hologram device, or a projector. In some embodiments, the display module 530 may be combined with at least one of the touch circuit or the sensor circuit of the input module 520 and implemented as a touch screen.

[0031] The interface module 540 may be provided for the interface between the electronic device 120 and an external device. Specifically, the interface module 540 may support a specified protocol for connecting to the external device either wired or wirelessly. Here, the external device may include at least one of a vehicle or the camera device 110. In one embodiment, when communicating with the camera device 110 in an analog manner, the interface module 540 may receive video information from the camera device 110 and convert it from an analog signal to digital data. In other embodiments, when communicating with the camera device 110 in a digital manner, the interface module 540 may receive video information from the camera device 110 and convert it from serial data to parallel data. In such a case, the interface module 540 may be implemented as a deserializator.

[0032] The audio output module 550 may output an audio signal outside the electronic device 120. For example, the audio output module 550 may include at least one of a speaker or a receiver. In one embodiment, within the speaker, there may be included at least one voice coil that provides vibration by means of a diaphragm, and a magnet that forms a magnetic field. When current flows through the voice coil, the magnetic field formed by the voice coil may interact with the magnetic field formed by the magnet to vibrate the voice coil. The diaphragm connected to the voice coil may vibrate based on the vibration of the voice coil. The speaker may output an audio signal based on the vibration of the diaphragm.

[0033] Memory 560 may store various data used by at least one component of the electronic device 120. For example, memory 560 may include at least one of a volatile memory or a non-volatile memory. The data may include at least one program and associated input data or output data. The program may be stored in memory 560 as software including at least one instruction, and may include, for example, at least one of an operating system, middleware, or an application. Memory 560 may include at least one of a first memory built into the electronic device 120 or a second memory removable with respect to the electronic device 120.

[0034] According to various embodiments, memory 560 may store a program for recording the operation intensity of pedals 320, 330, 420 based on videos 310, 410. According to various embodiments, memory 560 may store a database in which different positions of pedals 320, 330, 420 and different operation intensities of pedals 320, 330, 420 are respectively mapped to each other. Specifically, in the database, a plurality of positions and a plurality of operation intensities may be defined in the same number and may be respectively mapped to each other. The position may be defined by dividing the position range from the non-operated position of pedals 320, 330, 420 to the maximum-operated position of pedals 320, 330, 420 at a predetermined interval by a predetermined number. The operation intensity may be defined by dividing the operation intensity range from the minimum intensity defined for pedals 320, 330, 420 to the maximum intensity at a predetermined interval by the number of positions. According to some embodiments, the database may further store the video when pedals 320, 330, 420 are not operated and the video when pedals 320, 330, 420 are operated at maximum.

[0035] According to one embodiment, when the camera device 110 is installed in front of at least one of the brake pedal 320 or the accelerator pedal 330, as shown in Table 1 below, in the database, for each of the brake pedal 320 and the accelerator pedal 330, different positions and different operation intensities may be respectively mapped to each other. According to other embodiments, when the camera device 110 is installed on one side of the brake pedal 420, as shown in Table 2 below, in the database, for the brake pedal 420, different positions and different operation intensities may be respectively mapped to each other.

[0036]

Table 1

[0037]

Table 2

[0038] The processor 570 may control at least one component of the electronic device 120. Thereby, the processor 570 can execute data processing or operations. For example, the hardware for processing data may include an ALU (arithmetic and logic unit), an FPU (floating point unit), an FPGA (field programmable gate array), a CPU (central processing unit), and / or an AP (application processor). The processor 570 may have a single-core processor structure, or may have a multi-core processor structure such as a dual core, a quad core, a hexa core, or an octa core. According to various embodiments, the processor 570 may execute instructions stored in the memory 560.

[0039] According to various embodiments, the processor 570 may record the operation intensity of the pedals 320, 330, 420 based on the images 310, 410 of the pedals 320, 330, 420 acquired by the camera device 110. At this time, the processor 570 may record the operation intensity of the pedals 320, 330, 420 periodically. Specifically, the processor 570 may analyze the images 310, 410 to detect the positions of the pedals 320, 330, 420, and record the operation intensity of the pedals 320, 330, 420 corresponding to the positions of the pedals 320, 330, 420. This is because since the camera device 110 is fixed adjacent to the pedals 320, 330, 420, the positions of the pedals 320, 330, 420 in the images 310, 410 vary depending on the operation intensity of the pedals 320, 330, 420 by the driver. At this time, the processor 570 may utilize a pre-stored database to determine the operation intensity of the pedals 320, 330, 420 mapped to the positions of the pedals 320, 330, 420. Here, the positions of the pedals 320, 330, 420 may indicate the displacement from the initial positions, i.e., the positions of the pedals 320, 330, 420 when not operated. The processor 570 may record the operation intensity of at least one of the brake pedals 320, 420 or the accelerator pedal 330.

[0040] According to various embodiments, the processor 570 may pre-store a database in the memory 560. According to one embodiment, the processor 570 may directly construct a database and store it in the memory 560. When the vehicle-mounted video processing system 100 or the electronic device 120 is installed in the vehicle, or when the vehicle-mounted video processing system 100 or the electronic device 120 is initialized in the vehicle, the processor 570 may construct a database in cooperation with the driver. According to other embodiments, the processor 570 may download a database from an external device, such as a server, and store it in the memory 560. In such a case, the database may be created and provided in a batch for the vehicle models of the vehicle by the vehicle or the manufacturer of the electronic device 120. For example, the manufacturer may construct a database for different vehicle models respectively, and then provide the same database to the electronic device 120 installed in the vehicle corresponding to a specific vehicle model.

[0041] FIG. 6 is a flowchart showing an operation method of constructing a database of the electronic device 120 in various embodiments. FIGS. 7A and 7B are exemplary diagrams for explaining the characteristics of the operation of detecting the positions of the pedals 720 and 730 of the electronic device 120 in one embodiment. FIG. 8 is an exemplary diagram for explaining the characteristics of the operation of detecting the position of the pedal 820 of the electronic device 120 in another embodiment.

[0042] Referring to FIG. 6, at step 610, the electronic device 120 may sense an event for constructing a database. Here, the event may be a signal for turning on the electronic device 120 that occurs through the input module 520 after the vehicle-mounted video processing system 100 or the electronic device 120 is installed in the vehicle. Or, the event may indicate a signal for initializing the vehicle-mounted video processing system 100 or the electronic device 120 that occurs through the input module 520. Specifically, when an event occurs through the input module 520, the processor 570 senses the event.

[0043] Next, at step 620, the electronic device 120 may detect the positions of the pedals 720, 730, 820 when not operated, i.e., the first positions. Specifically, the processor 570 may analyze the images 711, 811 at the first time point acquired from the camera device 110 to detect the first positions. Next, at step 630, the electronic device 120 may detect the positions of the pedals 720, 730, 820 when maximally operated, i.e., the second positions. Specifically, the processor 570 may analyze the images 713, 813 at the second time point acquired from the camera device 110 to detect the second positions. After the processor 570 recognizes the preset reference axes 721, 731, 821 for the pedals 720, 730, 820 in the images 711, 811 at the first time point and the images 713, 813 at the second time point, respectively, the processor 570 may detect the first positions and the second positions of the reference axes 721, 731, 821, respectively. Here, the reference axes 721, 731, 821 may include at least one of the horizontal axis or the vertical axis. At this time, the processor 570 may detect the first positions and the second positions of at least one of the brake pedals 720, 820 or the accelerator pedal 730.

[0044] Therefore, the processor 570 may output a guidance message to the driver. Specifically, the processor 570 may output a guidance message for non-operation of the pedals 720, 730, 820 at the first time point, and may output a guidance message for maximum operation of the pedals 720, 730, 820 at the second time point. Here, the guidance message may include at least one of a visual message or an auditory message. As an example, the processor 570 may directly output the guidance message through the display module 530 or the audio output module 550. As another example, the processor 570 may transmit the guidance message to another electronic device (used by the driver) through the communication module 510 so that it is output through the other electronic device. As still another example, the processor 570 may transmit the guidance message to the vehicle through the interface module 540 so that it is output through at least one component of the vehicle.

[0045] According to one embodiment, when the camera device 110 is installed in front of at least one of the brake pedal 720 or the accelerator pedal 730, the processor 570, as shown on the left side of FIGS. 7a and 7b, after detecting the first position of the front surface of at least one of the brake pedal 720 or the accelerator pedal 730 from the video 711 at the first time point, as shown on the right side of FIGS. 7a and 7b, may detect the second position of the front surface of at least one of the brake pedal 720 or the accelerator pedal 730 from the video 713 at the second time point. Here, the reference axes 721, 731 may be the horizontal axes as shown in the figure, but are not limited thereto, and may be, for example, the vertical axis, or a combination of the horizontal axis and the vertical axis. More specifically, when the camera device 110 is installed in front of the brake pedal 720, or both the brake pedal 720 and the accelerator pedal 730, the processor 570, as shown on the left side of FIG. 7a, after detecting the first position of the reference axis 721 from the front surface of the brake pedal 720, as shown on the right side of FIG. 7a, may detect the second position of the reference axis 721 from the front surface of the brake pedal 720. On the other hand, when the camera device 110 is installed in front of the accelerator pedal 730, or both the brake pedal 720 and the accelerator pedal 730, the processor 570, as shown on the left side of FIG. 7b, after detecting the first position of the reference axis 731 from the front surface of the accelerator pedal 730, as shown on the right side of FIG. 7b, may detect the second position of the reference axis 731 from the front surface of the accelerator pedal 730. Here, the processor 570 may recognize the reference axes 721, 731 with respect to the lower bottom of the front surface as shown in the figure, but is not limited thereto, and it is also possible to recognize the reference axes 721, 731 with respect to the center of the front surface or the lower bottom of the front surface, for example.

[0046] According to another embodiment, when the camera device 110 is installed on one side of the brake pedal 820, the processor 570 may detect the first position of one side surface of the brake pedal 820 from the video 811 at the first time point as shown in the upper part of FIG. 8, and then detect the second position of the side surface of the brake pedal 820 from the video 813 at the second time point as shown in the lower part of FIG. 8. Here, the reference axis 821 may be a vertical axis, but is not limited thereto, and may be, for example, a horizontal axis or a combination of a horizontal axis and a vertical axis. More specifically, as shown in the upper part of FIG. 8, after detecting the first position of the reference axis 821 from the side surface of the brake pedal 820, the processor 570 may detect the second position of the reference axis 821 from the side surface of the brake pedal 820 as shown in the lower part of FIG. 8. Here, as shown in the figure, the processor 570 may recognize the reference axis 821 with respect to the center of the side surface, but is not limited thereto, and for example, it is also possible to recognize the reference axis 821 with respect to a point on the inner side surface (driver side) or the outer side surface (opposite side of the inner side surface) of the side surface.

[0047] According to still another embodiment, when the camera device 110 is installed on one side of the accelerator pedal, although not shown in the figure, the processor 570 may detect the first position of one side surface of the accelerator pedal from the video at the first time point, and then detect the second position of the side surface of the accelerator pedal from the video at the second time point. Here, the reference axis may be a vertical axis, a horizontal axis, or a combination of a horizontal axis and a vertical axis. More specifically, although not shown in the figure, after detecting the first position of the reference axis from the side surface of the accelerator pedal, the processor 570 may detect the second position of the reference axis from the side surface of the accelerator pedal. Here, the processor 570 may recognize the reference axis with respect to the center of the side surface, or a point on the inner side surface (driver side) or the outer side surface (opposite side of the inner side surface) of the side surface.

[0048] According to some embodiments, indicators (not shown) may be applied at the intersection points of the pedals 720, 730, 820 with the reference axes 721, 731, 821 so that the positions of the reference axes 721, 731, 821 from the pedals 720, 730, 820 can be easily identified. Thereby, the processor 570 can recognize the reference axes 721, 731, 821 by identifying the indicators in the videos 711, 713, 811, 813 and detect the positions of the reference axes 721, 731, 821. For example, the indicator may be generated with at least one of a specific pattern, color, or light for the pedals 720, 730, 820, or may be attached like a stamp.

[0049] Subsequently, at step 640, the electronic device 120 may define a plurality of positions within the position range from the unoperated positions of the pedals 720, 730, 820 to the maximum operated positions of the pedals 720, 730, 820. Specifically, the processor 570 may define a plurality of positions from the first position to the second position. At this time, the processor 570 may define the positions by dividing the position range at predetermined intervals by a predetermined number.

[0050] Subsequently, at step 650, the electronic device 120 may define a plurality of operating intensities within the range of operating intensities from the minimum intensity defined for the pedals 720, 730, 820 to the maximum intensity. Specifically, the processor 570 may define a plurality of operating intensities from the minimum intensity to the maximum intensity. At this time, the processor 570 may define the operating intensities by dividing the range of operating intensities at predetermined intervals by a predetermined number, i.e., the number of positions.

[0051] Finally, at step 650, the electronic device 120 may map the position and the operation intensity with respect to each other and store them in a database. Specifically, the processor 570 may sequentially map the position from the first position to the second position and the operation intensity from the minimum intensity to the maximum intensity with respect to each other and store them in the database. As a result, the processor 570 may store the database in the memory 560. According to some embodiments, the processor 570 may further map and store the images 711, 811 of the pedals 720, 730, 820 when not operated and the images 713, 813 of the pedals 720, 730, 820 when maximally operated corresponding to the first position and the second position, respectively. This may be utilized to prove the reliability of the database as needed.

[0052] FIG. 9 is a flowchart showing an operation method of recording the operation intensity of the pedals 320, 330, 420 of the electronic device 120 in various embodiments. FIG. 10 is an exemplary diagram for explaining the features of the operation of storing the operation intensity of the electronic device 120 in various embodiments. FIG. 11 is an exemplary diagram for explaining the features of the operation of displaying the operation intensity of the electronic device 120 in various embodiments.

[0053] Referring to FIG. 9, at step 910, the electronic device 120 may sense that a predetermined period for recording the operation intensity of the pedals 320, 330, 420 has arrived. Here, an event for recording the operation intensity may be generated according to the period. Specifically, the processor 570 may sense that the period has arrived based on the event.

[0054] Next, at step 920, the electronic device 120 may acquire images of the pedals 320, 330, 420. For this purpose, the interface module 540 may be communicably connected to the camera device 110, either wired or wirelessly. Specifically, the camera device 110 may capture images 310, 410 of the pedals 320, 330, 420. For this purpose, the camera device 110 may be fixedly positioned adjacent to the pedals 320, 330, 420 within the vehicle. Here, the pedals 320, 330, 420 may include at least one of the brake pedals 320, 420 or the accelerator pedal 330. According to one embodiment, the camera device 110 may be installed in front of at least one of the brake pedal 320 or the accelerator pedal 330. In such a case, as shown in FIGS. 3a and 3b, the camera device 110 may capture the image 310 with respect to the front surface of at least one of the brake pedal 320 or the accelerator pedal 330. According to another embodiment, the camera device 110 may be installed on one side of the brake pedal 420. In such a case, as shown in FIG. 4, the camera device 110 may capture the image 410 with respect to one side surface of the brake pedal 420. According to still another embodiment, the camera device 110 may be installed on one side surface of the accelerator pedal. In such a case, although not shown in the figure, the camera device 110 may capture an image with respect to one side surface of the accelerator pedal. Next, the camera device 110 may transmit the images 310, 410 to the electronic device 120. Thereby, the processor 570 may acquire the images 310, 410 from the camera device 110 through the interface module 540.

[0055] Subsequently, at stage 930, the electronic device 120 may analyze the images 310 and 410 to detect the positions of the pedals 320, 330, and 420. Specifically, the processor 570 may recognize the preset reference axes 321, 331, and 421 for the pedals 320, 330, and 420 in the images 310 and 410 and detect the positions of the reference axes 321, 331, and 421. Here, the reference axes 321, 331, and 421 may include at least one of the horizontal axis or the vertical axis. At this time, the processor 570 may detect the position of at least one of the brake pedals 320 and 420 or the accelerator pedal 330.

[0056] According to an embodiment, when the camera device 110 is installed in front of at least one of the brake pedal 320 or the accelerator pedal 330, the processor 570 may detect the position of the front surface of at least one of the brake pedal 320 or the accelerator pedal 330 from the image 310 as shown in FIGS. 3a and 3b. Here, the reference axes 321 and 331 may be the horizontal axis as shown in the figure, but are not limited thereto, and may be, for example, the vertical axis or a combination of the horizontal axis and the vertical axis. More specifically, when the camera device 110 is installed in front of the brake pedal 320 or both the brake pedal 320 and the accelerator pedal 330, the processor 570 may, as shown in FIG. 3a, recognize the reference axis 321 from the front surface of the brake pedal 320 and then detect the position of the reference axis 321. On the other hand, when the camera device 110 is installed in front of the accelerator pedal 330 or both the brake pedal 320 and the accelerator pedal 330, the processor 570 may, as shown in FIG. 3b, recognize the reference axis 331 from the front surface of the accelerator pedal 330 and then detect the position of the reference axis 331. Here, the electronic device 120 may recognize the reference axes 321 and 331 with respect to the lower bottom of the front surface as shown in the figure, but is not limited thereto, and it is also possible to recognize the reference axes 321 and 331 with respect to the center of the front surface or the upper bottom of the front surface, for example.

[0057] According to another embodiment, when the camera device 110 is installed on one side of the brake pedal 420, the processor 570 may detect the position of one side surface of the brake pedal 420 from the video 410 as shown in FIG. 4. Here, the reference axis 421 may be the vertical axis, but is not limited thereto. For example, it may be the horizontal axis or a combination of the horizontal axis and the vertical axis. More specifically, as shown in the figure, after recognizing the reference axis 421 from the side surface of the brake pedal 420, the processor 570 may detect the position of the reference axis 421. Here, as shown in the figure, the processor 570 may recognize the reference axis 421 with respect to the center of the side surface, but is not limited thereto. For example, it is also possible to recognize the reference axis 421 with respect to a point on the inner side surface (driver side) or the outer side surface (opposite side of the inner side surface) of the side surface.

[0058] According to still another embodiment, when the camera device 110 is installed on one side of the accelerator pedal, the processor 570 may detect the position of one side surface of the accelerator pedal from the video, although not shown in the figure. Here, the reference axis may be the vertical axis, the horizontal axis, or a combination of the horizontal axis and the vertical axis. More specifically, although not shown in the figure, after recognizing the reference axis from the side surface of the accelerator pedal, the processor 570 may detect the position of the reference axis. Here, the processor 570 may recognize the reference axis with respect to the center of the side surface, or a point on the inner side surface (driver side) or the outer side surface (opposite side of the inner side surface) of the side surface.

[0059] According to some embodiments, indicators (not shown) may be applied to the intersection points with the reference axes 321, 331, 421 on the pedals 320, 330, 420 so that the positions of the reference axes 321, 331, 421 can be easily identified from the pedals 320, 30, 420. Thereby, the processor 570 can recognize the reference axes 321, 331, 421 by identifying the indicators in the videos 310, 410 and detect the positions of the reference axes 321, 331, 421. For example, the indicator may be generated as at least one of a specific pattern, color, or light with respect to the pedals 320, 330, 420, or may be pasted like a stamp.

[0060] Subsequently, at step 940, the electronic device 120 may determine the operating intensity of the pedals 320, 330, 420 based on the position. At this time, the memory 560 may store a database in which different positions of the pedals 320, 330, 420 and different operating intensities of the pedals 320, 330, 420 are mapped to each other. Using such a database, the processor 570 can determine the operating intensity mapped to the position. Specifically, in the database, a plurality of positions and a plurality of operating intensities may be defined in the same number and may be mapped to each other.

[0061] Finally, at step 950, the electronic device 120 may store the operating intensity. Specifically, the processor 570 may store the operating intensity while storing the images 310, 410. At this time, the processor 570 may store each of the images 310, 410 and the operating intensity in at least one of a first memory built into the electronic device 120 or a second memory removable from the electronic device 120. Here, the processor 570 may store the operating intensity corresponding to the shooting time of the images 310, 410. According to one embodiment, the processor 570 may store the operating intensity in the file in which the images 310, 410 are recorded. According to other embodiments, the processor 570 may store the operating intensity in a file separate from the images 310, 410.

[0062] For example, when the files in which videos 310 and 410 are recorded are in the MPEG4 format, the processor 570 may store the operation intensity in the text processing field of the corresponding file. More specifically, an MPEG4 format file has a structure as shown in FIG. 10 and may include a file type box (ftyp), a movie box (moov) for storing metadata for the media, and a media data box (mdat) for storing the media. Here, the movie box may include a movie header box (mvhd), a track box (trak) for storing video metadata and audio metadata respectively, and a user-data box (udta) corresponding to the text processing field. In such a case, while storing videos 310 and 410 in the media data box, the processor 570 may store the operation intensity in the text processing field of the movie box.

[0063] According to various embodiments, the processor 570 may encrypt the operation intensity using a pre-set encryption key and then store it. The encryption key is uniquely set for the electronic device 120 and can also be known to the manufacturer of the electronic device 120. Here, the encryption key may be generated by a combination of the manufacturer of the electronic device 120, the model name, the serial number, etc. As an example, the encryption key may be generated by arithmetic operations on the model name and the last four digits of the serial number. As another example, the encryption key may be generated by arithmetic operations on the manufacturer, the first four digits of the serial number, and the last four digits of the serial number. On the other hand, the processor 570 may encrypt and store videos 310 and 410 using an encryption key that is the same as or different from the encryption key for the operation intensity.

[0064] According to various embodiments, the operation intensity may be displayed only through a predetermined viewer program. The viewer program may decrypt the encrypted operation intensity using an encryption key and then display the decrypted operation intensity. For this purpose, the viewer program is distributed by the manufacturer of the electronic device 120 so as to be able to grasp the encryption key of the electronic device 120. Here, the viewer program may display the videos 310, 410 and the operation intensity, or may display only the operation intensity. For example, the viewer program may be as shown in FIG. 11. In such a case, when the encryption key is input as a password, the viewer program may display at least one of the operation intensities of the videos 310, 410 and the pedals 320, 330, 420 corresponding to the videos 310, 410, that is, the brake pedals 320, 420 or the accelerator pedal 330. When the viewer program is not used, the operation intensity may not be displayed normally, and other values may be displayed, or blurring processing may be performed, or nothing may be displayed.

[0065] According to one embodiment, the processor 570 may store the viewer program in the memory 560 and, in response to an external request, use the viewer program to display the operation intensity through the display module 530. At this time, the processor 570 may decrypt the operation intensity using the encryption key and then display it. According to other embodiments, the processor 570 may transmit the operation intensity to an external device through the communication module 510 or the interface module 540. In such a case, the external device may store the viewer program and use the viewer program to display the operation intensity. According to still other embodiments, when the operation intensity is stored in a removable second memory with respect to the electronic device 120, the second memory may be attached to the external device. In such a case, the external device may store the viewer program and use the viewer program to display the operation intensity.

[0066] According to the present disclosure, the electronic device 120 records the operation intensity of the pedals 320, 330, 420 based on the images 310, 410 of the pedals 320, 330, 420. At this time, the electronic device 120 can record at least one of the operation intensity of the brake pedals 320, 420 or the operation intensity of the accelerator pedal 330. This can be used not only as proof of whether the pedals 320, 330, 420 are operated, but also as materials for proving the operation intensity of the pedals. Therefore, in a situation where it is difficult to investigate the cause, such as an accident suspected of sudden acceleration of the vehicle, the driver's fault can be judged relatively accurately. Here, when recording the operation intensity, the electronic device 120 can increase the reliability of the operation intensity by encryption and prevent indiscriminate sharing.

[0067] In short, the present disclosure provides a vehicle electronic device 120 and its operation method for recording the operation intensity of the pedals 320, 330, 420 based on an image.

[0068] In the present disclosure, the operation method of the electronic device 120 mounted on the vehicle may include a step of analyzing the images 310, 410 of the pedals 320, 330, 420 to detect the positions of the pedals 320, 330, 420 (step 930), a step of determining the operation intensity of the pedals 320, 330, 420 based on the positions (step 940), and a step of storing the operation intensity (step 950).

[0069] According to various embodiments, the electronic device 120 stores a database in which different positions of the pedals 320, 330, 420 and different operation intensities of the pedals 320, 330, 420 are mapped to each other, and the step of determining the operation intensity (step 940) may include a step of using the database to determine the operation intensity mapped to the position.

[0070] According to various embodiments, the step of detecting a position (step 930) may include recognizing preset reference axes 321, 331, 421 for pedals 320, 330, 420 in videos 310, 410, and detecting the positions of the reference axes 321, 331, 421.

[0071] According to various embodiments, the method of operating the electronic device 120 may further include building a database. The step of building the database may include analyzing videos 711, 811 of pedals 720, 730, 820 when the pedals 720, 730, 820 are not being operated to detect a first position of the pedals 720, 730, 820 (step 620), analyzing videos 713, 813 of pedals 720, 730, 820 when the pedals 720, 730, 820 are at maximum operation to detect a second position of the pedals 720, 730, 820 (step 630), defining positions arranged at predetermined intervals from the first position to the second position (step 640), dividing a defined operation intensity range for the pedals 720, 730, 820 by the number of positions to define an operation intensity (step 650), and mapping the positions and the operation intensity to each other and storing them in the database (step 660).

[0072] According to various embodiments, the step of building the database may be executed when the electronic device 120 is installed in the vehicle or when the electronic device 120 is initialized in the vehicle.

[0073] According to various embodiments, the database may be generated for the vehicle model of the vehicle by the vehicle or the manufacturer of the electronic device 120.

[0074] According to various embodiments, the step of storing the operation intensity (step 950) may include at least one of storing the operation intensity in the file in which the videos 310, 410 are recorded or storing the operation intensity in a file different from the videos 310, 410.

[0075] According to various embodiments, the step of storing the operation intensity (step 950) may include the step of encrypting the operation intensity using a preset encryption key and the step of storing the encrypted operation intensity.

[0076] According to various embodiments, the operation intensity may be displayed through a predetermined viewer program.

[0077] According to various embodiments, the step of storing the operation intensity in the file in which the videos 310 and 410 are recorded may include the step of storing the operation intensity in the text processing field of the file when the file in which the videos 310 and 410 are recorded is in the MPEG4 format.

[0078] According to various embodiments, the operation method of the electronic device 120 may be repeatedly performed periodically.

[0079] In the present disclosure, the electronic device 120 mounted on the vehicle includes a memory 560 and a processor 570 configured to be connected to the memory 560 and execute at least one instruction word stored in the memory 560. The processor 570 may analyze the videos 310 and 410 with respect to the pedals 320, 330, and 420 to detect the positions of the pedals 320, 330, and 420, determine the operation intensity of the pedals 320, 330, and 420 based on the positions, and be configured to store the operation intensity in the memory 560.

[0080] According to various embodiments, the memory 560 stores a database in which different positions of the pedals 320, 330, and 420 and different operation intensities of the pedals 320, 330, and 420 are respectively mapped to each other. The processor 570 may be configured to use the database to determine the operation intensity mapped to the position.

[0081] According to various embodiments, the processor 570 may be configured to recognize preset reference axes 321, 331, and 421 for the pedals 320, 330, and 420 in the videos 310 and 410 and detect the positions of the reference axes 321, 331, and 421.

[0082] According to various embodiments, the processor 570 analyzes the images 711, 811 of the pedals 720, 730, 820 when the pedals 720, 730, 820 are not being operated to detect the first positions of the pedals 720, 730, 820, analyzes the images 713, 813 of the pedals 720, 730, 820 when the pedals 720, 730, 820 are being maximally operated to detect the second positions of the pedals 720, 730, 820, defines positions arranged at a predetermined interval from the first position to the second position, divides the operation intensity range set for the pedals by the number of positions to define the operation intensity, and may be configured to construct a database by mapping the positions and the operation intensity to each other.

[0083] According to various embodiments, the processor 570 may be configured to construct a database when the electronic device 120 is mounted on the vehicle or when the electronic device 120 is initialized from the vehicle.

[0084] According to various embodiments, the database may be generated for the vehicle model of the vehicle by the vehicle or the manufacturer of the electronic device 120.

[0085] According to various embodiments, the processor 570 may be configured to store the operation intensity in the file in which the images 310, 410 are recorded or to store the operation intensity in a file separate from the images 310, 410.

[0086] According to various embodiments, the processor 570 may be configured to encrypt the operation intensity using a preset encryption key and store the encrypted operation intensity.

[0087] According to various embodiments, the processor 570 may be configured to decrypt the operation intensity encrypted using the encryption key and display the decrypted operation intensity through a predetermined viewer program.

[0088] According to various embodiments, the processor 570 may be configured to store the operation intensity in the text processing field of the file when the file in which the videos 310 and 410 are recorded is in the MPEG4 format.

[0089] According to various embodiments, the processor 570 may be configured to periodically store the operation intensity of the pedal.

[0090] In the present disclosure, a video processing system 100 mounted on a vehicle includes a camera device 110 that is fixed adjacent to pedals 320, 330, and 420 and is configured to capture videos 310 and 410 with respect to the pedals 320, 330, and 420, and an electronic device 120 that analyzes the videos and detects the positions of the pedals 320, 330, and 420, determines the operation intensity of the pedals 320, 330, and 420 based on the positions, and is configured to store the operation intensity.

[0091] According to various embodiments, the pedals 320, 330, and 420 include at least one of the brake pedals 320 and 420 or the accelerator pedal 330, and the camera device 110 may be installed in front of at least one of the brake pedal 320 or the accelerator pedal 330, or on one side of the brake pedal 420 or the accelerator pedal (not shown).

[0092] According to various embodiments, the electronic device 120 stores a database in which different positions of the pedals 320, 330, and 420 and different operation intensities of the pedals 320, 330, and 420 are respectively mapped to each other, and may be configured to use the database to determine the operation intensity mapped to the position.

[0093] According to various embodiments, the electronic device 120 may be configured to recognize preset reference axes 321, 331, and 421 for the pedals 320, 330, and 420 in the videos 310 and 410 and detect the positions of the reference axes 321, 331, and 421.

[0094] In the present disclosure, a method for operating an image processing system 100 mounted on a vehicle may include a step (step 210) in which a camera device 110 fixed adjacent to a pedal captures images 310 and 410 of the pedals 320, 330, and 420, a step (step 230) in which an electronic device 120 analyzes the images 310 and 410 to detect the positions of the pedals 320, 330, and 420, a step (step 240) in which the electronic device 120 determines the operation intensity of the pedals 320, 330, and 420 based on the positions, and a step (step 250) in which the electronic device 120 stores the operation intensity.

[0095] According to various embodiments, the pedals 320, 330, and 420 may include at least one of the brake pedals 320 and 420 or the accelerator pedal 330, and the camera device 110 may be installed in front of at least one of the brake pedal 320 or the accelerator pedal 330, or may be installed on one side of the brake pedal 420 or the accelerator pedal (not shown).

[0096] According to various embodiments, the electronic device 120 stores a database in which different positions of the pedals 320, 330, and 420 and different operation intensities of the pedals 320, 330, and 420 are respectively mapped to each other, and the step of determining the operation intensity (step 240) may include the step of determining the operation intensity mapped to the position using the database.

[0097] According to various embodiments, the step of detecting the position (step 230) may include the step of recognizing preset reference axes 321, 331, and 421 for the pedals in the images 310 and 410, and the step of detecting the positions of the reference axes 321, 331, and 421.

[0098] FIG. 12 is a block diagram showing a vehicle 2000 equipped with an image processing system 100 in various embodiments. FIG. 13 is a block diagram showing a control device 2100 of the vehicle in FIG. 12.

[0099] Referring to FIGS. 12 and 13, a video processing system 100 according to various embodiments may be mounted on a vehicle 2000, which may include a control device 2100. At this time, the vehicle 2000 may be an autonomous driving vehicle. In some embodiments, at least one component of the video processing system 100 may be integrated with at least one component of the control device 2100.

[0100] The control device 2100 may include a controller 2120 including a memory 2122 and a processor 2124, a sensor 2110, a wireless communication device 2130, a LIDAR 2140, and a camera module 2150.

[0101] The controller 2120 may be configured when manufactured by a vehicle manufacturer, or may be additionally configured to perform an autonomous driving function after manufacture. Alternatively, a configuration for performing continuous additional functions may be included by upgrading the controller 2120 configured at the time of manufacture.

[0102] The controller 2120 may transmit control signals to the sensor 2110, the engine 2006, the user interface (UI) 2008, the wireless communication device 2130, the LIDAR 2140, and the camera module 2150 included as other components in the vehicle. Also, although not shown in the figure, control signals may also be transmitted to an acceleration device, a brake system, a steering device, or a navigation device related to the running of the vehicle.

[0103] The controller 2120 can control the engine 2006. For example, when the vehicle 2000 is running, the controller 2120 can sense the speed limit of the road and control the engine 2006 so that the running speed does not exceed the speed limit, or control the engine 2006 so that the running speed of the vehicle 2000 accelerates within the range not exceeding the speed limit. Additionally, when the environment outside the vehicle is sensed by the sensing modules 2004a, 2004b, 2004c, 2004d and transmitted to the sensor 2110, the controller 2120 can receive this and generate a signal to control the engine 2006 or a steering device (not shown) to control the running of the vehicle.

[0104] When there is another vehicle or an obstacle in front of the vehicle, the controller 2120 can control the engine 2006 or the braking system so that the running vehicle decelerates. In addition to the speed, the controller 2120 can also control the trajectory, the running route, and the steering angle. Alternatively, the controller 2120 can generate necessary control signals according to the recognition information of other external environments such as the running lane and the running signal of the vehicle to control the running of the vehicle.

[0105] In addition to generating its own control signals, the controller 2120 can also control the running of the vehicle by communicating with surrounding vehicles or a central server and sending commands for controlling peripheral devices based on the received information.

[0106] In addition, when the position of the camera module 2150 is changed or the viewing angle is changed, the controller 2120 may not be able to accurately recognize the vehicle or the lane. Therefore, in order to prevent this, a control signal may be generated to control the execution of calibration of the camera module 2150. Accordingly, the controller 2120 causes the camera module 2150 to generate a calibration control signal, so that even if the mounting position of the camera module 2150 is changed due to vibrations or impacts generated by the movement of the autonomous vehicle 2000, the normal mounting position, direction, viewing angle, etc. of the camera module 2150 can be continuously maintained. The controller 2120 may generate a control signal to execute calibration of the camera module 2120 when the initial mounting position, direction, viewing angle information of the camera module 2120 stored in advance and the initial mounting position, direction, viewing angle information of the camera module 2120 measured during the running of the autonomous vehicle 2000 differ by more than a threshold value.

[0107] The controller 2120 may include a memory 2122 and a processor 2124. The processor 2124 may execute the software stored in the memory 2122 according to the control signal of the controller 2120. Specifically, the controller 2120 stores data and instructions for detecting a field-of-view image from the rear image of the vehicle 2000 in the memory 2122, and the instructions may be executed by the processor 2124 to implement one or more methods disclosed herein.

[0108] At this time, the memory 2122 may be stored in a non-volatile recording medium executable by the processor 2124. The memory 2122 may store software and data through appropriate internal and external devices. The memory 2122 may be composed of a RAM (random access memory), a ROM (read only memory), a hard disk, and a memory 2122 device connected to a dongle.

[0109] Memory 2122 may store at least an operating system (OS), user applications, and executable instructions. Memory 2122 may also store application data and array data structures.

[0110] Processor 2124 is a microprocessor or a suitable electronic processor, which may be a controller, a microcontroller, or a state machine.

[0111] Processor 2124 may be implemented by a combination of computing devices, and the computing devices may be a digital signal processor, a microprocessor, or a suitable combination thereof.

[0112] In addition, the control device 2100 may monitor the internal and external characteristics of the vehicle 2000 by at least one or more sensors 2110 to sense the state.

[0113] Sensor 2110 may be composed of at least one or more sensing modules 2004, and the sensing module 2004 may be realized at a specific position of the vehicle 2000 according to the purpose of sensing. It may be located at the lower part, rear end, front end, upper part, or side end of the vehicle 2000, or may be located on internal components or tires of the vehicle.

[0114] Thereby, the sensing module 2004 can sense information related to driving, such as the engine 2006, tires, steering angle, speed, vehicle weight, etc., as internal information of the vehicle. In addition, at least one or more sensing modules 2004 may be composed of an acceleration sensor 2110, a gyroscope, an image sensor 2110, RADAR, an ultrasonic sensor, a LiDAR sensor, etc., and may sense the movement information of the vehicle 2000.

[0115] The sensing module 2004 can receive, as external information, specific data regarding the state of the road on which the vehicle 2000 is located, information on surrounding vehicles, external environmental conditions such as weather, etc., and can also sense vehicle parameters based on this. The sensed information may be stored in the memory 2122 temporarily or long - term according to the purpose.

[0116] The sensor 2110 may integrate and collect the information of the sensing module 2004 for collecting information generated inside and outside the vehicle 2000.

[0117] The control device 2100 may further include a wireless communication device 2130.

[0118] The wireless communication device 2130 is configured to realize wireless communication between vehicles 2000. For example, it enables the vehicle 2000 to communicate with the user's mobile phone, or other wireless communication devices 2130, other vehicles, a central device (traffic control device), a server, etc. The wireless communication device 2130 may transmit and receive wireless signals according to a connection wireless protocol. The wireless communication protocol may be Wi - Fi, Bluetooth, Long - Term Evolution (LTE), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Global Systems for Mobile Communications (GSM), but the communication protocol should not be limited to these.

[0119] In addition, the vehicle 2000 can also realize communication between vehicles through the wireless communication device 2130. That is, the wireless communication device 2130 may execute communication with other vehicles on the road and other vehicles by vehicle-to-vehicle (V2V) communication. The vehicle 2000 may transmit and receive information such as driving warnings and traffic information through vehicle-to-vehicle communication, and it is also possible to request information from other vehicles or receive requests. For example, the wireless communication device 2130 may execute V2V communication with a dedicated short-range communication (DSRC) device or a C-V2V (Cellular-V2V) device. In addition to vehicle-to-vehicle communication, communication between the vehicle and other things (for example, electronic devices carried by pedestrians, etc.) (V2X, Vehicle to Everything communication) may also be realized through the wireless communication device 2130.

[0120] In addition, the control device 2100 may include a LIDAR device 2140. The LIDAR device 2140 may detect objects around the vehicle 2000 during operation using data sensed through the LIDAR sensor. The LIDAR device 2140 transmits the detected information to the controller 2120, and the controller 2120 may operate the vehicle 2000 based on the detected information. For example, when there is a vehicle traveling at a low speed as detected information, the controller 2120 may instruct to reduce the speed of the vehicle through the engine 2006. Or, it may also instruct to reduce the entering speed according to the curvature of the curve that the vehicle enters.

[0121] The control device 2100 may further include a camera module 2150. The controller 2120 may extract object information from the external image captured by the camera module 2150, and the controller 2120 may process the relevant information.

[0122] In addition, the control device 2100 may further include an imaging device for recognizing the external environment. In addition to the LIDAR 2140, RADAR, GPS devices, an odometry device, and other computer vision devices may be used, and these devices may be selected as needed or operate simultaneously to enable more precise sensing.

[0123] The vehicle 2000 may further include a user interface 2008 for user input to the control device 2100 described above. The user interface 2008 may allow the user to input information through appropriate interaction. For example, it may be implemented by a touch screen, keypad, operation buttons, etc. The user interface 2008 may send an input or command to the controller 2120, and the controller 2120 may execute a control operation of the vehicle in response to the input or command.

[0124] Also, the user interface 2008 may be a device external to the vehicle 2000 and may communicate with the vehicle 2000 through the wireless communication device 2130. For example, the user interface 2008 may be able to interact with a mobile phone, tablet, or other computer device.

[0125] Furthermore, although the vehicle 2000 has been described as including the engine 2006, it is also possible to include other types of propulsion systems. For example, the vehicle may be driven by electric energy, or may be driven by a hydrogen energy or a hybrid system combining these. Therefore, the controller 2120 may include a propulsion mechanism corresponding to the propulsion system of the vehicle 2000, and may provide a control signal corresponding thereto to the configuration of each propulsion mechanism.

[0126] Hereinafter, with reference to FIG. 13, the detailed configuration of the control device 2100 of the vehicle 2000 will be described in more detail.

[0127] The control device 2100 includes a processor 2124. The processor 2124 may be a general-purpose single or multi-chip microprocessor, a dedicated microprocessor, a microcontroller, a programmable gate array, or the like. The processor is also called a central processing unit (CPU). Also, the processor 2124 can be used by a combination of a plurality of processors.

[0128] The control device 2100 includes a memory 2122. The memory 2122 may be any electronic component capable of storing electronic information. The memory 2122 may also include a combination of memories in addition to a single memory.

[0129] According to various embodiments, the data and instruction words 2122a of the vehicle 2000 may be stored in the memory 2122. When the processor 2124 executes the instruction words 2122a, all or part of the instruction 2122a and the data 2122b necessary for the execution of the instruction may be loaded 2124a, 2124b onto the processor 2124.

[0130] The control device 2100 may include a transmitter 2130a, a receiver 2130b, or a transceiver 2130c for allowing transmission and reception of signals. One or more antennas 2132a, 2132b may be electrically connected to the transmitter 2130a, the receiver 2130b, or each transceiver 2130c, and may additionally include an antenna.

[0131] The control device 2100 may include a digital signal processor (DSP) 2170. Through the DSP 2170, the vehicle may be enabled to process digital signals quickly.

[0132] The control device 2100 may include a communication interface 2180. The communication interface 2180 may include one or more ports and / or communication modules for connecting other devices to the control device 2100. The communication interface 2180 may enable interaction between the user and the control device 2100.

[0133] The various components of the control device 2100 may be connected via one or more buses 2190, and the bus 2190 may include a power bus, a control signal bus, a status signal bus, a data bus, and the like. According to the control of the processor 2124, the components may transmit mutual information via the bus 2190 and execute the intended functions.

[0134] The devices described above may be implemented by hardware components, software components, and / or combinations of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as processors, controllers, ALUs (arithmetic logic units), digital signal processors, microcomputers, FPGAs (field programmable gate arrays), PLUs (programmable logic units), microprocessors, or various devices capable of executing instructions and responding. The processing device may execute an operating system (OS) and one or more software applications running on the OS. Also, the processing device may access data, record, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, although it may be described that one processing device is used, those skilled in the art will understand that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, the processing device may include multiple processors or one processor and one controller. Also, other processing configurations, such as parallel processors, are possible.

[0135] Software may include a computer program, code, instructions, or a combination of one or more of these, and may configure a processing device to operate as desired or instruct the processing device independently or collectively. Software and / or data may be embodied in any kind of machine, component, physical device, computer recording medium, or device for interpretation by a processing device or for providing instructions or data to the processing device. Software may be distributed and recorded or executed in a distributed state on a computer system connected by a network. Software and data may be recorded on one or more computer-readable recording media.

[0136] The method according to an embodiment may be realized in the form of program instructions executable by various computer means and recorded on a computer-readable medium. At this time, the medium may be one that continuously records a program executable by a computer, or one that temporarily records it for execution or download. Further, the medium may be various recording means or storage means in the form of a combination of single or multiple hardware, and may be not only a medium directly connected to a certain computer system, but also one that is distributed and exists on a network. Examples of the medium include magnetic media such as hard disks, floppy (registered trademark) disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to record program instructions such as ROMs, RAMs, and flash memories. Also, as examples of other media, it includes app stores that distribute applications, and recording media and storage media managed by sites, servers, etc. that supply and distribute other various software.

[0137] The various embodiments of this specification and the terms used herein are not intended to limit the technology described herein to specific embodiments, but should be understood to include various modifications, equivalents, and / or alternatives of the corresponding embodiments. In connection with the description of the drawings, similar reference numerals are used for similar components. Singular expressions may include plural expressions unless the context clearly dictates otherwise. In this specification, expressions such as "A or B", "at least one of A and / or B", "A, B, or C", or "at least one of A, B, and / or C" may include all possible combinations of the listed items. Expressions such as "first", "second", "the first", or "the second" modify the corresponding components regardless of order or importance and are used only to distinguish one component from another and do not limit the component. When a certain (e.g., first) component is described as being "functionally or communicatively coupled" or "connected" to another (e.g., second) component, the certain component may be directly connected to the other component or may be connected via another component (e.g., a third component).

[0138] The term "module" as used herein includes a unit composed of hardware, software, or firmware and may be used interchangeably with terms such as, for example, logic, logic block, component, or circuit. A module may be an integrally configured component or the smallest unit or a part thereof that executes one or more functions. For example, a module may be configured as an ASIC (application-specific integrated circuit).

[0139] According to various embodiments, each of the described components (e.g., modules or programs) may include one or more objects. According to various embodiments, one or more of the above-described components or steps may be omitted, or one or more different components or steps may be added. Alternatively or additionally, a plurality of components (e.g., modules or programs) may be integrated as one component. In such a case, the integrated component may perform one or more functions of each of the plurality of components in the same or similar manner as those performed by the corresponding component among the plurality of components before integration. According to various embodiments, the steps performed by a module, program, or other component may be performed sequentially, in parallel, iteratively, or heuristically, or one or more of the steps may be performed in a different order, omitted, or one or more different steps may be added.

Description of Reference Numerals

[0140] 100: Video processing system 110: Camera device 120: Electronic device

Claims

1. A method for operating an electronic device mounted on a vehicle, comprising: analyzing an image of a pedal to detect a position of the pedal; determining an actuation force of the pedal based on the position; and storing the operation strength Including, A method for operating an electronic device.

2. the electronic device stores a database in which different positions of the pedal and different actuation strengths of the pedal are respectively mapped to each other; The step of determining the operation strength includes: Utilizing the database to determine the manipulation intensity that is mapped to the location. Including, A method for operating the electronic device of claim 1.

3. The step of detecting the position includes: Recognizing a preset reference axis for the pedal in the image; and Detecting the position of the reference axis Including, A method for operating the electronic device of claim 1.

4. Constructing the database Further comprising: The step of constructing the database includes: detecting a first position of the pedal by analyzing an image of the pedal when the pedal is not operated; detecting a second position of the pedal by analyzing an image of the pedal when the pedal is fully operated; defining said positions arranged at predetermined intervals from said first position to said second position; defining said actuation strength by dividing a range of actuation strength defined for said pedal by the number of said positions; and Mapping the positions and the manipulation intensities to each other and storing them in the database. Including, A method for operating an electronic device according to claim 2.

5. The step of constructing the database includes: Executed when the electronic device is loaded onto the vehicle or when the electronic device is initialized on the vehicle. A method for operating an electronic device according to claim 4.

6. the database is generated for a vehicle model by a manufacturer of the vehicle or the electronic device; A method for operating an electronic device according to claim 2.

7. The step of storing the operation strength includes: storing said manipulation intensity in a file in which said image is recorded; or storing said manipulation strength in a file separate from said image; At least one of: A method for operating the electronic device of claim 1.

8. The step of storing the operation strength includes: encrypting the manipulation strength using a preset encryption key; and storing the encrypted operation strength Including, A method for operating the electronic device of claim 1.

9. The operation strength is displayed through a predetermined viewer program. A method for operating the electronic device of claim 1.

10. The step of storing the operation strength in the file in which the image is recorded includes: storing the manipulation strength in a text processing field of the file in which the image is recorded, if the file is in an MPEG4 format; Including, A method for operating an electronic device according to claim 7.

11. The method of operation is repeated periodically. A method for operating the electronic device of claim 1.

12. An electronic device mounted on a vehicle, Memory, and a processor in communication with the memory and configured to execute at least one instruction stored in the memory; The processor, Analyzing an image of the pedal to detect the position of the pedal; determining an operation strength of the pedal based on the position; configured to store the manipulation intensity in the memory; electronic equipment.

13. The memory includes: storing a database in which different positions of the pedal and different actuation strengths of the pedal are respectively mapped to each other; The processor, configured to utilize the database to determine the manipulation intensity mapped to the location; 13. The electronic device of claim 12.

14. The processor, Recognizing a reference axis that is preset with respect to the pedal in the image; configured to detect a position of the reference axis; 13. The electronic device of claim 12.

15. The processor, Analyzing an image of the pedal when the pedal is not being operated to detect a first position of the pedal; detecting a second position of the pedal by analyzing an image of the pedal when the pedal is fully operated; defining a group of positions arranged at predetermined intervals from the first position to the second position; defining the operation strength by dividing an operation strength range determined for the pedal by the number of positions; By mapping the position and the manipulation strength relative to each other, configured to construct the database; 14. The electronic device of claim 13.

16. The processor, configured to build the database when the electronic device is installed in the vehicle or when the electronic device is initialized in the vehicle; 16. The electronic device of claim 15.

17. the database is generated for a vehicle model by a manufacturer of the vehicle or the electronic device; 14. The electronic device of claim 13.

18. The processor, Store the manipulation strength in a file in which the video is recorded, or The operation strength is stored in a file separate from the image.

13. The electronic device of claim 12.

19. The processor, Encrypting the operation strength using a preset encryption key; and configured to store the encrypted operation strength.

13. The electronic device of claim 12.

20. The operation strength is displayed through a predetermined viewer program.

13. The electronic device of claim 12.

21. The processor, When the file in which the image is recorded is in MPEG4 format, the operation strength is stored in a text processing field of the file.

20. The electronic device of claim 18.

22. The processor, configured to periodically store the pedal operation strength; 13. The electronic device of claim 12.

23. An image processing system mounted on a vehicle, a camera device fixed adjacent to the pedal and configured to capture an image of the pedal; and an electronic device configured to analyze the video to detect a position of the pedal, determine an actuation strength of the pedal based on the position, and store the actuation strength. Including, Video processing system.

24. The pedal includes at least one of a brake pedal or an accelerator pedal; The camera device is installed in front of at least one of the brake pedal or the accelerator pedal, or on one side of the brake pedal or the accelerator pedal.

24. The video processing system of claim 23.

25. The electronic device comprises: storing a database in which different positions of the pedal and different actuation strengths of the pedal are respectively mapped to each other; configured to utilize the database to determine the manipulation intensity mapped to the location; 24. The video processing system of claim 23.

26. The electronic device comprises: Recognizing a reference axis that is preset with respect to the pedal in the image; configured to detect a position of the reference axis; 24. The video processing system of claim 23.

27. A method for operating an image processing system mounted on a vehicle, comprising: a camera device fixed adjacent to the pedal capturing an image of the pedal; an electronic device analyzing the image to detect the position of the pedal; the electronic device determining the force of actuation of the pedal based on the position; and the electronic device storing the operation strength. Including, A method for operating a video processing system.

28. The pedal includes at least one of a brake pedal or an accelerator pedal; The camera device is installed in front of at least one of the brake pedal or the accelerator pedal, or on one side of the brake pedal or the accelerator pedal.

28. A method of operating a video processing system according to claim 27.

29. the electronic device stores a database in which different positions of the pedal and different actuation strengths of the pedal are respectively mapped to each other; The step of determining the operation strength includes: Utilizing the database to determine the manipulation intensity that is mapped to the location. Including, 28. A method of operating a video processing system according to claim 27.

30. The step of detecting the position includes: Recognizing a preset reference axis for the pedal in the image; and Detecting the position of the reference axis Including, 28. A method of operating a video processing system according to claim 27.

Images