Server

Abstract

A server according to an embodiment of the present disclosure comprises: a communication unit for receiving event data and image data related to the event data from a vehicle; and a processor for generating context information on the basis of the event data or the image data, and calculating a driving score related to the event data on the basis of the context information. Accordingly, it is possible to provide a driving score on the basis of the event data and the image data.

Description

Server

[0001] The present disclosure relates to a server, and more particularly, to a server capable of providing a driving score based on event data and video data.

[0002] A vehicle is a device that allows the user to move in the desired direction. A representative example is an automobile.

[0003] Meanwhile, research is being conducted to improve driver behavior based on various event information generated in vehicles.

[0004] Prior art, U.S. Patent Publication No. US20150081404, relates to driver behavior improvement based on scoring, and discloses providing a performance risk score indicating an insurance risk level by analyzing stored data.

[0005] However, according to previous literature, there is a shortcoming in that there is no information on driving scores and no provision of information for improving driving habits.

[0006] The problem that the present disclosure seeks to solve is to provide a server capable of providing driving scores based on event data and video data.

[0007] Another problem that the present disclosure seeks to solve is to provide a server capable of providing a driving score based on context information.

[0008] According to one embodiment of the present disclosure for solving the above technical problem, a server includes a communication unit that receives event data and image data related to the event data from a vehicle, a processor that generates context information based on the event data or the image data, and calculates a driving score related to the event data based on the context information.

[0009] Meanwhile, the image data includes external image data of the vehicle and internal image data of the vehicle, and the processor can generate context information based on the external image data of the vehicle and the internal image data of the vehicle, and calculate a driving score related to the event data based on the context information and the event data.

[0010] Meanwhile, the processor can extract driving situation data based on the vehicle's external image data and extract driver status information based on the vehicle's internal image data.

[0011] Meanwhile, the processor may output first result data based on external image data of the vehicle, assign a first time index to the first result data, output second result data based on internal image data of the vehicle, assign a second time index to the second result data, and integrate the first result data and the second result data based on the first time index and the second time index.

[0012] Meanwhile, the processor can generate first level context information based on the external image data of the vehicle and the internal image data of the vehicle before the occurrence of the event data, and can generate second level context information lower than the first level based on the external image data of the vehicle and the internal image data of the vehicle after the occurrence of the event data.

[0013] Meanwhile, the processor can control the level of context information to increase as the duration of an event based on the vehicle's internal image data increases or the number of repetitions increases.

[0014] Meanwhile, the processor can set a level of context information based on the external image data of the vehicle and the internal image data of the vehicle, output the context information as key context information when the level of the context information is higher than a reference level, and calculate a driving score related to the event data based on the key context information and the event data.

[0015] Meanwhile, the processor may adjust a driving score related to event data if multiple events occur within a given period of time and the multiple events overlap at least partially.

[0016] Meanwhile, the processor may calculate a base score based on the event data, calculate a weight based on image data related to the event data, and calculate a driving score related to the event data based on the base score and the weight.

[0017] Meanwhile, the processor may calculate a base score based on the event data, generate context information based on image data related to the event data, calculate a weight based on the context information, and calculate a driving score related to the event data based on the base score and the weight.

[0018] Meanwhile, the processor may calculate a base score based on event data, generate context information based on image data related to the event data, calculate a weight based on key context information among the context information, and calculate a driving score related to the event data based on the base score and the weight.

[0019] Meanwhile, the processor may calculate a first base score based on the sudden brake event data when the event data is sudden brake event data, calculate a first weight when an object cut-in in front of the vehicle is detected based on image data related to the sudden brake event data, and calculate a first driving score related to the sudden brake event data based on the first base score and the first weight.

[0020] Meanwhile, the processor may calculate, based on image data related to the sudden brake event data, that the level of the first weight is lower when an object cut-in in front of the vehicle is detected than when a traffic signal change is detected.

[0021] Meanwhile, if the event data is rapid acceleration event data, the processor may calculate a second base score based on the rapid acceleration event data, calculate a second weight based on image data related to the rapid acceleration event data, and calculate a driving score related to the rapid acceleration event data based on the second base score and the second weight.

[0022] Meanwhile, the processor may calculate, based on image data related to rapid acceleration event data, a higher level of the second weight when driver drowsiness is detected than when a traffic signal change is detected.

[0023] A server according to one embodiment of the present disclosure further includes a memory for storing a driving score history including a driving score related to event data, and the processor can control to transmit the driving score history to an external server when a request for the driving score history is received from the external server.

[0024] A server according to another embodiment of the present disclosure includes a communication unit that receives event data and image data related to the event data from a vehicle, a processor that calculates a base score based on the event data, calculates a weight based on the image data related to the event data, and calculates a driving score related to the event data based on the base score and the weight.

[0025] According to one embodiment of the present disclosure, a server includes a communication unit that receives event data and image data related to the event data from a vehicle, a processor that generates context information based on the event data or the image data, and calculates a driving score related to the event data based on the context information. Accordingly, a driving score can be provided based on the event data and the image data. In particular, a driving score can be provided based on context information based on the event data and the image data.

[0026] Meanwhile, the video data includes external video data and internal video data of the vehicle. The processor can generate context information based on the external video data and internal video data of the vehicle, and calculate a driving score related to the event data based on the context information and event data. Accordingly, a driving score can be provided based on the event data and video data.

[0027] Meanwhile, the processor can extract driving situation data based on the vehicle's external image data and driver status information based on the vehicle's internal image data. Accordingly, a driving score can be provided based on event data and image data.

[0028] Meanwhile, the processor may output first result data based on the vehicle's external image data, assign a first time index to the first result data, output second result data based on the vehicle's internal image data, assign a second time index to the second result data, and integrate the first result data and the second result data based on the first time index and the second time index. Accordingly, a driving score may be provided based on the event data and the image data.

[0029] Meanwhile, the processor can generate first-level context information based on the vehicle's external image data and internal image data prior to the occurrence of the event data, and generate second-level context information lower than the first level based on the vehicle's external image data and internal image data subsequent to the occurrence of the event data. Accordingly, a driving score can be provided based on the event data and image data.

[0030] Meanwhile, the processor can control the level of contextual information to increase as the duration or number of repetitions of an event based on the vehicle's internal video data increases. This allows for a driving score to be provided based on the event data and video data.

[0031] Meanwhile, the processor sets a level of context information based on the vehicle's external image data and the vehicle's internal image data. If the level of context information is higher than a reference level, the processor outputs the context information as key context information. Based on the key context information and event data, the processor can calculate a driving score related to the event data. Accordingly, a driving score can be provided based on the event data and image data.

[0032] Meanwhile, the processor can adjust the driving score associated with the event data if multiple events occur within a given time period and the multiple events overlap at least partially. Accordingly, a driving score can be provided based on the event data and video data.

[0033] Meanwhile, the processor can calculate a base score based on the event data, calculate weights based on image data related to the event data, and calculate a driving score related to the event data based on the base score and weights. Accordingly, a driving score can be provided based on the event data and image data.

[0034] Meanwhile, the processor can calculate a base score based on event data, generate context information based on image data related to the event data, calculate weights based on the context information, and calculate a driving score related to the event data based on the base score and weights. Accordingly, a driving score can be provided based on the event data and image data.

[0035] Meanwhile, the processor can calculate a base score based on event data, generate context information based on image data related to the event data, calculate weights based on key context information among the context information, and calculate a driving score related to the event data based on the base score and weights. Accordingly, a driving score can be provided based on the event data and image data.

[0036] Meanwhile, if the event data is sudden braking event data, the processor may calculate a first base score based on the sudden braking event data, and if an object cut-in in front of the vehicle is detected based on image data related to the sudden braking event data, calculate a first weight, and calculate a first driving score related to the sudden braking event data based on the first base score and the first weight. Accordingly, a driving score may be provided based on the event data and image data.

[0037] Meanwhile, the processor may calculate a lower level of the first weight when a cut-in of an object ahead of the vehicle is detected, based on video data related to the sudden braking event data, than when a traffic signal change is detected. Accordingly, a driving score can be provided based on the event data and video data.

[0038] Meanwhile, if the event data is rapid acceleration event data, the processor may calculate a second base score based on the rapid acceleration event data, calculate a second weight based on image data related to the rapid acceleration event data, and calculate a driving score related to the rapid acceleration event data based on the second base score and the second weight. Accordingly, a driving score can be provided based on the event data and image data.

[0039] Meanwhile, the processor can calculate a higher level of the second weighting factor when driver drowsiness is detected, based on video data related to rapid acceleration event data, than when a traffic signal change is detected. Accordingly, a driving score can be provided based on the event data and video data.

[0040] According to one embodiment of the present disclosure, a server further includes a memory for storing a driving score history including a driving score related to event data, and the processor can control transmission of the driving score history to an external server when a request for the driving score history is received from the external server. Accordingly, a driving score can be provided based on the event data and image data.

[0041] According to another embodiment of the present disclosure, a server includes a communication unit that receives event data and image data related to the event data from a vehicle, and a processor that calculates a base score based on the event data, calculates a weight based on the image data related to the event data, and calculates a driving score related to the event data based on the base score and the weight. Accordingly, a driving score can be provided based on the event data and the image data. In particular, a driving score can be provided based on context information based on the event data and the image data.

[0042] Figure 1 is a diagram illustrating a vehicle system including a vehicle and a server.

[0043] FIG. 2 is a diagram illustrating the architecture of a vehicle signal processing system inside the vehicle of FIG. 1.

[0044] Figure 3a is a drawing showing an example of the arrangement of a vehicle display device inside a vehicle.

[0045] Figure 3b is a drawing showing another example of the arrangement of a vehicle display device inside a vehicle.

[0046] Fig. 4 is an example of an internal block diagram of the vehicle of Fig. 1.

[0047] FIG. 5 is an example of a block diagram of a vehicle control device according to an embodiment of the present disclosure.

[0048] FIG. 6 is an example of a block diagram of a server according to an embodiment of the present disclosure.

[0049] Figures 7 to 17b are drawings referenced in the operation description of Figure 6.

[0050] Hereinafter, the present disclosure will be described in more detail with reference to the drawings.

[0051] The suffixes "module" and "part" used in the following description are given solely for the convenience of writing this specification and do not impart any particularly significant meaning or role to the components themselves. Therefore, the terms "module" and "part" may be used interchangeably.

[0052] Figure 1 is a diagram illustrating a vehicle system including a vehicle and a server.

[0053] Referring to the drawing, the vehicle system (10) includes a vehicle (200) and a server (900) that exchanges vehicle (200) data.

[0054] The vehicle (200) is operated by a plurality of wheels (103FR, 103FL, 103RL, etc.) that rotate by a power source and a steering wheel (150) for controlling the direction of travel of the vehicle (200).

[0055] Meanwhile, the vehicle (200) may further be equipped with a camera (195) for capturing images of the front of the vehicle.

[0056] Meanwhile, the vehicle (200) may be equipped with multiple displays (180a, 180b) for displaying images, information, etc. inside.

[0057] In Fig. 1, a cluster display (180a) and an AVN (Audio Video Navigation) display (180b) are exemplified as multiple displays (180a, 180b). In addition, a HUD (Head Up Display) is also possible.

[0058] Meanwhile, the AVN (Audio Video Navigation) display (180b) may also be called a center information display.

[0059] Meanwhile, the vehicle (200) described in this specification may be a concept that includes all of a vehicle equipped with an engine as a power source, a hybrid vehicle equipped with an engine and an electric motor as a power source, and an electric vehicle equipped with an electric motor as a power source.

[0060] Meanwhile, a server (900) according to one embodiment of the present disclosure receives event data and image data related to the event data from a vehicle, and calculates a driving score related to the event data based on the event data or the image data. Accordingly, a driving score can be provided based on the event data and the image data.

[0061] Various operations of the server (900) according to one embodiment of the present disclosure are described below with reference to FIG. 6 and below.

[0062] FIG. 2 is a diagram illustrating the architecture of a vehicle signal processing system inside the vehicle of FIG. 1.

[0063] Referring to the drawing, the architecture (300a) of the vehicle signal processing system inside the vehicle (200) can correspond to a zone-based architecture.

[0064] Accordingly, sensor devices and processors inside the vehicle may be placed in each of the plurality of zones (Z1 to Z4), and a signal processing device (170a) including a vehicle communication gateway (GWDa) may be placed in the central area of ​​the plurality of zones (Z1 to Z4).

[0065] Meanwhile, the signal processing device (170a) may further include, in addition to the vehicle communication gateway (GWDa), an autonomous driving control module (ACC), a cockpit control module (CPG), etc.

[0066] The vehicle communication gateway (GWDa) within the signal processing device (170a) may be an HPC (High Performance Computing) gateway.

[0067] That is, the signal processing device (170a) of FIG. 2 is an integrated HPC and can exchange data with an external communication module (not shown) or a processor (not shown) within a plurality of zones (Z1 to Z4).

[0068] Figure 3a is a drawing showing an example of the arrangement of a vehicle display device inside a vehicle.

[0069] Referring to the drawing, the interior of the vehicle may be equipped with a cluster display (180a), an AVN (Audio Video Navigation) display (180b), a rear seat entertainment display (180c, 180d), a room mirror display (not shown), etc.

[0070] FIG. 3b is a drawing showing another example of the arrangement of a vehicle display device inside a vehicle.

[0071] A vehicle display device (100) according to an embodiment of the present disclosure may include a plurality of displays (180a to 180b), and a signal processing device (170) that performs signal processing for displaying images, information, etc. on the plurality of displays (180a to 180b) and outputs an image signal to at least one display (180a to 180b).

[0072] Among the plurality of displays (180a to 180b), the first display (180a) may be a cluster display (180a) for displaying driving status, operation information, etc., and the second display (180b) may be an AVN (Audio Video Navigation) display (180b) for displaying vehicle driving information, a navigation map, various entertainment information, or images.

[0073] The signal processing device (170) has a processor (175) therein and can execute a first virtual machine to a third virtual machine (not shown) on a hypervisor (not shown) within the processor (175).

[0074] A second virtual machine (not shown) can operate for the first display (180a), and a third virtual machine (not shown) can operate for the second display (180b).

[0075] Meanwhile, the first virtual machine (not shown) within the processor (175) can control the shared memory (508) based on the hypervisor (505) to be set for the same data transmission to the second virtual machine (not shown) and the third virtual machine (not shown). Accordingly, the same information or the same image can be displayed in synchronization on the first display (180a) and the second display (180b) within the vehicle.

[0076] Meanwhile, the first virtual machine (not shown) within the processor (175) shares at least a portion of data with the second virtual machine (not shown) and the third virtual machine (not shown) for data sharing processing. Accordingly, data can be shared and processed among multiple virtual machines for multiple displays within the vehicle.

[0077] Meanwhile, a first virtual machine (not shown) within a processor (175) may receive and process vehicle wheel speed sensor data, and transmit the processed wheel speed sensor data to at least one of a second virtual machine (not shown) or a third virtual machine (not shown). Accordingly, the vehicle wheel speed sensor data may be shared with at least one virtual machine.

[0078] Meanwhile, the vehicle display device (100) according to the embodiment of the present disclosure may further include a rear seat entertainment display (180c) for displaying driving status information, simple navigation information, various entertainment information, or images.

[0079] The signal processing device (170) can control the RSE display (180c) by executing a fourth virtual machine (not shown) in addition to the first virtual machine to the third virtual machine (not shown) on a hypervisor (not shown) within the processor (175).

[0080] Accordingly, it is possible to control various displays (180a to 180c) using one signal processing device (170).

[0081] Meanwhile, some of the multiple displays (180a~180c) may operate under Linux OS, while others may operate under Web OS.

[0082] The signal processing device (170) according to the embodiment of the present disclosure can control the same information or the same image to be displayed in synchronization on displays (180a to 180c) operating under various operating systems (OS).

[0083] Meanwhile, in FIG. 3b, a vehicle speed indicator (212a) and a vehicle interior temperature indicator (213a) are displayed on a first display (180a), a home screen (222) including a plurality of applications and a vehicle speed indicator (212b) and a vehicle interior temperature indicator (213b) are displayed on a second display (180b), and a second home screen (222b) including a plurality of applications and a vehicle interior temperature indicator (213c) are displayed on a third display (180c).

[0084] Fig. 4 is an example of an internal block diagram of the vehicle of Fig. 1.

[0085] Referring to the drawings, a vehicle (200) according to an embodiment of the present disclosure may include a lamp driving unit (751), a steering driving unit (752), a brake driving unit (753), a power source driving unit (754), a suspension driving unit (756), an air conditioning driving unit (757), a window driving unit (758), a seat driving unit (761), and a signal processing device (170).

[0086] Meanwhile, the vehicle (200) may further include an ECU (770), multiple sensor devices (SN), and multiple communication modules (EMa to EMd).

[0087] Meanwhile, a vehicle (200) according to an embodiment of the present disclosure may further include a vehicle display device (100).

[0088] A vehicle display device (100) according to an embodiment of the present disclosure may include an input unit (110), a communication device (120) for communication with an external device, a plurality of communication modules (EMa to EMd) for internal communication, a memory (140), a signal processing device (170), a plurality of displays (180a to 180c), an audio output unit (185), and a power supply unit (190).

[0089] A plurality of communication modules (EMa to EMd) can be arranged, for example, in a plurality of zones (Z1 to Z4) of FIG. 2, respectively.

[0090] Meanwhile, the signal processing device (170) may have a communication switch (736b) for data communication with each communication module (EM1 to EM4) inside.

[0091] Each communication module (EM1 to EM4) can perform data communication with multiple sensor devices (SN) or ECUs (770) or area signal processing devices (170Z).

[0092] Meanwhile, the plurality of sensor devices (SN) may include a camera (195), a lidar (196), a radar (197), or a position sensor (198).

[0093] The input unit (110) may be equipped with physical buttons, pads, etc. for button input, touch input, etc.

[0094] Meanwhile, the input unit (110) may be equipped with a microphone (not shown) for user voice input.

[0095] The communication device (120) can exchange data wirelessly with a mobile terminal (800) or a server (900).

[0096] In particular, the communication device (120) can wirelessly exchange data with the vehicle driver's mobile terminal. Various data communication methods are possible, such as Bluetooth, WiFi, WiFi Direct, and APiX.

[0097] The communication device (120) can receive weather information, road traffic information, for example, TPEG (Transport Protocol Expert Group) information, from a mobile terminal (800) or a server (900). To this end, the communication device (120) may be equipped with a mobile communication module (not shown).

[0098] Meanwhile, the communication device (120) can exchange data with an adjacent vehicle wirelessly.

[0099] For example, the communication device (120) can exchange vehicle messages with adjacent vehicles wirelessly through V2X (Vehicle-to-everything) communication.

[0100] A plurality of communication modules (EM1 to EM4) can receive sensor data, etc. from an ECU (770), a sensor device (SN), or an area signal processing device (170Z), and transmit the received sensor data to the signal processing device (170).

[0101] Here, the sensor data may include at least one of vehicle direction data, vehicle location data (GPS data), vehicle angle data, vehicle speed data, vehicle acceleration data, vehicle inclination data, vehicle forward / backward data, battery data, fuel data, tire data, vehicle lamp data, vehicle interior temperature data, and vehicle interior humidity data.

[0102] Such sensor data can be obtained from a heading sensor, a yaw sensor, a gyro sensor, a position module, a vehicle forward / backward sensor, a wheel sensor, a vehicle speed sensor, a body tilt detection sensor, a battery sensor, a fuel sensor, a tire sensor, a steering sensor by steering wheel rotation, a vehicle interior temperature sensor, a vehicle interior humidity sensor, etc.

[0103] Meanwhile, the position module may include a GPS module or a position sensor (198) for receiving GPS information.

[0104] Meanwhile, at least one of the plurality of communication modules (EM1 to EM4) can transmit location information data sensed by a GPS module or location sensor (198) to a signal processing device (170).

[0105] Meanwhile, at least one of the plurality of communication modules (EM1 to EM4) can receive vehicle front image data, vehicle side image data, vehicle rear image data, vehicle surrounding obstacle distance information, etc. from a camera (195), lidar (196), radar (197), etc., and transmit the received information to a signal processing device (170).

[0106] The memory (140) can store various data for the overall operation of the vehicle display device (100), such as a program for processing or controlling the signal processing device (170).

[0107] For example, the memory (140) may store data regarding a hypervisor, a first virtual machine, a third virtual machine, or the like, for execution within the processor (175).

[0108] The audio output unit (185) converts an electric signal from the signal processing device (170) into an audio signal and outputs it. For this purpose, a speaker or the like may be provided.

[0109] The power supply unit (190) can supply power required for the operation of each component under the control of the signal processing device (170). In particular, the power supply unit (190) can receive power from a battery or the like inside the vehicle.

[0110] The signal processing device (170) controls the overall operation of each unit in the vehicle display device (100) or the vehicle (200).

[0111] For example, the signal processing device (170) may include a processor (175) that performs signal processing for a vehicle display (180a, 180b).

[0112] The processor (175) can execute a first virtual machine to a third virtual machine (not shown) on a hypervisor (not shown) within the processor (175).

[0113] Among the first virtual machine to the third virtual machine (not shown), the first virtual machine (not shown) may be named a server virtual machine (Server Virtual Maschine), and the second virtual machine to the third virtual machine (not shown) may be named a guest virtual machine (Guest Virtual Maschine).

[0114] For example, a first virtual machine (not shown) within a processor (175) may receive, process, or output sensor data from a plurality of sensor devices, such as vehicle sensor data, location information data, camera image data, audio data, or touch input data.

[0115] In this way, by performing most of the data processing in the first virtual machine (not shown), data sharing in a 1:N manner becomes possible.

[0116] As another example, a first virtual machine (not shown) can directly receive and process CAN data, Ethernet data, audio data, radio data, USB data, and wireless communication data for a second virtual machine or a third virtual machine (not shown).

[0117] And, the first virtual machine (not shown) can transmit processed data to the second virtual machine or the third virtual machine (not shown).

[0118] Accordingly, among the first virtual machine to the third virtual machine (not shown), only the first virtual machine (not shown) receives sensor data, communication data, or external input data from multiple sensor devices and performs signal processing, thereby reducing the signal processing burden on other virtual machines, enabling 1:N data communication, and enabling synchronization when sharing data.

[0119] Meanwhile, the first virtual machine (not shown) can control the second virtual machine (not shown) and the third virtual machine (not shown) to share the same data by writing data to the shared memory (508).

[0120] For example, a first virtual machine (not shown) can record vehicle sensor data, the location information data, the camera image data, or the touch input data in shared memory (508) and control the same data to be shared with a second virtual machine (not shown) and a third virtual machine (not shown). Accordingly, data sharing in a 1:N manner becomes possible.

[0121] Ultimately, by performing most of the data processing on the first virtual machine (not shown), data sharing in a 1:N manner becomes possible.

[0122] Meanwhile, the first virtual machine (not shown) within the processor (175) can control the shared memory (508) based on the hypervisor (505) to be set for the same data transmission to the second virtual machine (not shown) and the third virtual machine (not shown).

[0123] Meanwhile, the signal processing device (170) can process various signals such as audio signals, video signals, and data signals. To this end, the signal processing device (170) can be implemented in the form of a system on chip (SOC).

[0124] FIG. 5 is an example of a block diagram of a vehicle control device according to an embodiment of the present disclosure.

[0125] Referring to the drawing, a vehicle control device (900) according to an embodiment of the present disclosure includes a communication device (120).

[0126] Meanwhile, the vehicle control device (900) according to the embodiment of the present disclosure may further include a signal processing device (170).

[0127] A communication device (120) according to an embodiment of the present disclosure can exchange vehicle messages with an adjacent vehicle in a wireless manner through V2X (Vehicle-to-everything) communication.

[0128] At this time, the communication device (120) according to the embodiment of the present disclosure performs filtering of vehicle messages based on road type information. Accordingly, a driving score can be provided based on event data and image data. In particular, a driving score can be provided based on event data and image data based on road type information.

[0129] Meanwhile, the communication device (120) can transmit the vehicle message passed through filtering to the signal processing device (170). In this way, by filtering the vehicle message, only the necessary vehicle message is transmitted to the signal processing device (170), thereby improving the efficiency of signal processing.

[0130] Meanwhile, the vehicle control device (900) according to the embodiment of the present disclosure may further include at least one display.

[0131] Meanwhile, the vehicle control device (900) according to the embodiment of the present disclosure may further include a steering drive unit (752), a brake drive unit (753), a power source drive unit (754), an ECU (770), or a plurality of sensor devices (SN) of FIG. 4.

[0132] Meanwhile, the vehicle control device (900) according to the embodiment of the present disclosure may further include a lamp driving unit (751), a suspension driving unit (756), an air conditioning driving unit (757), a window driving unit (758), a seat driving unit (761), or a plurality of communication modules (EMa to EMd) of FIG. 4.

[0133] In the drawing, at least one display is illustrated, a cluster display (180a) and an AVN display (180b).

[0134] Meanwhile, the vehicle control device (900) may further include a plurality of area signal processing devices (170Z1 to 170Z4).

[0135] The signal processing device (170) at this time is a high-performance centralized signal processing and control device having multiple CPUs (175), GPUs (178), NPUs (179), etc., and may be called an HPC (High Performance Computing) signal processing device or a central signal processing device.

[0136] A plurality of area signal processing devices (170Z1 to 170Z4) and a signal processing device (170) are connected by wired cables (CB1 to CB4).

[0137] Meanwhile, multiple area signal processing devices (170Z1 to 170Z4) can be connected to each other with wired cables (CBa to CBd).

[0138] The wired cable (CBa~CBd) at this time may include a CAN communication cable, an Ethernet communication cable, or a PCI Express cable.

[0139] Meanwhile, a signal processing device (170) according to an embodiment of the present disclosure may be equipped with at least one processor (175, 178, 177) and a large-capacity storage device (925).

[0140] For example, a signal processing device (170) according to an embodiment of the present disclosure may include a central processor (175, 177), a graphics processor (178), and a neural processor (179).

[0141] Meanwhile, sensor data may be transmitted from at least one of the multiple area signal processing devices (170Z1 to 170Z4) to the signal processing device (170). In particular, the sensor data may be stored in a storage device (925) within the signal processing device (170).

[0142] The sensor data at this time may include at least one of camera data, lidar data, radar data, vehicle direction data, vehicle location data (GPS data), vehicle angle data, vehicle speed data, vehicle acceleration data, vehicle inclination data, vehicle forward / backward data, battery data, fuel data, tire data, vehicle lamp data, vehicle interior temperature data, and vehicle interior humidity data.

[0143] In the drawing, it is exemplified that camera data from a camera (195a) and lidar data from a lidar sensor (196) are input to a first area signal processing device (170Z1), and the camera data and lidar data are transmitted to a signal processing device (170) via a second area signal processing device (170Z2), a third area signal processing device (170Z3), etc.

[0144] Meanwhile, since the data read speed or write speed to the storage device (925) is faster than the network speed when sensor data is transmitted from at least one of the plurality of area signal processing devices (170Z1 to 170Z4) to the signal processing device (170), it is preferable that multi-path routing be performed so that a network bottleneck does not occur.

[0145] To this end, the signal processing device (170) according to the embodiment of the present disclosure can perform multi-path routing based on a Software Defined Network (SDN). Accordingly, a stable network environment can be secured when reading or writing data from the storage device (925). Furthermore, since data can be transmitted to the storage device (925) using multiple paths, the network configuration can be dynamically changed to transmit data.

[0146] Data communication between a plurality of area signal processing devices (170Z1 to 170Z4) and a signal processing device (170) in a vehicle control device (900) according to an embodiment of the present disclosure is preferably Peripheral Component Interconnect Express communication for high-bandwidth, low-latency communication.

[0147] Meanwhile, the signal processing device (170) according to the embodiment of the present disclosure can receive an internal image from an internal camera (195i) and perform signal processing on the internal image.

[0148] Meanwhile, the signal processing device (170) according to the embodiment of the present disclosure can receive a front image from a front camera (195a) and perform signal processing on the front image.

[0149] FIG. 6 is an example of a block diagram of a server according to an embodiment of the present disclosure.

[0150] Referring to the drawings, a server (900) according to an embodiment of the present disclosure includes a communication unit (910) that receives event data and image data related to the event data from a vehicle, and a processor (970) that generates context information based on the event data or the image data and calculates a driving score related to the event data based on the context information.

[0151] Accordingly, driving scores can be provided based on event data and video data. In particular, driving scores can be provided based on contextual information based on event data and video data.

[0152] Meanwhile, the image data may include external image data of the vehicle and internal image data of the vehicle.

[0153] Meanwhile, the processor (970) can generate context information based on the vehicle's external image data and the vehicle's internal image data, and calculate a driving score related to the event data based on the context information and the event data. Accordingly, a driving score can be provided based on the event data and the image data.

[0154] Meanwhile, the processor (970) can extract driving situation data based on the vehicle's external image data and driver status information based on the vehicle's internal image data. Accordingly, a driving score can be provided based on the event data and image data.

[0155] Meanwhile, the processor (970) sets the level of context information based on the vehicle's external image data and the vehicle's internal image data, and if the level of the context information is higher than a reference level, outputs the context information as key context information, and calculates a driving score related to the event data based on the key context information and the event data. Accordingly, a driving score can be provided based on the event data and the image data.

[0156] Meanwhile, the processor (970) can calculate a base score based on event data, calculate a weight based on image data related to the event data, and calculate a driving score related to the event data based on the base score and the weight. Accordingly, a driving score can be provided based on the event data and image data.

[0157] Meanwhile, the processor (970) can calculate a base score based on event data, generate context information based on image data related to the event data, calculate weights based on the context information, and calculate a driving score related to the event data based on the base score and weights. Accordingly, a driving score can be provided based on the event data and image data.

[0158] Meanwhile, the processor (970) may calculate a base score based on event data, generate context information based on image data related to the event data, calculate weights based on key context information among the context information, and calculate a driving score related to the event data based on the base score and weights. Accordingly, a driving score may be provided based on the event data and image data.

[0159] Meanwhile, a server (900) according to one embodiment of the present disclosure may further include a memory (940) that stores a driving score history including a driving score related to event data.

[0160] Meanwhile, when a request for driving score history is received from an external server (900), the processor (970) can control the transmission of the driving score history to the external server (900). Accordingly, a driving score can be provided based on event data and image data.

[0161] A server (900) according to another embodiment of the present disclosure includes a communication unit (910) that receives event data and image data related to the event data from a vehicle, and a processor (970) that calculates a base score based on the event data, calculates a weight based on the image data related to the event data, and calculates a driving score related to the event data based on the base score and the weight.

[0162] Accordingly, driving scores can be provided based on event data and video data. In particular, driving scores can be provided based on contextual information based on event data and video data.

[0163] Figures 7 to 17b are drawings referenced in the operation description of Figure 6.

[0164] Figure 7 is a diagram illustrating an example of the operation of a server based on telematics data and image data.

[0165] Referring to the drawing, a driving vehicle (200) can transmit sensor data from a sensor device (SN) within the vehicle and camera data from a camera (195) to a server (900).

[0166] Telematics data may include sensor data from a sensor device (SN), and image data may include camera data from a camera (195).

[0167] Telematics data (DTb) or sensor data at this time may include vehicle speed data, vehicle direction data, location data such as GPS, vehicle acceleration data, or vehicle brake data.

[0168] Meanwhile, the image data (DTa) may include image data inside the vehicle, image data in front of the vehicle, image data in the rear of the vehicle, or image data on the side of the vehicle.

[0169] The server (900) can receive telematics data (DTb) and image data (DTb) from the vehicle (200).

[0170] Meanwhile, the processor (970) within the server (900) may include a context extractor (971) that generates or extracts context information, and a score calculator (979) that calculates a driving score related to event data based on the context information.

[0171] Meanwhile, the context extractor (971) can generate or extract context information based on image data related to event data from the vehicle.

[0172] Meanwhile, the image data may include external image data of the vehicle and internal image data of the vehicle.

[0173] Meanwhile, the context extractor (971) may include an image calibrator (974) that calibrates the external image data of the vehicle and the internal image data of the vehicle, an external image analyzer (972) that analyzes the external image data of the vehicle, and an internal image analyzer (973) that analyzes the internal image data of the vehicle.

[0174] Meanwhile, the external image analyzer (972) can extract external image data of the vehicle in units of frames and output analysis result data based on learning for each frame image.

[0175] For example, an external image analyzer (972) can extract driving situation data based on external image data of the vehicle.

[0176] In particular, the external image analyzer (972) can perform learning on frame images based on DNN or CNN-based Deep Learning, and output analysis result data based on the learning.

[0177] Meanwhile, an external image analyzer (972) may perform analysis on a frame image of the first frequency and output analysis result data of a second frequency lower than the first frequency.

[0178] For example, the external image analyzer (972) can output result data such as a cut in of a side vehicle, a change in a front traffic signal, bicycle detection, pedestrian detection, or sudden stop of a front vehicle, based on front image data or side image data.

[0179] As another example, the external image analyzer (972) can output result data, such as the approach of a rear vehicle, based on the rear image data.

[0180] Meanwhile, the internal image analyzer (973) can extract the vehicle's internal image data in units of frames and output analysis result data based on learning for each frame image.

[0181] For example, the internal image analyzer (973) can extract driver status information based on the internal image data of the vehicle.

[0182] In particular, the internal image analyzer (973) can perform learning on frame images based on DNN or CNN-based Deep Learning, and output analysis result data based on the learning.

[0183] Meanwhile, the internal image analyzer (973) may perform analysis on a frame image of the first frequency and output analysis result data of a second frequency lower than the first frequency.

[0184] For example, the internal image analyzer (973) can output result data such as drowsiness, distraction, smoking, and calling based on internal image data.

[0185] Meanwhile, the context extractor (971) can generate or extract context information (DTc) based on the result data of the external image analyzer (972) and the internal image analyzer (973), and output the context information (DTc).

[0186] Context information (DTc) at this time may include information such as a side vehicle cutting in, a change in a traffic signal ahead, a bicycle detection, a pedestrian detection, or a sudden stop of a front vehicle, drowsiness, distraction, smoking, or calling.

[0187] Meanwhile, the context extractor (971) can output first result data based on external image data of the vehicle, assign a first time index to the first result data, output second result data based on internal image data of the vehicle, assign a second time index to the second result data, and integrate the first result data and the second result data based on the first time index and the second time index.

[0188] Meanwhile, the context extractor (971) can generate first level context information based on the external image data of the vehicle and the internal image data of the vehicle before the time point of occurrence of the event data, and can generate second level context information lower than the first level based on the external image data of the vehicle and the internal image data of the vehicle after the time point of occurrence of the event data.

[0189] Meanwhile, the context extractor (971) can be controlled to increase the level of context information as the duration of an event based on the vehicle's internal image data increases or the number of repetitions increases.

[0190] Meanwhile, the context extractor (971) can set the level of context information based on the external image data of the vehicle and the internal image data of the vehicle, and if the level of the context information is higher than the reference level, the context information can be output as key context information.

[0191] Next, the score calculator (979) can calculate a driving score related to the event data based on context information (DTc) from the context extractor (971).

[0192] Meanwhile, the score calculator (979) may include a multiplexer (976) that fuses or synthesizes telematics data (DTb) and context information (DTc), a model verifier (977) that performs model verification based on data from the multiplexer (976), and a risk analyzer (978) that performs risk analysis based on driving factors.

[0193] Meanwhile, the score calculator (979) can calculate a driving score related to event data based on telematics data (DTb), context information (DTc), model verification data, risk analysis data, etc.

[0194] Meanwhile, telematics data (DTb) may include vehicle speed data, vehicle direction data, location data such as GPS, vehicle acceleration data, or vehicle brake data.

[0195] That is, telematics data (DTb) may be event data related to the vehicle.

[0196] Meanwhile, the score calculator (979) within the processor (970) can calculate a driving score related to event data based on context information.

[0197] Specifically, a score calculator (979) within the processor (970) can calculate a driving score related to event data based on context information and event data.

[0198] Meanwhile, the score calculator (979) within the processor (970) can calculate a driving score related to the event data based on the key context information and the event data.

[0199] Meanwhile, the score calculator (979) within the processor (970) can adjust the driving score related to the event data when there are multiple events occurring within a predetermined time and the multiple events overlap at least partially.

[0200] Meanwhile, the score calculator (979) within the processor (970) can calculate a base score based on event data, calculate a weight based on image data related to the event data, and calculate a driving score related to the event data based on the base score and the weight. Accordingly, a driving score can be provided based on the event data and image data.

[0201] Meanwhile, the score calculator (979) within the processor (970) can calculate a base score based on event data, generate context information based on image data related to the event data, calculate weights based on the context information, and calculate a driving score related to the event data based on the base score and weights. Accordingly, a driving score can be provided based on the event data and image data.

[0202] Meanwhile, the score calculator (979) within the processor (970) can calculate a base score based on event data, generate context information based on image data related to the event data, calculate weights based on key context information among the context information, and calculate a driving score related to the event data based on the base score and weights. Accordingly, a driving score can be provided based on the event data and image data.

[0203] Meanwhile, the score calculator (979) within the processor (970) may calculate a first base score based on the sudden braking event data when the event data is sudden braking event data, calculate a first weight based on image data related to the sudden braking event data when an object cut-in in front of the vehicle is detected, and calculate a first driving score related to the sudden braking event data based on the first base score and the first weight. Accordingly, a driving score may be provided based on the event data and the image data.

[0204] Meanwhile, the score calculator (979) within the processor (970) may calculate a lower level of the first weight when an object cut-in ahead of the vehicle is detected, based on image data related to the sudden braking event data, than when a traffic signal change is detected. Accordingly, a driving score can be provided based on the event data and image data.

[0205] Meanwhile, the score calculator (979) within the processor (970) can calculate a second base score based on the sudden acceleration event data when the event data is sudden acceleration event data, calculate a second weight based on image data related to the sudden acceleration event data, and calculate a driving score related to the sudden acceleration event data based on the second base score and the second weight. Accordingly, a driving score can be provided based on the event data and image data.

[0206] Meanwhile, the score calculator (979) within the processor (970) may calculate a second weighting level to be higher when driver drowsiness is detected than when a traffic signal change is detected based on image data related to rapid acceleration event data. Accordingly, a driving score can be provided based on the event data and image data.

[0207] Meanwhile, when a request for driving score history is received from an external server (1000a), the processor (970) can control the transmission of the driving score history to the external server (1000a).

[0208] Alternatively, when a request for driving context information is received from the processor (970) and an external server (1000a), the processor (970) can control driving factor information and driving context information (DTd) to be transmitted to the external server (1000a).

[0209] Meanwhile, the external server (1000a) may be an automobile insurance related server.

[0210] Meanwhile, the external server (1000a) can exchange data with a separate DOI server (1000b). In particular, the external server (1000a) can exchange certification data with the separate DOI server (1000b).

[0211] Referring to FIG. 7, the server (900) can calculate a driving score based on event data and image data, and provide the calculated driving score to an external server (1000a), etc.

[0212] By calculating driving scores based on event and video data, more accurate driving scores can be calculated in situations such as sudden braking or rapid acceleration. This allows for improved driving scores compared to those generated solely based on event data.

[0213] Figure 8 is a diagram illustrating another example of server operation based on telematics data and image data.

[0214] Referring to the drawing, the telematics data may include vehicle speed data (DTba), vehicle acceleration data (DTbb), and vehicle brake data (DTbc).

[0215] The video data may include vehicle exterior video data (DTa1, DTa3) and vehicle interior video data (DTa2, DTa4).

[0216] Meanwhile, the server (900) can receive telematics data and image data from at least one vehicle.

[0217] For example, the server (900) can receive telematics data (DTba~DTbc) and image data (DTa1, DTa2) from the first vehicle.

[0218] As another example, the server (900) can receive telematics data and image data (DTa2, DTa) from a second vehicle.

[0219] Meanwhile, a processor (970) within a server (900) may include a context extractor (971) that generates or extracts context information based on image data (DTa1 to DTa4), and a score calculator (979) that calculates a driving score related to event data based on telematics data.

[0220] Meanwhile, the context extractor (971) may include a preprocessing unit (974) that preprocesses image data, an external image analyzer (972) that analyzes external image data of the vehicle, and an internal image analyzer (973) that analyzes internal image data of the vehicle.

[0221] Meanwhile, the preprocessing unit (974) can correspond to the image calibrator (974) of FIG. 8.

[0222] Meanwhile, the score calculator (979) can calculate a base score based on event data (982) among telematics data.

[0223] For example, the score calculator (979) may calculate a base score based on a rapid acceleration event or a base score based on a harsh braking event, based on telematics data.

[0224] As another example, the score calculator (979) may calculate a base score based on a bump event based on telematics data.

[0225] In the drawing, the base score based on a bump event is 65, the base score based on a rapid acceleration event or a harsh braking event is 70.

[0226] That is, the score calculator (979) can set the level of the base score based on a rapid acceleration event or a harsh braking event to be higher than the base score based on a bump event. Accordingly, a driving score for safe driving can be calculated.

[0227] Meanwhile, the context extractor (971) can generate context information (984) based on image data related to event data (982).

[0228] For example, the context extractor (971) can generate context information such as no surrounding vehicles, curved roads, cross lanes, off-roads, and advertising billboards based on external image data.

[0229] Meanwhile, the context extractor (971) can generate context information such as driver head bowing, food intake, and looking at the front lane based on internal image data.

[0230] As another example, the context extractor (971) can generate context information such as traffic congestion, straight road, stop, cut-off, clear, etc. based on external image data.

[0231] Meanwhile, the context extractor (971) can generate context information such as looking at the front lane and hands on the steering wheel based on internal image data.

[0232] Meanwhile, the score calculator (979) within the processor (970) can calculate a driving score (986) related to the event data based on the context information (984) and the event data (982).

[0233] Meanwhile, the score calculator (979) within the processor (970) can calculate a base score based on event data, generate context information based on image data related to the event data, calculate weights based on the context information, and calculate a driving score related to the event data based on the base score and weights. Accordingly, a driving score can be provided based on the event data and image data.

[0234] For example, a score calculator (979) within a processor (970) may calculate a base score based on a bump event based on telematics data (DTba to DTbc) from a first vehicle, calculate weights based on contextual information such as off-road information, billboard information, or forward lane attention based on image data before or after the bump event, and calculate a driving score related to the event data based on the base score and the weights.

[0235] Specifically, the score calculator (979) within the processor (970) can calculate a driving score of 70 levels by adding a 65-level base score based on a bump event and a 5-level weight based on contextual information such as off-road information, billboard information, or forward lane attention.

[0236] As another example, the score calculator (979) within the processor (970) may calculate a base score based on a harsh breaking event based on telematics data from the second vehicle, calculate weights based on contextual information such as cut-off and clear based on video data before or after the harsh breaking event, and calculate a driving score related to the event data based on the base score and weights.

[0237] Specifically, the score calculator (979) within the processor (970) can calculate a driving score of level 95 by adding a 25-level weight based on contextual information such as cut-off and clear to a 70-level base score based on a sudden braking event.

[0238] That is, the score calculator (979) within the processor (970) can control the weight to increase and ultimately increase the driving score by judging that the driver's forward gaze level decreases based on the vehicle's internal image data, thereby indicating the driver's carelessness.

[0239] Meanwhile, the higher the final confirmed driving score, the higher the vehicle insurance premium determined by an external server (1000a) such as an insurance company may be.

[0240] Referring to FIG. 8, the processor (970) may calculate a base score based on event data based on telematics data, match the event data with video data related to the event data, calculate a weight based on the video data related to the event data, and calculate a driving score related to the event data based on the base score and the weight. Accordingly, a driving score may be provided based on the event data and video data. In particular, a driving score suitable for the driver's driving situation may be provided.

[0241] FIG. 9 is an example of an internal block diagram of a processor within a server according to an embodiment of the present disclosure.

[0242] Referring to the drawing, the vehicle (200) can transmit event data from a sensor device (SN), internal image data of the vehicle from an internal camera (195i) of the vehicle, and external image data of the vehicle from an external camera (195a) of the vehicle to the server (900).

[0243] Accordingly, the communication unit (910) of the server (900) can receive event data and image data related to the event data from the vehicle (200).

[0244] For example, the communication unit (910) of the server (900) can receive video data related to event data in the form of a video clip, etc.

[0245] The memory (940) of the server (900) can store event data and image data related to the event data.

[0246] Meanwhile, the processor (970) of the server (900) generates context information based on event data and image data related to the event data, and calculates a driving score related to the event data based on the context information.

[0247] To this end, the processor (970) of the server (900) may include a context extractor (971) that generates or extracts context information, and a score calculator (979) that calculates a driving score related to event data based on the context information.

[0248] Meanwhile, the context extractor (971) can extract driving situation data based on the vehicle's external image data and driver's status information based on the vehicle's internal image data, and generate or output context information based on the driving situation data and the driver's status information.

[0249] To this end, the context extractor (971) may be equipped with an external image analyzer (972) that analyzes external image data of the vehicle, an internal image analyzer (973) that analyzes internal image data of the vehicle, an integrator (967) that synchronizes internal image data and external image data, and a context evaluator (969) that generates context information and evaluates the influence of the context information based on data from the integrator (967).

[0250] Meanwhile, the integrator (967) can output first result data based on external image data of the vehicle, assign a first time index to the first result data, output second result data based on internal image data of the vehicle, assign a second time index to the second result data, and integrate the first result data and the second result data based on the first time index and the second time index.

[0251] Meanwhile, a context evaluator (969) can generate first-level context information based on the external image data of the vehicle and the internal image data of the vehicle before the occurrence of the event data, and can generate second-level context information lower than the first level based on the external image data of the vehicle and the internal image data of the vehicle after the occurrence of the event data.

[0252] Meanwhile, the context evaluator (969) can control the level of context information to increase as the duration of an event based on the vehicle's internal image data increases or the number of repetitions increases.

[0253] Meanwhile, the context evaluator (969) can set the level of context information based on the external image data of the vehicle and the internal image data of the vehicle, and if the level of the context information is higher than the reference level, the context information can be output as key context information.

[0254] Meanwhile, the context evaluator (969) can adjust the driving score related to the event data if there are multiple events occurring within a given time period and the multiple events overlap at least partially.

[0255] Meanwhile, the score calculator (979) can calculate a driving score related to event data based on context information.

[0256] Meanwhile, the score calculator (979) can calculate a base score based on event data, calculate a weight based on image data related to the event data, and calculate a driving score related to the event data based on the base score and the weight.

[0257] Meanwhile, the score calculator (979) can calculate a base score based on event data, calculate a weight based on key context information related to the event data, and calculate a driving score related to the event data based on the base score and the weight.

[0258] For example, if the event data is sudden braking event data, the score calculator (979) may calculate a first base score based on the sudden braking event data, calculate a first weight based on image data related to the sudden braking event data, and calculate a first driving score related to the sudden braking event data based on the first base score and the first weight.

[0259] Meanwhile, the score calculator (979) can calculate a lower level of the first weight when an object cut-in ahead of the vehicle is detected, based on image data related to brake event data, than when a traffic signal change is detected. Accordingly, a driving score can be provided based on the event data and image data.

[0260] As another example, when the event data is rapid acceleration event data, the score calculator (979) may calculate a second base score based on the rapid acceleration event data, calculate a second weight based on image data related to the rapid acceleration event data, and calculate a driving score related to the rapid acceleration event data based on the second base score and the second weight.

[0261] Meanwhile, the score calculator (979) can calculate a higher level of the second weight when driver drowsiness is detected, based on video data related to rapid acceleration event data, than when a traffic signal change is detected. Accordingly, a driving score can be provided based on the event data and video data.

[0262] FIG. 10 is another example of an internal block diagram of a processor within a server according to an embodiment of the present disclosure.

[0263] Referring to the drawing, the processor (970) in the server (900b) of FIG. 10 differs from the processor (970) of FIG. 9 in that it further includes a driving score calculator (991), a driving score history calculator (992), a long-term score calculator (994), and a short-term score calculator (996). Hereinafter, the differences will be mainly described.

[0264] Meanwhile, the score calculator (979) can receive ID information based on driver profile, driving route, etc., and provide a driving score based on ID information.

[0265] Meanwhile, the driving score calculator (991) calculates the driving score based on the event-based driving score from the score calculator (979).

[0266] For example, when multiple events occur during driving, the driving score calculator (991) can calculate a driving score based on multiple driving scores from the score calculator (979).

[0267] Specifically, the driving score calculator (991) can calculate a driving score by adding up multiple driving scores from the score calculator (979).

[0268] Meanwhile, the calculated driving score can be stored in memory (940).

[0269] Next, the driving score history calculator (992) can calculate the driving score history based on the calculated driving score.

[0270] For example, the driving score history calculator (992) can calculate the driving score history based on multiple driving scores over a predetermined period of time.

[0271] Meanwhile, the calculated driving score history can be stored in memory (940).

[0272] Meanwhile, the long-term score calculator (994) can calculate the driving score for the first period based on the driving score history at the request of an external server (1000a) or the like. Accordingly, data corresponding to the request of the external server (1000a) or the like can be calculated and output.

[0273] Meanwhile, the long-term score calculator (994) can calculate the driving score for the first period based on server data such as an external server (1000a), accident statistics data, etc., in addition to the driving score history.

[0274] Meanwhile, the short-term score calculator (996) can calculate a driving score for a second period shorter than the first period based on the driving score history at the request of an external server (1000a) or the like. Accordingly, data corresponding to the request of the external server (1000a) or the like can be calculated and output.

[0275] Figure 11 is a drawing referenced in the description of the operation of the context extractor of Figure 9.

[0276] Referring to the drawing, the context extractor (971) may include an external image analyzer (972) that analyzes external image data of the vehicle, an internal image analyzer (973) that analyzes internal image data of the vehicle, an integrator (967) that synchronizes internal image data and external image data, and a context evaluator (969).

[0277] Meanwhile, an external image analyzer (972) or an internal image analyzer (973) can perform learning on a frame image based on DNN or CNN-based Deep Learning, and output analysis result data based on the learning.

[0278] Meanwhile, an external image analyzer (972) or an internal image analyzer (973) can provide frame-by-frame analysis result data for input image data.

[0279] Meanwhile, an external image analyzer (972) or an internal image analyzer (973) can perform sampling on image frame data. For example, the external image analyzer (972) or the internal image analyzer (973) can sample image data at 30 fps and output image data at 15 fps.

[0280] Meanwhile, the external image analyzer (972) or the internal image analyzer (973) can provide the output as a list of situation information for each frame.

[0281] Meanwhile, the integrator (967) can wait until it receives both the first result data from the external image analyzer (972) and the second result data from the internal image analyzer (973).

[0282] Meanwhile, the integrator (967) can receive first result data based on external image data of the vehicle from an external image analyzer (972) and assign a first time index to the first result data.

[0283] Meanwhile, the integrator (967) can receive second result data based on the internal image data of the vehicle from the internal image analyzer (973) and assign a second time index to the second result data.

[0284] For example, the integrator (967) can assign a first time index or a second time index at 0.5 second intervals for the first result data or the second result data based on 30 fps.

[0285] And, the integrator (967) can integrate the first result data and the second result data based on the first time index and the second time index.

[0286] For example, the integrator (967) can integrate the first result data and the second result data based on the time index.

[0287] Meanwhile, the integrator (967) can generate a time table based on the analysis result data of the external image data of the vehicle that occurred before or after the event data and the analysis result data of the internal image data of the vehicle.

[0288] Meanwhile, the integrator (967) can create a table time table for each event data when there are multiple event data in one image data.

[0289] Meanwhile, the context evaluator (969) can calculate weight calculations related to influence or key context information for an event based on the type, occurrence time, frequency, or intensity of context information based on the analysis result data of the vehicle's external image data and the analysis result data of the vehicle's internal image data.

[0290] Meanwhile, the context evaluator (969) may also generate a situation description script using key context information.

[0291] Meanwhile, the context evaluator (969) can analyze the analysis result data of the vehicle's external image data and the analysis result data of the vehicle's internal image data by type and set the time interval and influence.

[0292] For example, the context evaluator (969) can analyze the drowsiness context by setting a relatively wide time period.

[0293] Meanwhile, the context evaluator (969) can set the influence area and weight of the context based on the influence table.

[0294] Meanwhile, the context evaluator (969) can calculate the influence of the context on the event.

[0295] Meanwhile, the context evaluator (969) can give a higher weight to a context that occurred at a previous time closer to the event.

[0296] For example, in the case of vehicle interior image data, the context evaluator (969) may assign a higher weight to a longer duration and a higher degree of repetition in the case of drowsiness, distraction, or behavior.

[0297] As another example, the context evaluator (969) may assign a higher weight to the front image data of the vehicle as the distance to the vehicle (200) becomes closer.

[0298] Meanwhile, the context evaluator (969) can normalize the weights by context type and derive contexts that are above a certain standard as key contexts.

[0299] Figure 12a is a flowchart showing an example of the operation of the external image analyzer, internal image analyzer, and integrator of Figure 11.

[0300] Referring to the drawing, the external image analyzer (972) and the internal image analyzer (973) perform analysis based on the external image data of the vehicle and the internal image data of the vehicle (S1110).

[0301] Meanwhile, the integration device (967) can determine whether analysis is complete based on the vehicle's external image data and the vehicle's internal image data (S1115).

[0302] For example, the integrator (967) may wait until it receives both the first result data from the external image analyzer (972) and the second result data from the internal image analyzer (973).

[0303] Meanwhile, the integrator (967) converts the time index-based context when analysis is completed based on the vehicle's external image data and the vehicle's internal image data (S1120).

[0304] Meanwhile, when the integrator (967) receives both the first result data from the external image analyzer (972) and the second result data from the internal image analyzer (973), it determines that the analysis is complete and can convert the unit context of the frame image data into a time unit.

[0305] For example, the integrator (967) can change the results of multiple frame image data into a single time unit. At this time, methods such as Max, average, moving average, and Weight sum can be used.

[0306] Next, the integrator (967) can detect the first event data (S1125) and create a time table for the first event data (S1130).

[0307] Next, the integrator (967) determines whether the creation of a time table for all event data is completed (S1135), and if not, detects the next event data (S1137) and creates a time table for the next event data.

[0308] Next, the integrator (967) can create a time table list having multiple time tables when the creation of time tables for all event data is completed (S1140).

[0309] Figure 12b illustrates an example of a time table list for event data.

[0310] Referring to the drawing, the integrator (967) can generate a time table based on the internal image data of the vehicle.

[0311] In the drawing, the time table list is illustrated to include a drowsiness event and a calling event corresponding to time indices from 0 to 59.

[0312] Meanwhile, unlike the drawing, the integrator (967) may also generate a separate time table list based on the external image data of the vehicle.

[0313] Figure 12c illustrates another example of a time table list for event data.

[0314] In the drawing, the integrator (967) can change the time index based on the event occurrence time.

[0315] That is, the time table list may include drowsiness events, calling events, and cut-in events corresponding to time indices of -30 to 29.

[0316] In this way, when changing the time index based on the event occurrence time, it corresponds to the timestamp when the actual event occurred.

[0317] Figure 13a is a flowchart showing an example of the operation of the context evaluator of Figure 11.

[0318] Referring to the drawing, the context evaluator (969) receives a time table list from the integrator (967) (S1210).

[0319] Next, the context evaluator (969) extracts a first time table from the time table list (S1215) and calculates a context score based on the first time table (S1220).

[0320] For example, the context evaluator (969) can calculate a context score based on the context influence table of the first time table.

[0321] Next, the context evaluator (969) can add the context score as a key context if the context score is greater than or equal to the reference value (S1227).

[0322] Meanwhile, the context evaluator (969) determines whether the score calculation of all time tables in the time table list has been completed (S1230), and if not, extracts the next table (S1232) and calculates the context score for the next table.

[0323] Meanwhile, the context evaluator (969) can generate or output a key context list including the key context when the score calculation of all time tables in the time table list is completed (S1235).

[0324] Figure 13b illustrates an example of a context influence table.

[0325] Referring to the drawing, the context influence table may include values ​​for a drowsiness event and values ​​for a calling event, respectively, corresponding to contexts of -30 to 90.

[0326] Meanwhile, the context evaluator (969) can perform normalization on values ​​in a drowsiness event or a calling event, etc. Accordingly, it is possible to evaluate key contexts based on the same reference value for different events.

[0327] Meanwhile, the context evaluator (969) can extract key contexts using different criteria when normalization is not performed on values ​​in a drowsiness event or values ​​in a calling event.

[0328] Meanwhile, the context evaluator (969) may add separate weights to values ​​in a drowsiness event or values ​​in a calling event.

[0329] For example, the context evaluator (969) can evaluate the context by weighting the number of times an event occurs repeatedly.

[0330] Meanwhile, the context evaluator (969) can add various weights depending on the context, such as the number of pedestrians, distance from the preceding vehicle, and duration.

[0331] Figure 14a is a flowchart showing an example of the operation of the score calculator of Figure 10.

[0332] Referring to the drawing, the score calculator (979) receives a key context list from the context evaluator (969) (S1410).

[0333] Next, the score calculator (979) calculates an event score for each key context in the key context list (S1415).

[0334] For example, the score calculator (979) can calculate a base score based on event data for each key context.

[0335] Meanwhile, the score calculator (979) can calculate a weight based on image data related to the event data, and calculate a driving score related to the event data based on the base score and the weight.

[0336] Next, the score calculator (979) checks the dependency between events and corrects the event score (S1420).

[0337] Specifically, the score calculator (979) can check or evaluate overlapping relationships between events, etc., when multiple events occur within a certain time interval.

[0338] In addition, the score calculator (979) can correct the driving score by taking into account the overlap when there is overlap between multiple events.

[0339] For example, the core operator (979) may lower the weight if there is overlap between multiple events.

[0340] Finally, the score calculator (979) can calculate and output a driving score for each event based on the correction of the event score.

[0341] Next, the score calculator (979) generates a description script for each event (S1425).

[0342] Figure 14b illustrates an example of weights for each event.

[0343] Referring to the drawing, the score calculator (979) can set weights of 1, 0.5, 5, 2, etc. for events such as sudden braking, sudden acceleration, collision, and speeding, respectively.

[0344] Meanwhile, the score calculator (979) can calculate a driving score related to event data by multiplying the base score by the weight of FIG. 14b.

[0345] Accordingly, among the driving scores for events such as sudden braking, sudden acceleration, collision, and speeding, the driving score may be the highest because the weight of the collision is the highest at 5, followed by the weight of speeding at 2, so the driving score may be the next highest, and the driving score may be the lowest because the weight of sudden acceleration is the lowest at 0.5.

[0346] Figure 14c illustrates another example of weights for each event.

[0347] Referring to the drawing, the score calculator (979) can set weights for each of cut-in, distraction, drowsiness, phone, smoking, sudden stop of the vehicle in front, speeding, etc.

[0348] Meanwhile, the score calculator (979) can set the weight as the average of two combinations when there are multiple contexts.

[0349] Meanwhile, the score calculator (979) can generate a template-based script according to a combination of events and contexts.

[0350] For example, a script could be exemplified as "Acceleration occurs while drowsy."

[0351] Figure 15a illustrates an example of context based on front image data of a vehicle.

[0352] Referring to the drawing, a processor (970) within a server (900) can calculate a graph (GRaa) related to a cut-in event, a graph (GRab) related to a forward approach event, a graph (GRac) related to a pedestrian event, and a graph (GRad) related to a traffic signal event based on front image data (1510) of a vehicle.

[0353] That is, the processor (970) within the server (900) can synchronize and calculate context related to a cut-in event, context related to a forward approach and event, context related to a pedestrian event, and context related to a traffic signal event based on the front image data (1510) of the vehicle.

[0354] Figure 15b illustrates an example of context based on interior image data of a vehicle.

[0355] Referring to the drawing, a processor (970) within a server (900) can calculate a graph (GRba) related to a distraction event, a graph (GRbb) related to a drowsiness event, a graph (GRbc) related to sleep, and a graph (GRbd) related to calling or drinking or smoking based on the vehicle's internal image data (1520).

[0356] That is, the processor (970) within the server (900) can synchronize and compute context related to a distraction event, context related to a drowsiness event, context related to sleep, and context related to calling, drinking, or smoking based on the vehicle's internal image data (1520).

[0357] Figure 15c illustrates a context that synthesizes the context of Figure 15a and the context of Figure 15b.

[0358] Referring to the drawing, a processor (970) within a server (900) can calculate cut-in events, forward approach events, pedestrian events, traffic signal events, etc., based on front image data (1510) of the vehicle and internal image data (1520) of the vehicle, by time.

[0359] Figure 16a illustrates an example of sudden braking.

[0360] Referring to the drawing, a processor (970) within a server (900) can calculate a driving score for a sudden braking event based on sensor data related to sudden braking, forward image data (1610) of the vehicle, and internal image data (1620) of the vehicle.

[0361] Meanwhile, the processor (970) within the server (900) may calculate a base score based on sensor data related to sudden braking, and set the weight to '1' when there is no vehicle ahead in the vehicle's front image data (1610) and the driver is looking ahead in the vehicle's internal image data (1620).

[0362] Accordingly, the processor (970) within the server (900) can calculate a base score based on sensor data related to sudden braking as a driving score related to sudden braking.

[0363] Figure 16b illustrates another example of sudden braking.

[0364] Referring to the drawing, a processor (970) within a server (900) can calculate a driving score for a sudden braking event based on sensor data related to sudden braking, forward image data (1610b) of the vehicle, and internal image data (1620) of the vehicle.

[0365] Meanwhile, the processor (970) within the server (900) may calculate a base score based on sensor data related to sudden braking, and set the weight to '0.5' when the front vehicle (1622) is cut in (1624) in the front image data (1610b) of the vehicle and the driver is looking forward in the internal image data (1620) of the vehicle.

[0366] Accordingly, the processor (970) within the server (900) can calculate half of the base score based on sensor data related to sudden braking as a driving score related to sudden braking.

[0367] That is, according to Figure 16b, compared to Figure 16a, the driving score associated with sudden braking is lowered due to the preceding vehicle cutting in. Consequently, the driving score can be calculated in response to the driving situation.

[0368] Meanwhile, the processor (970) within the server (900) can generate a description script (1626) for a driving score related to sudden braking.

[0369] Figure 16c illustrates an example of rapid acceleration.

[0370] Referring to the drawing, a processor (970) within a server (900) can calculate a driving score for a rapid acceleration event based on sensor data related to rapid acceleration, forward image data (1630) of the vehicle, and internal image data (1640) of the vehicle.

[0371] Meanwhile, the processor (970) within the server (900) may calculate a base score based on sensor data related to rapid acceleration, and set the weight to '1' when there is no vehicle ahead in the vehicle's front image data (1630) and the driver is looking ahead in the vehicle's internal image data (1640).

[0372] Accordingly, the processor (970) within the server (900) can calculate a base score based on sensor data related to rapid acceleration into a driving score related to rapid acceleration.

[0373] Figure 16d illustrates another example of rapid acceleration.

[0374] Referring to the drawing, a processor (970) within a server (900) can calculate a driving score for a rapid acceleration event based on sensor data related to rapid acceleration, forward image data (1630) of the vehicle, and internal image data (1640) of the vehicle.

[0375] Meanwhile, the processor (970) within the server (900) may calculate a base score based on sensor data related to rapid acceleration, and may set the weight to '3' when there is a speed limit sign (1642) in the front image data (1630) of the vehicle and the driver is in a drowsy state (1644) in the internal image data (1640) of the vehicle.

[0376] Accordingly, the processor (970) within the server (900) can calculate a driving score related to rapid acceleration that is three times the base score based on sensor data related to rapid acceleration.

[0377] That is, according to Figure 16d, compared to Figure 16c, the driving score associated with sudden braking is significantly higher due to the speed limit sign and driver drowsiness. Ultimately, the driving score can be calculated in response to the driving situation.

[0378] Meanwhile, the processor (970) within the server (900) can generate a description script (1646) for a driving score related to rapid acceleration.

[0379] Figure 17a is an example of the weight of rapid acceleration.

[0380] Referring to the drawing, the processor (970) within the server (900) can set the weights to 3, 0, 0.5, 1.5, etc. depending on drowsiness, speed below minimum, speed limit increase, traffic signal, etc.

[0381] For example, the processor (970) within the server (900) may set the weight to 1 in the case of rapid acceleration when there is no vehicle ahead.

[0382] As another example, the processor (970) within the server (900) may set the weight to 3 in the case of rapid acceleration in a drowsy situation.

[0383] As another example, the processor (970) within the server (900) may set the weight to 1.5 in the case of rapid acceleration without looking ahead even when the traffic signal is green.

[0384] As another example, the processor (970) within the server (900) may set the weight to 0.5 in the case of rapid acceleration to maintain a safe speed when entering a highway.

[0385] Figure 17b is an example of the weight of sudden braking.

[0386] Referring to the drawing, the processor (970) within the server (900) can set various weights according to cut-in, distraction, drowsiness, red signal, green signal, sudden stop of the vehicle in front, phone, etc.

[0387] For example, the processor (970) within the server (900) may set the weight to 0 in the case of sudden braking due to a motorcycle cutting in during normal driving.

[0388] As another example, the processor (970) within the server (900) may set the weight to 2 in the case of sudden braking due to distraction without looking ahead in a situation where another vehicle cuts in.

[0389] As another example, the processor (970) within the server (900) may set the weight to 3 in the case of sudden braking due to phone use during a red light.

[0390] As another example, the processor (970) within the server (900) may set the weight to 0.5 in the case of sudden braking due to sudden stop of the front vehicle when starting due to a green light change.

[0391] Meanwhile, the processor (970) within the server (900) can set the weight to 1 when maintaining the right gaze during a right turn.

[0392] Meanwhile, the processor (970) within the server (900) can set the weight to 2 when the driver consumes food or beverage while cornering.

[0393] Meanwhile, the processor (970) within the server (900) can set the weight to 1.5 when there are multiple surrounding vehicles.

[0394] Meanwhile, the processor (970) within the server (900) may set the weight to 2 in the case of a narrow road or a large number of pedestrians in a single lane.

[0395] Accordingly, driving scores can be provided based on event data and video data.

[0396] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person skilled in the art to which the present invention pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. A communication unit that receives event data from a vehicle and image data related to the event data; A server including a processor that generates context information based on the event data or the image data, and calculates a driving score related to the event data based on the context information.

2. In paragraph 1, The above video data is, Contains the vehicle's external image data and the vehicle's internal image data, The above processor, A server that generates context information based on external image data of the vehicle and internal image data of the vehicle, and calculates a driving score related to the event data based on the context information and the event data.

3. In paragraph 2, The above processor, Based on the external image data of the above vehicle, driving situation data is extracted, A server that extracts driver status information based on internal image data of the above vehicle.

4. In paragraph 2, The above processor, Outputting first result data based on external image data of the vehicle, and assigning a first time index to the first result data; Outputting second result data based on the internal image data of the vehicle, and assigning a second time index to the second result data; A server that integrates the first result data and the second result data based on the first time index and the second time index.

5. In paragraph 2, The above processor, Based on the external image data of the vehicle and the internal image data of the vehicle prior to the occurrence of the event data, first level context information is generated, A server that generates second level context information lower than the first level based on external image data of the vehicle and internal image data of the vehicle after the occurrence time of the event data.

6. In paragraph 2, The above processor, A server that controls the level of the context information to increase as the duration of an event based on the internal image data of the vehicle increases or the number of repetitions increases.

7. In paragraph 2, The above processor, A server that sets the level of the context information based on the external image data of the vehicle and the internal image data of the vehicle, outputs the context information as key context information when the level of the context information is equal to or higher than a reference level, and calculates a driving score related to the event data based on the key context information and the event data.

8. In paragraph 7, The above processor, A server that adjusts a driving score related to the event data when multiple events occur within a given period of time and the multiple events at least partially overlap.

9. In paragraph 1, The above processor, Compute the base score based on the above event data, Based on the image data related to the above event data, the weights are calculated, A server that calculates a driving score related to the event data based on the base score and the weight.

10. In paragraph 1, The above processor, Compute the base score based on the above event data, Based on the image data related to the above event data, context information is generated, and based on the context information, weights are calculated, A server that calculates a driving score related to the event data based on the base score and the weight.

11. In paragraph 1, The above processor, Compute the base score based on the above event data, Based on the image data related to the above event data, context information is generated, and based on key context information among the context information, a weight is calculated, A server that calculates a driving score related to the event data based on the base score and the weight.

12. In paragraph 1, The above processor, If the above event data is sudden brake event data, the first base score is calculated based on the sudden brake event data, Based on the image data related to the above-mentioned sudden brake event data, if an object cut-in in front of the vehicle is detected, the first weight is calculated, A server that calculates a first driving score related to the sudden brake event data based on the first base score and the first weight.

13. In paragraph 12, The above processor, A server that calculates that the level of the first weight is lowered when an object cut-in in front of the vehicle is detected based on image data related to the above-mentioned sudden brake event data rather than when a traffic signal change is detected.

14. In paragraph 1, The above processor, If the above event data is rapid acceleration event data, the second base score is calculated based on the rapid acceleration event data, Based on the image data related to the above rapid acceleration event data, the second weight is calculated, A server that calculates a driving score related to the sudden acceleration event data based on the second base score and the second weight.

15. In paragraph 14, The above processor, A server that calculates, based on image data related to the above rapid acceleration event data, the level of the second weight to be higher when driver drowsiness is detected rather than when a traffic signal change is detected.

16. In paragraph 1, Further comprising a memory storing a driving score history including a driving score related to the above event data; The above processor, A server that controls transmission of the driving score history to the external server when a request for the driving score history is received from the external server.

17. A communication unit that receives event data from a vehicle and image data related to the event data; A server including a processor that calculates a base score based on the event data, calculates a weight based on image data related to the event data, and calculates a driving score related to the event data based on the base score and the weight.

18. In paragraph 17, The above processor, If the above event data is sudden brake event data, the first base score is calculated based on the sudden brake event data, Based on the image data related to the above-mentioned sudden brake event data, if an object cut-in in front of the vehicle is detected, the first weight is calculated, A server that calculates a first driving score related to the sudden brake event data based on the first base score and the first weight.

19. In Article 18, The above processor, A server that calculates that the level of the first weight is lowered when an object cut-in in front of the vehicle is detected based on image data related to the above-mentioned sudden brake event data rather than when a traffic signal change is detected.

20. In paragraph 1, The above processor, If the above event data is rapid acceleration event data, the second base score is calculated based on the rapid acceleration event data, Based on the image data related to the above rapid acceleration event data, the second weight is calculated, A server that calculates a driving score related to the sudden acceleration event data based on the second base score and the second weight.

Images