Signal processing device and vehicle display device comprising same

Abstract

Provided are a signal processing device and a vehicle display device comprising same. According to an embodiment of the present disclosure, the signal processing device comprises: at least one neural processor; and a central processor for executing an application for driving a vehicle. The central processor: when it is determined that the operation of the application fails, controls another central processor or another signal processing device to execute a second application corresponding to the application; and changes a reference fallback guarantee time for the operation failure of the application on the basis of a safety level of the application. Accordingly, the application for driving the vehicle can be stably performed.

Description

Signal processing device and vehicle display device having the same

[0001] The present disclosure relates to a signal processing device and a vehicle display device having the same, and more particularly, to a signal processing device capable of stably performing an application for driving a vehicle and a vehicle display device having the same.

[0002] A vehicle is a device that allows the user to move in the desired direction. A representative example is an automobile.

[0003] Meanwhile, for the convenience of vehicle users, a vehicle signal processing device is installed inside the vehicle.

[0004] Meanwhile, the vehicle signal processing device can execute various applications for vehicle driving.

[0005] For example, a signal processing device for a vehicle can execute an Advanced Driver Assistance Systems (ADAS) application or an Automatic Driving (AD) application.

[0006] Meanwhile, when running applications for vehicle driving, various failures may occur in the vehicle, and methods for stably controlling the vehicle when failures occur are being studied.

[0007] The problem to be solved by the present disclosure is to provide a signal processing device capable of stably performing an application for driving a vehicle and a vehicle display device having the same.

[0008] Another problem that the present disclosure seeks to solve is to provide a signal processing device capable of stably performing an application for driving a vehicle based on a safety level and a vehicle display device having the same.

[0009] A signal processing device and a vehicle display device including the same according to one embodiment of the present disclosure include at least one neural processor and a central processor that executes an application for driving a vehicle, wherein the central processor controls a second application corresponding to the application to be executed in another central processor or another signal processing device when the application is determined to have failed to operate, and varies a standard fallback guarantee time for the application to fail to operate based on a safety level of the application.

[0010] Meanwhile, the central processor may set the reference fallback guarantee time to the first time when the safety level of the application is the first safety level, and may set the reference fallback guarantee time to the second time when the safety level of the application is the second safety level higher than the first safety level.

[0011] Meanwhile, the central processor may set the reference fallback guarantee time to a third time shorter than the first time if the safety level of the application is a third safety level lower than the first safety level.

[0012] Meanwhile, the central processor can control a preset operation to be performed if the calculated fallback guarantee time is greater than or equal to the reference fallback guarantee time.

[0013] Meanwhile, the central processor can control a second application corresponding to the application to be executed on another central processor or another signal processing device if the calculated fallback guarantee time is less than the reference fallback guarantee time.

[0014] Meanwhile, the central processor can control the fallback operation to be performed if the calculated fallback guarantee time is less than the reference fallback guarantee time.

[0015] Meanwhile, the central processor can control which of the multiple fallback operations is performed if the calculated fallback guarantee time is less than the reference fallback guarantee time.

[0016] Meanwhile, multiple fallback actions may include a vehicle handover request, blind braking, lane-keeping braking, lane-changing braking, shoulder stop, or path-based stop.

[0017] Meanwhile, the central processor can control the execution of a high-safety level application to be restricted as the calculated fallback guaranteed time decreases and the difference between the calculated estimated reference fallback guaranteed time and the reference fallback guaranteed time decreases while the calculated fallback guaranteed time is greater than the reference fallback guaranteed time.

[0018] Meanwhile, the central processor may control an application corresponding to the first safety level to be executed when the difference between the calculated estimated reference fallback guaranteed time and the reference fallback guaranteed time is a first time, and may control an application corresponding to the second safety level higher than the first safety level to be executed when the difference between the calculated estimated reference fallback guaranteed time and the reference fallback guaranteed time is a second time greater than the first time.

[0019] Meanwhile, the central processor can vary the standard fallback guarantee time depending on the vehicle's driving environment.

[0020] Meanwhile, the central processor may be controlled to calculate the fallback guarantee time based on at least one of the temperature of the system, performance based on a safety level, error history of the system, delay of the application, or load of the system.

[0021] Meanwhile, the neural processor can perform neural processing based on at least one of the temperature of the system, performance based on a safety level, error history of the system, delay of the application, or load of the system, and calculate a fallback guarantee time based on the neural processing.

[0022] Meanwhile, the neural processor can perform neural processing by feeding back power data or temperature data among the result data of neural processing.

[0023] Meanwhile, the central processor executes a hypervisor, executes multiple virtual machines on the hypervisor, and a first virtual machine among the multiple virtual machines can execute a neural system service for controlling the neural processor.

[0024] Meanwhile, a second virtual machine among the plurality of virtual machines may run an ADAS application, and a third virtual machine among the plurality of virtual machines may run a driver monitoring system (DMS) application or an augmented reality application.

[0025] Meanwhile, the central processor may control, when a second virtual machine among multiple virtual machines runs an application with a higher security level than a third virtual machine, the standard fallback guarantee time of the application running in the second virtual machine to be greater than the standard fallback guarantee time of the application running in the third virtual machine.

[0026] A signal processing device and a vehicle display device including the same according to another embodiment of the present disclosure include a central processor that executes an application for driving a vehicle, and the central processor varies a standard fallback guarantee time for an operation failure of the application based on a safety level of the application, and controls a fallback operation or a fail operation to be performed when an operation failure of the application is determined based on the standard fallback guarantee time and the calculated fallback guarantee time.

[0027] A signal processing device and a vehicle display device including the same according to one embodiment of the present disclosure include at least one neural processor and a central processor that executes an application for driving a vehicle, wherein the central processor controls a second application corresponding to the application to be executed in another central processor or another signal processing device when the application is determined to have failed to operate, and varies a standard fallback guarantee time for the application to fail to operate based on the safety level of the application. Accordingly, the application for driving a vehicle can be stably performed. In particular, the application for driving a vehicle can be stably performed based on the safety level.

[0028] Meanwhile, the central processor may set the standard fallback guarantee time to a first time period if the application's safety level is the first safety level, and may set the standard fallback guarantee time to a second time period longer than the first time period if the application's safety level is the second safety level, which is higher than the first safety level. Accordingly, the application for vehicle driving can be stably performed.

[0029] Meanwhile, if the application's safety level is a third safety level lower than the first safety level, the central processor can set the standard fallback guarantee time to a third time period shorter than the first time period. This allows the application for vehicle driving to be performed reliably.

[0030] Meanwhile, the central processor can control the execution of a preset operation if the calculated fallback guarantee time is greater than or equal to the reference fallback guarantee time. This enables stable execution of applications for vehicle operation.

[0031] Meanwhile, if the calculated fallback guarantee time is less than the reference fallback guarantee time, the central processor can control the execution of a second application corresponding to the application on another central processor or another signal processing device. This enables stable execution of the application for vehicle driving.

[0032] Meanwhile, the central processor can control the fallback operation to be performed if the calculated fallback guarantee time is less than the reference fallback guarantee time. This enables stable execution of applications for vehicle driving.

[0033] Meanwhile, the central processor can control the execution of one of multiple fallback operations if the calculated guaranteed fallback time is less than the reference guaranteed fallback time. This enables stable execution of applications for vehicle driving.

[0034] Meanwhile, multiple fallback actions may include vehicle handover requests, blind braking, lane-keeping braking, lane-changing braking, shoulder-stopping, or path-based stopping. This allows for stable operation of vehicle driving applications.

[0035] Meanwhile, the central processor can control the execution of high-safety applications to be restricted as the calculated fallback guarantee time decreases and the difference between the calculated estimated reference fallback guarantee time and the reference fallback guarantee time decreases, while the calculated fallback guarantee time is greater than the reference fallback guarantee time. This allows applications for vehicle driving to be executed reliably.

[0036] Meanwhile, the central processor can control the execution of an application corresponding to the first safety level when the difference between the calculated estimated reference fallback guaranteed time and the reference fallback guaranteed time is a first time period, and can control the execution of an application corresponding to the second safety level higher than the first safety level when the difference between the calculated estimated reference fallback guaranteed time and the reference fallback guaranteed time is a second time period greater than the first time period. Accordingly, applications for vehicle driving can be stably performed.

[0037] Meanwhile, the central processor can vary the standard fallback guarantee time depending on the vehicle's driving environment. This allows for stable execution of vehicle driving applications.

[0038] Meanwhile, the central processor can be controlled to calculate a guaranteed fallback time based on at least one of the following: system temperature, performance based on safety levels, system error history, application delays, or system load. This allows stable execution of applications for vehicle operation.

[0039] Meanwhile, the neural processor can perform neural processing based on at least one of the following: system temperature, performance based on safety levels, system error history, application delay, or system load. Based on the neural processing, the neural processor can calculate a guaranteed fallback time. This enables stable execution of vehicle driving applications.

[0040] Meanwhile, the neural processor can perform neural processing by feeding back power or temperature data from the neural processing results. This enables stable execution of vehicle driving applications.

[0041] Meanwhile, the central processor executes a hypervisor, runs multiple virtual machines on the hypervisor, and the first virtual machine among the multiple virtual machines executes a neural system service for controlling the neural processor. This enables stable execution of applications for vehicle driving.

[0042] Meanwhile, among the multiple virtual machines, a second virtual machine can run an ADAS (Advanced Driver Assistance Systems) application, and a third virtual machine can run a Driver Monitoring System (DMS) application or an augmented reality application. This allows stable execution of applications for vehicle driving.

[0043] Meanwhile, if a second virtual machine among multiple virtual machines runs an application with a higher security level than a third virtual machine, the central processor can control the standard fallback guarantee time of the application running on the second virtual machine to be greater than the standard fallback guarantee time of the application running on the third virtual machine. Accordingly, applications for vehicle driving can be stably executed.

[0044] A signal processing device and a vehicle display device including the same according to another embodiment of the present disclosure include a central processor that executes an application for driving a vehicle, and the central processor varies a reference fallback guarantee time for an operation failure of the application based on a safety level of the application, and controls a fallback operation or a fail operation to be performed when the application is determined to have an operation failure based on the reference fallback guarantee time and the calculated fallback guarantee time. Accordingly, the application for driving a vehicle can be stably performed. In particular, the application for driving a vehicle can be stably performed based on a safety level.

[0045] Figure 1 is a drawing showing an example of the exterior and interior of a vehicle.

[0046] Figure 2 is a drawing illustrating an example of a vehicle communication gateway.

[0047] Figure 3a is a drawing showing an example of the arrangement of a vehicle display device inside a vehicle.

[0048] Figure 3b is a drawing showing another example of the arrangement of a vehicle display device inside a vehicle.

[0049] Fig. 4 is an example of an internal block diagram of the vehicle display device of Fig. 3b.

[0050] FIGS. 5A to 5D are drawings showing various examples of vehicle display devices.

[0051] FIG. 6 is an example of a block diagram of a vehicle display device according to an embodiment of the present disclosure.

[0052] FIG. 7 is an example of an internal block diagram of a signal processing device according to an embodiment of the present disclosure.

[0053] FIG. 8 is a diagram illustrating an example of a system driven by a signal processing device according to an embodiment of the present disclosure.

[0054] FIG. 9 is another example of a block diagram of a vehicle display device according to an embodiment of the present disclosure.

[0055] Fig. 10 is a flowchart showing an operation method of a vehicle display device according to an embodiment of the present disclosure.

[0056] Figures 11 to 13 are drawings referenced in the operation description of Figure 9 or Figure 10.

[0057] Hereinafter, the present disclosure will be described in more detail with reference to the drawings.

[0058] The suffixes "module" and "part" used in the following description are given solely for the convenience of writing this specification and do not impart any particularly significant meaning or role to the components themselves. Therefore, the terms "module" and "part" may be used interchangeably.

[0059] Figure 1 is a drawing showing an example of the exterior and interior of a vehicle.

[0060] Referring to the drawing, the vehicle (200) is operated by a plurality of wheels (103FR, 103FL, 103RL, etc.) that rotate by a power source and a steering wheel (150) for controlling the direction of travel of the vehicle (200).

[0061] Meanwhile, the vehicle (200) may further be equipped with a camera (195) for capturing images of the front of the vehicle.

[0062] Meanwhile, the vehicle (200) may be equipped with multiple displays (180a, 180b) for displaying images, information, etc. inside.

[0063] In Fig. 1, a cluster display (180a) and an AVN (Audio Video Navigation) display (180b) are exemplified as multiple displays (180a, 180b). In addition, a HUD (Head Up Display) is also possible.

[0064] Meanwhile, the AVN (Audio Video Navigation) display (180b) may also be named a center information display.

[0065] Meanwhile, the vehicle (200) described in this specification may be a concept that includes all of a vehicle equipped with an engine as a power source, a hybrid vehicle equipped with an engine and an electric motor as a power source, and an electric vehicle equipped with an electric motor as a power source.

[0066] Figure 2 is a drawing illustrating an example of a vehicle communication gateway.

[0067] Referring to the drawing, the architecture (300a) of the vehicle communication gateway can correspond to a zone-based architecture.

[0068] Accordingly, sensor devices and processors inside the vehicle may be placed in each of the plurality of zones (Z1 to Z4), and a signal processing device (170a) including a vehicle communication gateway (GWDa) may be placed in the central area of ​​the plurality of zones (Z1 to Z4).

[0069] Meanwhile, the signal processing device (170a) may further include, in addition to the vehicle communication gateway (GWDa), an autonomous driving control module (ACC), a cockpit control module (CPG), etc.

[0070] The vehicle communication gateway (GWDa) within the signal processing device (170a) may be an HPC (High Performance Computing) gateway.

[0071] That is, the signal processing device (170a) of FIG. 2 is an integrated HPC and can exchange data with an external communication module (not shown) or a processor (not shown) within a plurality of zones (Z1 to Z4).

[0072] Figure 3a is a drawing showing an example of the arrangement of a vehicle display device inside a vehicle.

[0073] Referring to the drawing, the interior of the vehicle may be equipped with a cluster display (180a), an AVN (Audio Video Navigation) display (180b), a rear seat entertainment display (180c, 180d), a room mirror display (not shown), etc.

[0074] Figure 3b is a drawing showing another example of the arrangement of a vehicle display device inside a vehicle.

[0075] A vehicle display device (100) according to an embodiment of the present disclosure may include a plurality of displays (180a to 180b), and a signal processing device (170) that performs signal processing for displaying images, information, etc. on the plurality of displays (180a to 180b) and outputs an image signal to at least one display (180a to 180b).

[0076] Among the plurality of displays (180a to 180b), the first display (180a) may be a cluster display (180a) for displaying driving status, operation information, etc., and the second display (180b) may be an AVN (Audio Video Navigation) display (180b) for displaying vehicle driving information, a navigation map, various entertainment information, or images.

[0077] The signal processing device (170) has a processor (175) therein and can execute a first virtual machine to a third virtual machine (not shown) on a hypervisor (not shown) within the processor (175).

[0078] A second virtual machine (not shown) can operate for the first display (180a), and a third virtual machine (not shown) can operate for the second display (180b).

[0079] Meanwhile, the first virtual machine (not shown) within the processor (175) can control the shared memory (508) based on the hypervisor (505) to be set for the same data transmission to the second virtual machine (not shown) and the third virtual machine (not shown). Accordingly, the same information or the same image can be displayed in synchronization on the first display (180a) and the second display (180b) within the vehicle.

[0080] Meanwhile, the first virtual machine (not shown) within the processor (175) shares at least a portion of data with the second virtual machine (not shown) and the third virtual machine (not shown) for data sharing processing. Accordingly, data can be shared and processed among multiple virtual machines for multiple displays within the vehicle.

[0081] Meanwhile, a first virtual machine (not shown) within a processor (175) may receive and process vehicle wheel speed sensor data, and transmit the processed wheel speed sensor data to at least one of a second virtual machine (not shown) or a third virtual machine (not shown). Accordingly, the vehicle wheel speed sensor data may be shared with at least one virtual machine.

[0082] Meanwhile, the vehicle display device (100) according to the embodiment of the present disclosure may further include a rear seat entertainment display (180c) for displaying driving status information, simple navigation information, various entertainment information, or images.

[0083] The signal processing device (170) can control the RSE display (180c) by executing a fourth virtual machine (not shown) in addition to the first virtual machine to the third virtual machine (not shown) on a hypervisor (not shown) within the processor (175).

[0084] Accordingly, it is possible to control various displays (180a to 180c) using one signal processing device (170).

[0085] Meanwhile, some of the multiple displays (180a~180c) may operate under Linux OS, while others may operate under Web OS.

[0086] The signal processing device (170) according to the embodiment of the present disclosure can control the same information or the same image to be displayed in synchronization on displays (180a to 180c) operating under various operating systems (OS).

[0087] Meanwhile, in FIG. 3b, a vehicle speed indicator (212a) and a vehicle interior temperature indicator (213a) are displayed on a first display (180a), a home screen (222) including a plurality of applications and a vehicle speed indicator (212b) and a vehicle interior temperature indicator (213b) are displayed on a second display (180b), and a second home screen (222b) including a plurality of applications and a vehicle interior temperature indicator (213c) are displayed on a third display (180c).

[0088] Fig. 4 is an example of an internal block diagram of the vehicle display device of Fig. 3b.

[0089] Referring to the drawings, a vehicle display device (100) according to an embodiment of the present disclosure may include an input unit (110), a communication unit (120) for communication with an external device, a plurality of communication modules (EMa to EMd) for internal communication, a memory (140), a signal processing unit (170), a plurality of displays (180a to 180c), an audio output unit (185), and a power supply unit (190).

[0090] A plurality of communication modules (EMa to EMd) can be arranged, for example, in a plurality of zones (Z1 to Z4) of FIG. 2, respectively.

[0091] Meanwhile, the signal processing device (170) may have a communication switch (736b) for data communication with each communication module (EM1 to EM4) inside.

[0092] Each communication module (EM1 to EM4) can perform data communication with multiple sensor devices (SN) or ECUs (770) or area signal processing devices (170Z).

[0093] Meanwhile, the plurality of sensor devices (SN) may include a camera (195), a lidar (196), a radar (197), or a position sensor (198).

[0094] The input unit (110) may be equipped with physical buttons, pads, etc. for button input, touch input, etc.

[0095] Meanwhile, the input unit (110) may be equipped with a microphone (not shown) for user voice input.

[0096] The communication unit (120) can exchange data wirelessly with a mobile terminal (800) or a server (900).

[0097] In particular, the communication unit (120) can wirelessly exchange data with the vehicle driver's mobile terminal. Various data communication methods are possible, such as Bluetooth, WiFi, WiFi Direct, and APiX.

[0098] The communication unit (120) can receive weather information, road traffic information, for example, TPEG (Transport Protocol Expert Group) information, from a mobile terminal (800) or a server (900). To this end, the communication unit (120) may be equipped with a mobile communication module (not shown).

[0099] A plurality of communication modules (EM1 to EM4) can receive sensor data, etc. from an ECU (770), a sensor device (SN), or an area signal processing device (170Z), and transmit the received sensor data to the signal processing device (170).

[0100] Here, the sensor data may include at least one of vehicle direction data, vehicle location data (GPS data), vehicle angle data, vehicle speed data, vehicle acceleration data, vehicle inclination data, vehicle forward / backward data, battery data, fuel data, tire data, vehicle lamp data, vehicle interior temperature data, and vehicle interior humidity data.

[0101] Such sensor data can be obtained from a heading sensor, a yaw sensor, a gyro sensor, a position module, a vehicle forward / backward sensor, a wheel sensor, a vehicle speed sensor, a body tilt detection sensor, a battery sensor, a fuel sensor, a tire sensor, a steering sensor by steering wheel rotation, a vehicle interior temperature sensor, a vehicle interior humidity sensor, etc.

[0102] Meanwhile, the position module may include a GPS module or a position sensor (198) for receiving GPS information.

[0103] Meanwhile, at least one of the plurality of communication modules (EM1 to EM4) can transmit location information data sensed by a GPS module or location sensor (198) to a signal processing device (170).

[0104] Meanwhile, at least one of the plurality of communication modules (EM1 to EM4) can receive vehicle front image data, vehicle side image data, vehicle rear image data, vehicle surrounding obstacle distance information, etc. from a camera (195), lidar (196), radar (197), etc., and transmit the received information to a signal processing device (170).

[0105] The memory (140) can store various data for the overall operation of the vehicle display device (100), such as a program for processing or controlling the signal processing device (170).

[0106] For example, the memory (140) may store data regarding a hypervisor, a first virtual machine, a third virtual machine, or the like, for execution within the processor (175).

[0107] The audio output unit (185) converts an electric signal from the signal processing device (170) into an audio signal and outputs it. For this purpose, a speaker or the like may be provided.

[0108] The power supply unit (190) can supply power required for the operation of each component under the control of the signal processing device (170). In particular, the power supply unit (190) can receive power from a battery or the like inside the vehicle.

[0109] The signal processing device (170) controls the overall operation of each unit within the vehicle display device (100).

[0110] For example, the signal processing device (170) may include a processor (175) that performs signal processing for a vehicle display (180a, 180b).

[0111] The processor (175) can execute a first virtual machine to a third virtual machine (not shown) on a hypervisor (not shown) within the processor (175).

[0112] Among the first virtual machine to the third virtual machine (not shown), the first virtual machine (not shown) may be named a server virtual machine (Server Virtual Maschine), and the second virtual machine to the third virtual machine (not shown) may be named a guest virtual machine (Guest Virtual Maschine).

[0113] For example, a first virtual machine (not shown) within a processor (175) may receive, process, or output sensor data from a plurality of sensor devices, such as vehicle sensor data, location information data, camera image data, audio data, or touch input data.

[0114] In this way, by performing most of the data processing in the first virtual machine (not shown), data sharing in a 1:N manner becomes possible.

[0115] As another example, a first virtual machine (not shown) can directly receive and process CAN data, Ethernet data, audio data, radio data, USB data, and wireless communication data for a second virtual machine or a third virtual machine (not shown).

[0116] And, the first virtual machine (not shown) can transmit processed data to the second virtual machine or the third virtual machine (not shown).

[0117] Accordingly, among the first virtual machine to the third virtual machine (not shown), only the first virtual machine (not shown) receives sensor data, communication data, or external input data from multiple sensor devices and performs signal processing, thereby reducing the signal processing burden on other virtual machines, enabling 1:N data communication, and enabling synchronization when sharing data.

[0118] Meanwhile, the first virtual machine (not shown) can control the second virtual machine (not shown) and the third virtual machine (not shown) to share the same data by writing data to the shared memory (508).

[0119] For example, a first virtual machine (not shown) can record vehicle sensor data, the location information data, the camera image data, or the touch input data in shared memory (508) and control the same data to be shared with a second virtual machine (not shown) and a third virtual machine (not shown). Accordingly, data sharing in a 1:N manner becomes possible.

[0120] Ultimately, by performing most of the data processing on the first virtual machine (not shown), data sharing in a 1:N manner becomes possible.

[0121] Meanwhile, the first virtual machine (not shown) within the processor (175) can control the shared memory (508) based on the hypervisor (505) to be set for the same data transmission to the second virtual machine (not shown) and the third virtual machine (not shown).

[0122] Meanwhile, the signal processing device (170) can process various signals such as audio signals, video signals, and data signals. To this end, the signal processing device (170) can be implemented in the form of a system on chip (SOC).

[0123] Meanwhile, the signal processing device (170) in the display device (100) of FIG. 4 may be the same as the signal processing device (170, 170a1, 170a2) of the vehicle display device of FIG. 5a or lower.

[0124] FIGS. 5A to 5D are drawings showing various examples of vehicle display devices.

[0125] FIG. 5A illustrates an example of a vehicle display device according to an embodiment of the present disclosure.

[0126] Referring to the drawings, a vehicle display device (800a) according to an embodiment of the present disclosure includes a signal processing device (170a1, 170a2) and a plurality of area signal processing devices (170Z1 to 170Z4).

[0127] Meanwhile, in the drawing, two signal processing devices (170a1, 170a2) are exemplified, but this is for backup purposes, etc., and one is also possible.

[0128] Meanwhile, the signal processing device (170a1, 170a2) may also be named an HPC (High Performance Computing) signal processing device.

[0129] Multiple area signal processing devices (170Z1 to 170Z4) are arranged in each area (Z1 to Z4) and can transmit sensor data to signal processing devices (170a1, 170a2).

[0130] The signal processing device (170a1, 170a2) receives data via a wire from multiple area signal processing devices (170Z1 to 170Z4) or a communication device (120).

[0131] In the drawing, data is exchanged based on wired communication between a signal processing device (170a1, 170a2) and multiple area signal processing devices (170Z1 to 170Z4), and the signal processing device (170a1, 170a2) and the server (400) exchange data based on wireless communication. However, data may be exchanged based on wireless communication between a communication device (120) and a server (400), and the signal processing device (170a1, 170a2) and the communication device (120) may exchange data based on wired communication.

[0132] Meanwhile, data received by the signal processing device (170a1, 170a2) may include camera data or sensor data.

[0133] For example, sensor data within a vehicle may include at least one of vehicle wheel speed data, vehicle direction data, vehicle location data (GPS data), vehicle angle data, vehicle speed data, vehicle acceleration data, vehicle inclination data, vehicle forward / backward data, battery data, fuel data, tire data, vehicle lamp data, vehicle interior temperature data, vehicle interior humidity data, vehicle exterior radar data, and vehicle exterior lidar data.

[0134] Meanwhile, camera data may include vehicle exterior camera data and vehicle interior camera data.

[0135] Meanwhile, the signal processing device (170a1, 170a2) can execute multiple virtual machines (820, 830, 840) based on safety standards.

[0136] In the drawing, it is illustrated that a processor (175) within a signal processing device (170a) executes a hypervisor (505) and, on the hypervisor (505), executes first to third virtual machines (820 to 840) according to an automotive safety integrity level (Automotive SIL; ASIL).

[0137] The first virtual machine (820) may be a virtual machine corresponding to Quality Management (QM), which is the lowest safety level in the Automotive Safety Integrity Level (ASIL) and is a non-enforceable grade.

[0138] The first virtual machine (820) can execute an operating system (822), a container runtime (824) on the operating system (822), and containers (827, 829) on the container runtime (824).

[0139] The second virtual machine (820) may be a virtual machine corresponding to ASIL A or ASIL B, where the sum of severity, exposure, and controllability is 7 or 8 in the automotive safety integrity level (ASIL).

[0140] The second virtual machine (820) can execute an operating system (832), a container runtime (834) on the operating system (832), and containers (837, 839) on the container runtime (834).

[0141] The third virtual machine (840) may be a virtual machine corresponding to ASIL C or ASIL D, in which the sum of severity, exposure, and controllability is 9 or 10 in the automotive safety integrity level (ASIL).

[0142] Meanwhile, ASIL D can correspond to the grade that requires the highest safety level.

[0143] The third virtual machine (840) can run a safety operating system (842) and an application (845) on the operating system (842).

[0144] Meanwhile, the third virtual machine (840) may also execute a safety operating system (842), a container runtime (844) on the safety operating system (842), and a container (847) on the container runtime (844).

[0145] Meanwhile, unlike the drawing, the third virtual machine (840) can also be executed through a separate core rather than the processor (175). This will be described later with reference to FIG. 5b.

[0146] FIG. 5b illustrates another example of a vehicle display device according to an embodiment of the present disclosure.

[0147] Referring to the drawings, a vehicle display device (800b) according to an embodiment of the present disclosure includes a signal processing device (170a1, 170a2) and a plurality of area signal processing devices (170Z1 to 170Z4).

[0148] The vehicle display device (800b) of FIG. 5b is similar to the vehicle display device (800a) of FIG. 5a, but the signal processing device (170a1) has some differences from the signal processing device (170a1) of FIG. 5a.

[0149] To describe the difference, the signal processing device (170a1) may include a processor (175) and a second processor (177).

[0150] The processor (175) within the signal processing unit (170a1) executes a hypervisor (505), and executes first and second virtual machines (820 to 830) on the hypervisor (505) according to the automotive safety integrity level (Automotive SIL; ASIL).

[0151] The first virtual machine (820) can execute an operating system (822), a container runtime (824) on the operating system (822), and containers (827, 829) on the container runtime (824).

[0152] The second virtual machine (820) can execute an operating system (832), a container runtime (834) on the operating system (832), and containers (837, 839) on the container runtime (834).

[0153] Meanwhile, the second processor (177) within the signal processing device (170a1) can execute a third virtual machine (840).

[0154] The third virtual machine (840) can execute a safety operating system (842), an auto-execution (845) on the operating system (842), and an application (845) on the auto-execution (845). That is, unlike FIG. 5A, an auto-execution (846) on the operating system (842) can be executed.

[0155] Meanwhile, the third virtual machine (840) may, similarly to FIG. 5a, execute a safety operating system (842), a container runtime (844) on the safety operating system (842), and a container (847) on the container runtime (844).

[0156] Meanwhile, the third virtual machine (840) requiring a high level of security is preferably executed on a second processor (177), which is a different core or different processor, unlike the first and second virtual machines (820 to 830).

[0157] Meanwhile, in the signal processing devices (170a1, 170a2) of FIGS. 5a and 5b, when the first signal processing device (170a) malfunctions, the second signal processing device (170a2), which is a backup device, can operate.

[0158] Alternatively, it is also possible for the signal processing devices (170a1, 170a2) to operate simultaneously, with the first signal processing device (170a) operating as the main device and the second signal processing device (170a2) operating as the sub device. This will be described with reference to FIGS. 5c and 5d.

[0159] FIG. 5c illustrates another example of a vehicle display device according to an embodiment of the present disclosure.

[0160] Referring to the drawings, a vehicle display device (800c) according to an embodiment of the present disclosure includes a signal processing device (170a1, 170a2) and a plurality of area signal processing devices (170Z1 to 170Z4).

[0161] Meanwhile, in the drawing, two signal processing devices (170a1, 170a2) are exemplified, but this is for backup purposes, etc., and one is also possible.

[0162] Meanwhile, the signal processing device (170a1, 170a2) may also be named an HPC (High Performance Computing) signal processing device.

[0163] Multiple area signal processing devices (170Z1 to 170Z4) are arranged in each area (Z1 to Z4) and can transmit sensor data to signal processing devices (170a1, 170a2).

[0164] The signal processing device (170a1, 170a2) receives data via a wire from multiple area signal processing devices (170Z1 to 170Z4) or a communication device (120).

[0165] In the drawing, data is exchanged based on wired communication between a signal processing device (170a1, 170a2) and multiple area signal processing devices (170Z1 to 170Z4), and the signal processing device (170a1, 170a2) and the server (400) exchange data based on wireless communication. However, data may be exchanged based on wireless communication between a communication device (120) and a server (400), and the signal processing device (170a1, 170a2) and the communication device (120) may exchange data based on wired communication.

[0166] Meanwhile, data received by the signal processing device (170a1, 170a2) may include camera data or sensor data.

[0167] Meanwhile, among the signal processing devices (170a1, 170a2), the processor (175) in the first signal processing device (170a1) executes a hypervisor (505) and can execute a safety virtualization machine (860) and a non-safety virtualization machine (870) on the hypervisor (505), respectively.

[0168] Meanwhile, among the signal processing devices (170a1, 170a2), the processor (175b) in the second signal processing device (170a2) executes the hypervisor (505b) and can execute only the safety virtualization machine (880) on the hypervisor (505).

[0169] In this way, since the processing for safety is separated between the first signal processing device (170a1) and the second signal processing device (170a2), it is possible to improve stability and processing speed.

[0170] Meanwhile, high-speed network communication can be performed between the first signal processing device (170a1) and the second signal processing device (170a2).

[0171] FIG. 5d illustrates another example of a vehicle display device according to an embodiment of the present disclosure.

[0172] Referring to the drawings, a vehicle display device (800d) according to an embodiment of the present disclosure includes a signal processing device (170a1, 170a2) and a plurality of area signal processing devices (170Z1 to 170Z4).

[0173] The vehicle display device (800d) of FIG. 5d is similar to the vehicle display device (800c) of FIG. 5c, but the second signal processing device (170a2) has some differences from the second signal processing device (170a2) of FIG. 5c.

[0174] The processor (175b) in the second signal processing device (170a2) of FIG. 5d executes a hypervisor (505b) and can execute a safety virtualization machine (880) and a non-safety virtualization machine (890) on the hypervisor (505).

[0175] That is, unlike FIG. 5c, the difference is that the processor (175b) within the second signal processing device (170a2) further executes a non-safety virtualization machine (890).

[0176] In this way, since the processing for safety and non-safety is separated into the first signal processing device (170a1) and the second signal processing device (170a2), it is possible to improve stability and processing speed.

[0177] FIG. 6 is an example of a block diagram of a vehicle display device according to an embodiment of the present disclosure.

[0178] Referring to the drawings, a vehicle display device (900) according to an embodiment of the present disclosure includes a signal processing device (170) and at least one display.

[0179] In the drawing, at least one display is illustrated, including a cluster display (180a), an AVN display (180b), and a network display (180c, 180d).

[0180] Meanwhile, the cluster display (180a) and the AVN display (180b) can each be connected to a display port.

[0181] Meanwhile, the network displays (180c, 180d) can be connected to an in-vehicle network via a network port. The network may be an Ethernet network based on Ethernet communication.

[0182] In the drawing, the network displays (180c, 180d) are exemplified as being connected to the third area signal processing device (170Z3) and the fourth area signal processing device (170Z4), respectively; however, alternatively, they may be connected to other area signal processing devices or directly connected to the signal processing device (170).

[0183] Meanwhile, the vehicle display device (900) may further include a plurality of area signal processing devices (170Z1 to 170Z4).

[0184] The signal processing device (170) at this time is a high-performance centralized signal processing and control device having multiple CPUs (175), GPUs (178), NPUs (179), etc., and may be called an HPC (High Performance Computing) signal processing device or a central signal processing device.

[0185] A plurality of area signal processing devices (170Z1 to 170Z4) and a signal processing device (170) are connected by wired cables (CB1 to CB4).

[0186] Meanwhile, multiple area signal processing devices (170Z1 to 170Z4) can be connected to each other with wired cables (CBa to CBd).

[0187] The wired cable (CBa~CBd) at this time may include a CAN communication cable, an Ethernet communication cable, or a PCI Express cable.

[0188] Meanwhile, a signal processing device (170) according to an embodiment of the present disclosure may be equipped with at least one processor (175, 178, 177) and a large-capacity storage device (925).

[0189] For example, a signal processing device (170) according to an embodiment of the present disclosure may include a central processor (175, 177), a graphics processor (178), and a neural processor (179).

[0190] Meanwhile, sensor data may be transmitted from at least one of the multiple area signal processing devices (170Z1 to 170Z4) to the signal processing device (170). In particular, the sensor data may be stored in a storage device (925) within the signal processing device (170).

[0191] The sensor data at this time may include at least one of camera data, lidar data, radar data, vehicle direction data, vehicle location data (GPS data), vehicle angle data, vehicle speed data, vehicle acceleration data, vehicle inclination data, vehicle forward / backward data, battery data, fuel data, tire data, vehicle lamp data, vehicle interior temperature data, and vehicle interior humidity data.

[0192] In the drawing, it is exemplified that camera data from a camera (195a) and lidar data from a lidar sensor (196) are input to a first area signal processing device (170Z1), and the camera data and lidar data are transmitted to a signal processing device (170) via a second area signal processing device (170Z2), a third area signal processing device (170Z3), etc.

[0193] Meanwhile, since the data read speed or write speed to the storage device (925) is faster than the network speed when sensor data is transmitted from at least one of the plurality of area signal processing devices (170Z1 to 170Z4) to the signal processing device (170), it is preferable that multi-path routing be performed so that a network bottleneck does not occur.

[0194] To this end, the signal processing device (170) according to the embodiment of the present disclosure can perform multi-path routing based on a Software Defined Network (SDN). Accordingly, a stable network environment can be secured when reading or writing data from the storage device (925). Furthermore, since data can be transmitted to the storage device (925) using multiple paths, the network configuration can be dynamically changed to transmit data.

[0195] Data communication between a plurality of area signal processing devices (170Z1 to 170Z4) and a signal processing device (170) in a vehicle display device (900) according to an embodiment of the present disclosure is preferably Peripheral Component Interconnect Express communication for high-bandwidth, low-latency communication.

[0196] FIG. 7 is an example of an internal block diagram of a signal processing device according to an embodiment of the present disclosure.

[0197] Referring to the drawing, a signal processing system (1000) according to one embodiment of the present disclosure may include a central signal processing device (170) and an area signal processing device (170z).

[0198] Meanwhile, a signal processing device (170) in a system (1000) according to one embodiment of the present disclosure has a plurality of processor cores (CR1 to CRn, MR).

[0199] Meanwhile, some (CR1 to CRn) of the multiple processor cores (CR1 to CRn, MR) may correspond to processor cores in the central processor (CPU) of FIG. 6.

[0200] For example, some (CR1 to CRn) of the multiple processor cores (CR1 to CRn, MR) may correspond to application processor cores within the central processor (CPU) of FIG. 6.

[0201] Meanwhile, some (CR1 to CRn) of the multiple processor cores (CR1 to CRn, MR) operate based on a hypervisor (505), and the hypervisor can execute multiple virtual machines (820 to 850).

[0202] Meanwhile, some of the other processor cores (CR1 to CRn, MR) can correspond to M cores or MCUs (micom nuit).

[0203] Meanwhile, some other processor cores (MR) among the plurality of processor cores (CR1 to CRn, MR) can execute an operating system (805a) corresponding to a second safety level such as ASIL D without executing a hypervisor (505), and execute a fourth virtual machine (840) on the operating system (805a).

[0204] Meanwhile, the fourth virtual machine (840) can execute an application corresponding to a second safety level, such as ASIL D, or a microservice (843) corresponding to an application corresponding to the second safety level. Accordingly, the application or microservice (843) corresponding to the second safety level can be stably performed.

[0205] Meanwhile, among the plurality of processor cores (CR1 to CRn, MR), the first processor core (CR1) can execute a hypervisor (505), execute an operating system (805b) corresponding to a second safety level such as ASIL D on the hypervisor (505), and execute a first virtual machine (850) on the operating system (805b).

[0206] Meanwhile, the first virtual machine (850) can execute an application corresponding to a first safety level, such as ASIL B, or a microservice (853a, 853b) corresponding to an application corresponding to the first safety level. Accordingly, the application or microservice (853a, 853b) corresponding to the first safety level can be stably performed.

[0207] Meanwhile, unlike the drawing, the first processor core (CR1) among the multiple processor cores (CR1 to CRn, MR) may execute an operating system corresponding to the first safety level, such as ASIL B, on the hypervisor (505).

[0208] Meanwhile, among the plurality of processor cores (CR1 to CRn, MR), the second processor core (CR2) and the third processor core (CR3) can execute a hypervisor (505), execute an operating system (805c) corresponding to a first safety level such as ASIL B on the hypervisor (505), and execute a second virtualization machine (850) on the operating system (805c).

[0209] Meanwhile, the second virtual machine (850) can execute a third application corresponding to a first safety level, such as ASIL B, or a microservice (833a to 833d) corresponding to the third application corresponding to the first safety level, on an operating system (805c) corresponding to the first safety level. Accordingly, the application or microservice (833a to 833d) corresponding to the first safety level can be stably performed.

[0210] Meanwhile, among the plurality of processor cores (CR1 to CRn, MR), the remaining processor cores (CR4 to CRn) can execute a hypervisor (505), execute an operating system (805d) corresponding to a third safety level such as QM on the hypervisor (505), and execute a third virtualization machine (820) on the operating system (805d).

[0211] Meanwhile, the third virtual machine (820) can execute a fourth application corresponding to the third safety level, such as QM, or a microservice (823a to 823d) corresponding to the fourth application corresponding to the third safety level, on an operating system (805d) corresponding to a third safety level lower than the first safety level. Accordingly, the application or microservice (823a to 823d) corresponding to the third safety level can be stably performed.

[0212] Meanwhile, the area signal processing device (170z) may be equipped with a plurality of application processor cores (CRR1 to CRRm) and an M core (MRb) for executing applications of ASIL D corresponding to the second safety level, which is the highest safety level.

[0213] Meanwhile, among the plurality of processor cores (CRR1 to CRRm, MRb) within the area signal processing device (170z), some (RR1 to CRRm) may execute an operating system (806b) corresponding to a first safety level such as ASIL B, and may execute a virtual machine (830b) corresponding to the first safety level on the operating system (806a).

[0214] Meanwhile, a virtual machine (830b) corresponding to the first safety level can execute an application corresponding to the first safety level, such as ASIL B, or a microservice (830ba to 830bd) corresponding to the application corresponding to the first safety level. Accordingly, the application or microservice (830ba to 830bd) corresponding to the first safety level can be stably performed.

[0215] Meanwhile, among the plurality of processor cores (CRR1 to CRRm, MRb) within the area signal processing device (170z), another part (MRb) may execute an operating system (806a) corresponding to a second safety level such as ASIL D, and may execute a virtual machine (840b) corresponding to a second safety level such as ASIL D on the operating system (806a).

[0216] Meanwhile, a virtual machine (840b) corresponding to the second safety level can execute an application corresponding to the second safety level, such as ASIL D, or a microservice (843b) corresponding to the application corresponding to the second safety level. Accordingly, the application or microservice (843b) corresponding to the second safety level can be stably performed.

[0217] FIG. 8 is a diagram illustrating an example of a system driven by a signal processing device according to an embodiment of the present disclosure.

[0218] Referring to the drawings, a signal processing device (170) in a signal processing system (1000) according to one embodiment of the present disclosure includes a central processor (175) and at least one neural processor (179a to 179c).

[0219] Meanwhile, the signal processing device (170) according to the embodiment of the present disclosure may further include a graphics processor (178).

[0220] Meanwhile, the central processor (175) according to the embodiment of the present disclosure executes a hypervisor (505).

[0221] Meanwhile, a system (1100) driven by a signal processing device (170) according to an embodiment of the present disclosure executes a plurality of virtual machines (810 to 850) on a hypervisor (505).

[0222] Specifically, a central processor (175) in a signal processing device (170) according to an embodiment of the present disclosure executes a hypervisor (505) and executes a plurality of virtual machines (810 to 850) on the hypervisor (505).

[0223] Meanwhile, the central processor (175) in the signal processing device (170) according to the embodiment of the present disclosure executes an application for driving the vehicle.

[0224] Meanwhile, if the central processor (175) determines that the application has failed to operate, it controls a second application corresponding to the application to be executed in another central processor or another signal processing device, and varies the standard fallback guarantee time for the application's operation failure based on the safety level of the application.

[0225] Accordingly, applications for vehicle operation can be performed reliably. In particular, applications for vehicle operation can be performed reliably based on safety levels.

[0226] Meanwhile, a signal processing device (170) according to one embodiment of the present disclosure may further include a shared memory (508).

[0227] In the drawing, it is illustrated that a hypervisor (505) is executed on a central processor (175) and a shared memory (508) is executed within the hypervisor (505).

[0228] Meanwhile, a signal processing device (170) according to an embodiment of the present disclosure may receive data from a camera device (195), a sensor device (700), a communication device (120), or a lidar device (not shown), and perform signal processing using at least one of a central processor (175), a graphic processor (178), and a plurality of neural processors (179a to 179c).

[0229] Meanwhile, the sensor device (700) can continuously output sensor data to the signal processing device (170) during vehicle operation.

[0230] The sensor data at this time is data from sensor devices (700) of various vehicles, and may include at least one of vehicle direction data, vehicle location data (GPS data), vehicle angle data, vehicle speed data, vehicle acceleration data, vehicle inclination data, vehicle forward / backward data, battery data, fuel data, tire data, vehicle lamp data, vehicle internal temperature data, and vehicle internal humidity data.

[0231] Meanwhile, the camera device (195) can continuously output camera data to the signal processing device (170) during vehicle operation.

[0232] Meanwhile, Lidar (not shown) can continuously output Lidar data to a signal processing device (170) while the vehicle is in operation.

[0233] Meanwhile, the neural processor (179) can detect an object based on camera data, and operate at a variable frame rate based on the object or output result data including the object.

[0234] Meanwhile, the neural processor (179) can receive camera data at a fixed frame rate, detect an object based on the camera data, and operate at a variable frame rate based on the object or output result data including the object.

[0235] Meanwhile, among the multiple virtualization machines (810 to 850), the first virtualization machine (810), which is a server virtualization machine, controls the operation of the neural processor (179).

[0236] Meanwhile, among the multiple virtualization machines (810 to 850), the second virtualization machine (850) and the third virtualization machine (830), which are guest virtualization machines, can each execute applications.

[0237] In the drawing, a second virtual machine (850) is illustrated executing an ADAS (Advanced Driver Assistance Systems) application (Nad) or an autonomous driving application, and a third virtual machine (830) is illustrated executing a Driver Monitoring System (DMS) application (Ndm) and an Augmented Reality (AR) application (Nar).

[0238] The first virtual machine (810) sequentially receives a request for a first operation, a request for a second operation, and a request for a third operation from a plurality of applications running on at least one of the plurality of virtual machines (810 to 850), and if the first operation and the third operation can be processed in parallel, the first neural processor (179a) controls the first operation and the third operation to be processed in parallel, and controls the second operation to be processed after the first operation and the third operation are completed. Accordingly, the neural processor can be operated efficiently. Furthermore, power consumption can be reduced.

[0239] Meanwhile, if the first virtual machine (810) receives a request for a fourth operation after the third operation has been requested and sharing of the operation layers between the second and fourth operations is possible, the first neural processor (179a) can be controlled to sequentially process the second and fourth operations after the first and third operations are completed. Accordingly, the neural processor can be operated efficiently.

[0240] Meanwhile, the first virtual machine (810) can control the arrangement of data for multiple operations within the internal memory (1805) of the first neural processor (179a) to vary when multiple operations are requested from multiple applications. Accordingly, the neural processor can be operated efficiently.

[0241] Meanwhile, the first virtual machine (810) can execute a neural system service (1110) to control at least one neural processor (179a to 179c).

[0242] Meanwhile, the neural system service (1110) can control the arrangement of data for multiple operations within the internal memory (1805) of the first neural processor (179a) to vary when multiple operations are requested from multiple applications. This enables efficient operation of the neural processor.

[0243] Meanwhile, the neural system service (1110) may execute or be equipped with a neural manager (1113) for managing at least one neural processor (179a to 179c), a neural controller (1115) for determining or controlling an inference method of at least one neural processor (179a to 179c), and a neural interface (1118) for interfacing with at least one neural processor (179a to 179c).

[0244] Meanwhile, the neural system service (1110) may further execute or include a model container (509) that performs interface of model parameters related to the operation of the neural processor (179) and version management of learning files.

[0245] The neural manager (1113) can perform artificial intelligence model management, learning model management, camera data management, sensor data management, or command queue management.

[0246] The neural controller (1115) can determine an optimal inference method of at least one neural processor (179a to 179c), perform queue, partition, caching, or scalable coding, or control at least one neural processor (179a to 179c).

[0247] The neural interface (1118) can execute an application program interface (API) related to the accelerator of at least one neural processor (179a-179c).

[0248] Meanwhile, the interface (522) within the first virtual machine (810) can perform interfacing between the neural system service (1110) and the model container (509) or the neural system service (1110) and the shared memory (508).

[0249] Meanwhile, the interface (522) within the first virtual machine (810) can perform interfacing to the first virtual machine (810).

[0250] Meanwhile, the interface (522) within the first virtual machine (810) can perform interfacing to a vehicle driving assistance application (Nad) running within the second virtual machine (850) or a driver monitoring system application (Ndm) or an augmented reality application (Nar) running within the third virtual machine (830).

[0251] For example, the interface (522) within the first virtual machine (810) can be controlled to transmit camera data, sensor data, or voice data to the neural processor (179) using the shared memory (508).

[0252] Meanwhile, the interface (522) within the first virtual machine (810) can control the transmission of result data output from the neural processor (179) and recorded in the shared memory (508) to the neural system service (1110).

[0253] Meanwhile, the interface (522) within the first virtual machine (810) can control the transmission of result data output from the neural processor (179) and recorded in the shared memory (508) to a vehicle driving assistance application (Nad) running within the second virtual machine (850) or a driver monitoring system application (Ndm) or an augmented reality application (Nar) running within the third virtual machine (830).

[0254] Meanwhile, the first virtual machine (810) may be executed based on the first operating system (805), the second virtual machine (850) may be executed based on the second operating system (805b) with a high safety level, and the third virtual machine (830) may be executed based on the third operating system (805c).

[0255] That is, multiple virtual machines (810 to 850) may be run based on different operating systems, or may be run based on at least two operating systems.

[0256] Meanwhile, the neural manager (1113) can manage the operating requirements of an artificial neural network-based application, control neural network weight data, and process necessary input data.

[0257] Meanwhile, the neural manager (1113) can sequentially process the optimized command queue through a hardware accelerator and transmit the operation results to the application.

[0258] Driving requirements can include the computational priorities, dependencies, and accuracy of a neural network. Operational priorities are preset values, such as whether the first operation should always be processed before the second, or, in the case of a safety-critical neural network, whether it should be processed first in the command queue before other candidate neural networks.

[0259] Meanwhile, neural network weight data refers to a file in which the element values ​​of each matrix are structured and stored in the process of inferring the results of a neural network calculated through a series of matrix operations.

[0260] Neural network weight data can be stored in advance as a model container (509) within the neural system service (1110) through an API call of the neural system service (1110) during the application installation process.

[0261] Meanwhile, the basic weight data loaded into the model container (509) can be automatically converted and stored in various discretization levels during the system initialization process. For example, if the basic weight is defined as FP32, it can be sub-discretized into INT8, INT16, and FP16, and a total of four weight files can be stored.

[0262] Required input data refers to input signals required for the current neural network to operate, such as vehicle speed, current location, radar, lidar, camera images, and intermediate or final calculation results of the preceding neural network.

[0263] The input data can be transmitted in real time to the shared memory (508) within the hypervisor (505) through an interface operating through the central processor (175) in the server virtualization machine.

[0264] A command queue is a memory buffer of a sequential FIFO data structure that can define a series of orders for processing artificial neural networks through hardware accelerators.

[0265] A single neural network operation request entering the command queue can be transmitted along with metadata such as the application name, the location of the application virtualization machine, the storage destination of the operation result, hardware accelerator control settings, the memory location of the input data, and the memory location information for each discretization level of the weight data.

[0266] The hardware accelerator control settings may include a unique number of the hardware accelerator in charge of the operation, the current target discretization level of the weight data (INT8, INT16, FP16, FP32, etc.), and a target neural network weight location mapping table for each hardware accelerator internal memory address.

[0267] Meanwhile, the neural controller (1115) can schedule an optimized command queue based on the requested artificial neural network operation commands and the availability of current hardware resources, and control the actual hardware accelerator to match the expected operation of the command queue.

[0268] The neural controller (1115) can receive neural network operation requirements from the neural manager (1113) and optimize the command queue.

[0269] The optimization process, in other words, can be calculated through a simulated scheduling that checks the priority, dependency, and accuracy metadata for each slot in the current command queue, and applies various queue optimization techniques (such as Partition, Caching, and Accuracy Coding) to all candidate commands in the current command queue, and finds a combination that maximizes hardware utilization and minimizes the latency of individual operation requests within a unit of time.

[0270] Based on the optimal slot location obtained in this way, a weight file (learning model) can be requested from the neural manager (1113) and loaded into the hardware internal memory.

[0271] If two different neural networks are managed as a single virtual neural network using Partition as an optimization technique and input as a hardware operation request, the start and end positions of the weights of the first operation corresponding to the address of the hardware internal memory can be recorded in a mapping table, and then the start and end positions of the weights of the second operation can be recorded in the mapping table.

[0272] Through this, the hardware accelerator performs the process of parallel processing of a virtual neural network, but the neural controller (1115) can separate the results of the operation into the results of the first operation and the second operation through a mapping table and transmit them separately to individual applications.

[0273] After completing the above initialization process, the neural controller (1115) can receive a sequential processing request of the command queue from the neural manager (1113).

[0274] At this time, the neural controller (1115) can be controlled to extract input data prepared in advance by the neural manager (1113) from the input data queue, pair the neural network weights with the corresponding input data, and perform computational processing through the hardware accelerator API.

[0275] If, unlike the initial driving requirements, the discretization level of the current neural network is changed according to a specific situation, the neural controller (1115) can perform a bitwise concanate operation that concatenates the weight conversion difference value (Delta) of the hardware internal memory to the basic weight of the current internal memory, thereby converting the discretization level of the basic weight of the internal memory in real time.

[0276] Meanwhile, the central processor (175) executes an application for driving the vehicle, and when it is determined that the application has failed to operate, it controls a second application corresponding to the application to be executed in another central processor (175) or another signal processing device (170), and varies the standard fallback guarantee time for the application's operation failure based on the safety level of the application.

[0277] Meanwhile, the above-described fallback guarantee time may mean the time from the fallback start time to the fallback end time.

[0278] Alternatively, the fallback guarantee time may mean the time from when the application determines that the operation has failed or when a failure has been determined to the time when the fallback starts to the time when the fallback ends.

[0279] Meanwhile, the safety level may mean the Automotive Safety Integrity Level (ASIL), the autonomous driving level, or a combination of the Automotive Safety Integrity Level and the autonomous driving level.

[0280] Accordingly, applications for vehicle operation can be performed reliably. In particular, applications for vehicle operation can be performed reliably based on safety levels.

[0281] Meanwhile, the central processor (175) may set the reference fallback guarantee time to a first time period when the safety level of the application is the corresponding first safety level, and may set the reference fallback guarantee time to a second time period longer than the first time period when the safety level of the application is the second safety level higher than the first safety level. Accordingly, the application for vehicle driving can be stably performed.

[0282] For example, the central processor (175) may set the standard fallback guarantee time to a first time, approximately 10 seconds, for a first application corresponding to ASIL D when the autonomous driving level is Level 3, and may set the standard fallback guarantee time to a second time, approximately 30 seconds, for a second application corresponding to ASIL D when the autonomous driving level is Level 4. Accordingly, it is possible to stably perform applications for driving a vehicle based on the safety level.

[0283] As another example, the central processor (175) may set the standard fallback guarantee time to approximately 7 seconds for a third application corresponding to ASIL B when the autonomous driving level is Level 3, and may set the standard fallback guarantee time to approximately 10 seconds for a fourth application corresponding to ASIL D when the autonomous driving level is Level 3. Accordingly, it is possible to stably perform applications for driving the vehicle based on the safety level.

[0284] As another example, the central processor (175) may set the reference fallback guaranteed time to a first time period of approximately 10 seconds when the safety level of the driver monitoring system application (Ndm) is a first safety level corresponding to ASIL B, and may set the reference fallback guaranteed time to a second time period of approximately 30 seconds when the safety level of the vehicle driving assistance application (Nad) is a second safety level corresponding to ASIL D, which is higher than ASIL B. Accordingly, it is possible to stably perform an application for driving the vehicle based on the safety level.

[0285] Meanwhile, the central processor (175) can set the standard fallback guarantee time to a third time shorter than the first time when the safety level of the application is a third safety level lower than the first safety level. Accordingly, the application for vehicle driving can be stably performed.

[0286] For example, the central processor (175) can set the standard fallback guarantee time to approximately 1 second, which is the third time, for the fifth application corresponding to ASIL D or ASIL B when the autonomous driving level is Level 2.

[0287] As another example, the central processor (175) may set the standard fallback guarantee time to approximately 0.7 seconds for the sixth application corresponding to QM when the autonomous driving level is Level 2.

[0288] As another example, the central processor (175) may set the standard fallback guarantee time to the third time, approximately 0.5 seconds, when the safety level of the augmented reality application (Nar) corresponds to a QM lower than ASIL B. Accordingly, the application for driving the vehicle can be stably performed based on the safety level.

[0289] Meanwhile, the central processor (175) can control the standard fallback guarantee time of the application executed in the second virtual machine (850) among the plurality of virtual machines (810, 830, 850) to be greater than the standard fallback guarantee time of the application executed in the third virtual machine, when the second virtual machine (850) executes an application with a higher security level than the third virtual machine (830).

[0290] For example, the central processor (175) can be set so that when the second virtual machine (850) executes a first application with an autonomous driving level of Level 4, the standard fallback guarantee time is approximately 30 seconds, and when the third virtual machine (830) executes a second application with an autonomous driving level of Level 3, the standard fallback guarantee time is approximately 10 seconds. Accordingly, it is possible to stably perform applications for vehicle driving based on a safety level.

[0291] As another example, the central processor (175) may set the standard fallback guarantee time of the vehicle driving assistance application (Nad) to be approximately 30 seconds when the second virtual machine (850) executes a vehicle driving assistance application (Nad) corresponding to ASIL D, and may set the standard fallback guarantee time of the driver monitoring system application (Ndm) to be approximately 10 seconds when the third virtual machine (830) executes a driver monitoring system application (Ndm) corresponding to ASIL B. Accordingly, it is possible to stably perform applications for vehicle driving based on a safety level.

[0292] FIG. 9 is another example of a block diagram of a vehicle display device according to an embodiment of the present disclosure.

[0293] Referring to the drawing, a signal processing system (1200) in a vehicle display device according to an embodiment of the present disclosure may include a first central signal processing device (170a1) and a second central signal processing device (170a2).

[0294] Meanwhile, the first central signal processing unit (170a1) may be a primary signal processing unit, and the second central signal processing unit (170a2) may be a secondary signal processing unit.

[0295] That is, when a failure of the first central signal processing device (170a1) is determined, the second central signal processing device (170a2) can operate as a fallback.

[0296] At this time, the first central signal processing device (170a1) may correspond to the central signal processing device (170) of FIG. 8.

[0297] Meanwhile, the second central signal processing unit (170a2), similar to the central signal processing unit (170) of FIG. 8, has a central processor (175b) and at least one neural processor (179b).

[0298] Meanwhile, the second central signal processing device (170a2) according to the embodiment of the present disclosure may further include a graphics processor (178b).

[0299] Meanwhile, the central processor (175b) according to the embodiment of the present disclosure can execute a hypervisor (505b).

[0300] Meanwhile, a system (1100b) driven by a second central signal processing device (170a2) according to an embodiment of the present disclosure can execute multiple virtual machines (810b to 850b) on a hypervisor (505b).

[0301] Specifically, the central processor (175b) in the second central signal processing device (170a2) according to the embodiment of the present disclosure can execute a hypervisor (505b) and execute a plurality of virtual machines (810 to 850b) on the hypervisor (505b).

[0302] Meanwhile, among the multiple virtualization machines (810b to 850b), the first virtualization machine (810b), which is a server virtualization machine, can control the operation of the neural processor (179b).

[0303] Meanwhile, among the multiple virtualization machines (810b to 850b), the second virtualization machine (850b) and the third virtualization machine (830b), which are guest virtualization machines, can each execute applications.

[0304] In the drawing, a second virtual machine (850b) is illustrated executing an ADAS application (Nadb), and a third virtual machine (830b) is illustrated executing a driver monitoring system (DMS) application (Ndmb) and an augmented reality (AR) application (Narb).

[0305] Meanwhile, the first virtual machine (810b) can execute a neural system service (1110b) to control at least one neural processor (179b).

[0306] Meanwhile, the neural system service (1110b) may execute or include a neural manager (1113b) for managing at least one neural processor (179b), a neural controller (1115b) for determining or controlling an inference method of at least one neural processor (179b), and a neural interface (1118b) for interfacing with at least one neural processor (179b).

[0307] Meanwhile, the neural system service (1110b) may further execute or include a model container (509b) that performs interface of model parameters related to the operation of the neural processor (179b) and version management of learning files.

[0308] For example, if the vehicle driving assistance application (Nad) is determined to have an operation failure or malfunction during execution in the first central signal processing unit (170a1), the second central signal processing unit (170a2) may, as a substitute operation, execute the second vehicle driving assistance application (Nadb) corresponding to the vehicle driving assistance application (Nad). Accordingly, the application for vehicle driving can be stably executed.

[0309] As another example, if the first central signal processing unit (170a1) determines that the autonomous driving application has failed or malfunctioned while running, the second central signal processing unit (170a2) can, as a replacement, execute a second autonomous driving application corresponding to the autonomous driving application. This enables stable execution of the application for vehicle driving.

[0310] As another example, if the driver monitoring system application (Ndm) is determined to have an operational failure or malfunction during execution in the first central signal processing unit (170a1), the second central signal processing unit (170a2) may, as a substitute operation, execute the second driver monitoring system application (Ndmb) corresponding to the driver monitoring system application (Ndm). Accordingly, the application for vehicle driving can be stably executed.

[0311] Meanwhile, it is preferable that the standard fallback guarantee time when the vehicle driving assistance application (Nad) is executed in the first central signal processing unit (170a1) be greater than the standard fallback guarantee time when the driver monitoring system application (Ndm) is executed in the first central signal processing unit (170a1).

[0312] In particular, when the safety level of the vehicle driving assistance application (Nad) is greater than the safety level of the driver monitoring system application (Ndm), it is preferable that the standard fallback guarantee time when the vehicle driving assistance application (Nad) is executed be greater than the standard fallback guarantee time when the driver monitoring system application (Ndm) is executed. Accordingly, the application for driving the vehicle can be stably performed based on the safety level.

[0313] Fig. 10 is a flowchart showing an operation method of a vehicle display device according to an embodiment of the present disclosure.

[0314] Referring to the drawing, the central processor (175) in the signal processing device (170) can execute an application for driving the vehicle (S1010).

[0315] Meanwhile, the central processor (175) in the signal processing device (170) can be controlled to calculate a fallback guaranteed time corresponding to the replacement expected time based on the vehicle's sensor data or camera data or the operating status of the signal processing device (S1015).

[0316] For example, the central processor (175) may be controlled to calculate a fallback guarantee time based on at least one of the temperature of the system, performance based on a safety level, error history of the system, delay of an application, or load of the system.

[0317] Specifically, the neural processor (179) can perform neural processing based on at least one of the following: system temperature, performance based on safety levels, system error history, application delay, or system load, and calculate a guaranteed fallback time based on the neural processing. Accordingly, applications for vehicle driving can be stably performed.

[0318] Meanwhile, the central processor (175) within the signal processing device (170) determines whether the application has failed to operate (S1020), and if so, compares the calculated fallback guarantee time with the standard fallback guarantee time (S1025), and if the calculated fallback guarantee time is less than the standard fallback guarantee time, controls the fallback operation to be performed (S1030).

[0319] For example, the central processor (175) within the signal processing device (170) can control one of a plurality of fallback operations to be performed when the calculated fallback guarantee time is less than the reference fallback guarantee time.

[0320] Multiple fallback actions at this time may include a driver takeover request, blind brake, lane keeping & blind braking, emergency lane change & braking, pull over on the shoulder, or long distance route planning.

[0321] Meanwhile, in step 1025 (S1025), the central processor (175) can control a preset operation to be performed if the calculated fallback guarantee time is greater than or equal to the reference fallback guarantee time (S1035).

[0322] Meanwhile, the central processor (175) can control a preset fail operation to be performed when the calculated fallback guarantee time is greater than or equal to the reference fallback guarantee time. At this time, it is preferable that an appropriate fail operation be performed.

[0323] For example, if the calculated fallback guarantee time is less than the reference fallback guarantee time, the central processor (175) can control a second application corresponding to the application to be executed on another central processor (175) or another signal processing device (170). Accordingly, the application for driving the vehicle can be stably executed.

[0324] Meanwhile, the central processor (175) can control the execution of an application with a high safety level to be restricted as the calculated fallback guaranteed time decreases and the difference between the calculated estimated reference fallback guaranteed time and the reference fallback guaranteed time becomes smaller, while the calculated fallback guaranteed time is greater than the reference fallback guaranteed time.

[0325] For example, the central processor (175) may control an application corresponding to a first safety level to be executed when the difference between the calculated estimated reference fallback guaranteed time and the reference fallback guaranteed time is a first time, in a state where the calculated fallback guaranteed time is greater than the reference fallback guaranteed time, and may control an application corresponding to a second safety level higher than the first safety level to be executed when the difference between the calculated estimated reference fallback guaranteed time and the reference fallback guaranteed time is a second time, which is greater than the first time.

[0326] That is, the central processor (175) controls an application corresponding to a second safety level higher than the first safety level to be executed when the difference between the calculated estimated reference fallback guaranteed time and the reference fallback guaranteed time is a second time that is greater than the first time, while the calculated estimated reference fallback guaranteed time is greater than the reference fallback guaranteed time, and controls an application corresponding to a second safety level higher than the first safety level to be executed when the difference between the calculated estimated reference fallback guaranteed time and the reference fallback guaranteed time is the first time, and restricts the execution of an application corresponding to the second safety level and controls an application corresponding to the first safety level to be executed instead.

[0327] Specifically, the central processor (175) can control the execution of a driver monitoring system application (Ndm) corresponding to a first safety level when the difference between the calculated estimated reference fallback guaranteed time and the reference fallback guaranteed time is a first time, and can control the execution of a vehicle driving assistance application (Nad) having a higher safety level than the driver monitoring system application (Ndm) when the difference between the reference fallback guaranteed time and the reference fallback guaranteed time is a second time that is greater than the first time. Accordingly, it is possible to stably perform an application for driving a vehicle based on a safety level.

[0328] Meanwhile, the central processor (175) can vary the standard fallback guarantee time depending on the driving environment of the vehicle.

[0329] For example, the central processor (175) can control the reference fallback guarantee time to increase as the vehicle's driving distance increases.

[0330] As another example, the central processor (175) can control the standard fallback guarantee time to increase as the number of vehicle component maintenance operations increases. Accordingly, applications can be reliably performed in response to the vehicle's condition.

[0331] Meanwhile, unlike the drawing, between steps 1010 (S1010) and 1015 (S1015), the central processor (175) within the signal processing device (170) may perform learning on a correctable failure based on sensor data or camera data of the vehicle or the operating status of the signal processing device, and may control to perform preemptive control based on the learning.

[0332] That is, the central processor (175) within the signal processing device (170) can perform learning on correctable failures during execution of the application and, based on the learning, perform preemptive control before the application fails or malfunctions.

[0333] Preemptive control at this time may include performing route changes that take into account the safety zone, or displaying autonomous driving level limits and warning signs based on warning signs in response to correctable failures through a display.

[0334] Meanwhile, the central processor (175) within the signal processing device (170) can control the strength of the preemptive control to vary depending on the safety level when performing preemptive control.

[0335] For example, the central processor (175) within the signal processing device (170) can control the intensity of the preemptive control to be greater when the autonomous driving level is level 4 than when the autonomous driving level is level 3 when performing preemptive control. Accordingly, stable control becomes possible.

[0336] Meanwhile, the central processor (175) within the signal processing device (170) can control the fallback operation to be performed based on the result of comparing the calculated pullback entry time with the reference fallback entry time when, despite preemptive control or the like, an operation failure or malfunction is determined during the execution of the application, as in step S1020. Accordingly, the application for driving the vehicle can be stably performed.

[0337] Figures 11 to 13 are drawings referenced in the operation description of Figure 9 or Figure 10.

[0338] Figure 11 is a diagram illustrating the calculation of the fallback guarantee time using a neural network.

[0339] Referring to the drawing, the neural processor (179) can calculate the fallback guarantee time using a neural network (1220).

[0340] Meanwhile, the neural processor (179) can set the hardware monitoring information (1203) of the signal processing device (170), vehicle status information (1207), and system information (1209) of the signal processing device as input layers and use a neural network to calculate the fallback guarantee time.

[0341] Meanwhile, the hardware monitoring information (1203) of the signal processing device (170) may include performance information based on the temperature information and safety level of the system of the signal processing device (170).

[0342] Meanwhile, vehicle status information (1207) may include vehicle error repair history information or system error history.

[0343] Meanwhile, the system information (1209) of the signal processing device may include the system's error history, application delay information, or system load information.

[0344] That is, the neural processor (179) can perform neural processing based on at least one of the temperature of the system, performance based on a safety level, error history of the system, delay of the application, or load of the system, and output result data based on the neural processing.

[0345] The result data at this time may include the calculated fallback guarantee time (1223) and power data or temperature data (1224).

[0346] Meanwhile, the neural processor (179) can perform neural processing by feeding back power data or temperature data among the neural processing result data. Accordingly, applications for vehicle driving can be stably performed.

[0347] Meanwhile, the neural network may include a state-of-the-art neural network for efficient learning.

[0348] For example, the neural network may include a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a transformer-based neural network utilizing federated learning or self-attention.

[0349] Meanwhile, the neural network model can be varied based on vehicle mileage, driving history, repair or parts replacement history, application updates, hardware updates, etc.

[0350] Alternatively, the neural processor (179) may calculate the fallback guarantee time based on vehicle mileage, driving history, repair or parts replacement history, application updates, hardware updates, etc.

[0351] FIG. 12 is a diagram for reference in explaining the operation of a fail operation manager based on the calculated fallback guarantee time of FIG. 11.

[0352] Referring to the drawing, the fail operation manager (819) can operate based on the calculated fallback guarantee time (1223).

[0353] The fail operation manager (819) can control to perform a fallback operation (Sfa) or a fail operation (Sop) based on the result of comparing the calculated fallback guarantee time and the reference fallback guarantee time.

[0354] For example, the fail action manager (819) can control one of the multiple fallback actions to be performed when the calculated fallback guarantee time is less than the reference fallback guarantee time.

[0355] As another example, the fail operation manager (819) can control a preset fail operation (Sop) to be performed when the calculated fallback guarantee time is greater than or equal to the reference fallback guarantee time.

[0356] Meanwhile, in order to perform a fail operation, the fail operation manager (819) can control one of a plurality of fail operations (Sop) to be performed based on, in addition to the calculated fallback guarantee time and the standard fallback guarantee time, a Non-corrected Failure (1302), a driving level (SAE Level) (1305), a vehicle driving state (1307), or a driving application (1304). Accordingly, the application for vehicle driving can be stably performed.

[0357] Figure 13 is a diagram illustrating multiple fallback operations.

[0358] Referring to the drawing, the central processor (175) can control one of a plurality of fallback operations to be performed when the calculated fallback guarantee time is less than the reference fallback guarantee time.

[0359] The multiple fallback actions at this time may include, as shown in the drawing, a vehicle handover request, blind braking (S0), lane-keeping braking (S1), lane-changing braking (S2), shoulder stopping, or path-based stopping. Accordingly, applications for vehicle driving can be performed stably.

[0360] Meanwhile, the central processor (175) in the signal processing device (170) according to another embodiment of the present disclosure varies the standard fallback guarantee time for application operation failure based on the safety level of the application, and controls the fallback operation or fail operation to be performed when the application operation failure is determined based on the standard fallback guarantee time and the calculated fallback guarantee time. Accordingly, the application for vehicle driving can be stably performed. In particular, the application for vehicle driving can be stably performed based on the safety level.

[0361] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person skilled in the art to which the present invention pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. In a signal processing device, At least one neural processor; A central processor that runs an application for driving the vehicle; The central processor, If the above application is judged to have failed to operate, a second application corresponding to the above application is controlled to be executed in another central processor or another signal processing device. A signal processing device that varies the standard fallback guarantee time for operation failure of the application based on the safety level of the application.

2. In paragraph 1, The central processor, If the safety level of the above application is the first safety level, the above-mentioned standard fallback guarantee time is set to the first time, A signal processing device that sets the reference fallback guarantee time to a second time longer than the first time when the safety level of the application is a second safety level higher than the first safety level.

3. In paragraph 2, The central processor, A signal processing device that sets the reference fallback guarantee time to a third time shorter than the first time when the safety level of the application is a third safety level lower than the first safety level.

4. In paragraph 1, The central processor, A signal processing device that controls a preset operation to be performed when the calculated fallback guarantee time is greater than or equal to the above-mentioned standard fallback guarantee time.

5. In paragraph 1, The central processor, A signal processing device that controls a second application corresponding to the application to be executed in another central processor or another signal processing device when the calculated fallback guarantee time is less than the above-mentioned standard fallback guarantee time.

6. In paragraph 1, The central processor, A signal processing device that controls a fallback operation to be performed when the calculated fallback guarantee time is less than the above-mentioned standard fallback guarantee time.

7. In paragraph 1, The central processor, A signal processing device that controls one of a plurality of fallback operations to be performed when the calculated fallback guarantee time is less than the above-mentioned reference fallback guarantee time.

8. In paragraph 7, The above multiple fallback operations are: A signal processing device including a vehicle handover request, blind braking, lane keeping and braking, lane change and braking, shoulder stopping, or path-based stopping.

9. In paragraph 1, The central processor, A signal processing device that controls execution of a high-safety level application so that, when the calculated fallback guarantee time is greater than the reference fallback guarantee time, the calculated fallback guarantee time decreases, and the difference between the calculated estimated reference fallback guarantee time and the reference fallback guarantee time becomes smaller.

10. In paragraph 1, The central processor, If the difference between the calculated estimated above-mentioned reference fallback guaranteed time and the above-mentioned reference fallback guaranteed time is the first time, the application corresponding to the first safety level is controlled to be executed, A signal processing device that controls an application corresponding to a second safety level higher than the first safety level to be executed when the difference between the calculated estimated reference fallback guarantee time and the reference fallback guarantee time is a second time greater than the first time.

11. In paragraph 1, The central processor, A signal processing device that varies the standard fallback guarantee time depending on the driving environment of the vehicle.

12. In paragraph 1, The central processor, A signal processing device that controls to calculate a fallback guarantee time based on at least one of a temperature of the system, performance based on the safety level, an error history of the system, a delay of the application, or a load of the system.

13. In paragraph 1, The above neural processor, A signal processing device that performs neural processing based on at least one of the temperature of the system, performance based on the safety level, error history of the system, delay of the application, or load of the system, and calculates a fallback guarantee time based on the neural processing.

14. In paragraph 13, The above neural processor, A signal processing device that performs the neural processing by feeding back power data or temperature data among the result data of the neural processing.

15. In paragraph 1, The central processor, Run a hypervisor and run multiple virtual machines on the hypervisor, Among the above multiple virtual machines, the first virtual machine is: A signal processing device that executes a neural system service for controlling the above neural processor.

16. In paragraph 15, Among the above multiple virtual machines, the second virtual machine is: Run an Advanced Driver Assistance Systems (ADAS) application, A signal processing device in which a third virtual machine among the plurality of virtual machines executes a driver monitoring system (DMS) application or an augmented reality application.

17. In paragraph 15, The central processor, If the second virtual machine among the above multiple virtual machines runs an application with a higher security level than the third virtual machine, A signal processing device that controls the standard fallback guarantee time of an application executed in the second virtual machine to be greater than the standard fallback guarantee time of an application executed in the third virtual machine.

18. A central processor for executing an application for driving the vehicle; The central processor, Based on the safety level of the above application, the standard fallback guarantee time for the operation failure of the above application is varied. A signal processing device that controls a fallback operation or a fail operation to be performed when the application is determined to have failed to operate based on the above-mentioned standard fallback guarantee time and the calculated fallback guarantee time.

19. At least one display; A signal processing device for outputting a video signal to the display; The above signal processing device, A vehicle display device comprising a signal processing device according to any one of claims 1 to 18.

Images