Signal processing device and vehicle display device comprising same

Abstract

A signal processing device and a vehicle display device comprising same, according to an embodiment of the present disclosure, comprise a processor which executes a core orchestrator and a vehicle orchestrator, wherein the processor executes at least one application on the basis of a container runtime, and the vehicle orchestrator performs monitoring and management of applications, microservices, and device resources on the basis of communication with the core orchestrator. Accordingly, a mixed operation of microservices can be performed.

Description

Signal processing device and vehicle display device having the same

[0001] The present disclosure relates to a signal processing device and a vehicle display device having the same, and more particularly, to a signal processing device capable of performing mixed operation of microservices and a vehicle display device having the same.

[0002] A vehicle is a device that allows the user to move in the desired direction. A representative example is an automobile.

[0003] Meanwhile, for the convenience of vehicle users, a vehicle signal processing device is installed inside the vehicle.

[0004] The signal processing device inside the vehicle receives and processes sensor data from various sensor devices inside the vehicle.

[0005] Meanwhile, as the types and number of sensors installed in vehicles increase due to advanced driver assistance systems (ADAS) and autonomous driving, the amount of data that needs to be processed is increasing, and the number of applications being executed is also increasing.

[0006] On the other hand, if the distribution or installation of applications running on a vehicle is through a different route or from a different source, there is a disadvantage in that related operations or updates between applications cannot be performed.

[0007] The problem to be solved by the present disclosure is to provide a signal processing device capable of performing mixed operation of microservices and a vehicle display device equipped with the same.

[0008] Another problem that the present disclosure seeks to solve is to provide a signal processing device capable of performing partial connection or partial sharing of heterogeneous applications or heterogeneous microservices and a vehicle display device having the same.

[0009] Another problem that the present disclosure seeks to solve is to provide a signal processing device capable of managing an associated application or microservice during application execution and a vehicle display device having the same.

[0010] A signal processing device and a vehicle display device including the same according to one embodiment of the present disclosure include a core orchestrator and a processor that executes the vehicle orchestrator, the processor executes at least one application based on a container runtime, and the vehicle orchestrator performs monitoring and management of applications, microservices, and device resources based on communication with the core orchestrator.

[0011] Meanwhile, the vehicle orchestrator can select and control target applications or target microservices.

[0012] Meanwhile, the vehicle orchestrator can control some connection or some sharing of heterogeneous applications or heterogeneous microservices based on communication with the core orchestrator.

[0013] Meanwhile, the vehicle orchestrator can set the update time of the running application based on communication with the core orchestrator.

[0014] Meanwhile, the vehicle orchestrator can set the update timing of the application running while the vehicle is driving.

[0015] Meanwhile, the vehicle orchestrator or core orchestrator can control the execution, update, or termination of the second application based on sensor data corresponding to the first application when the first application and the second application are executed.

[0016] Meanwhile, the vehicle orchestrator can monitor the microservice corresponding to the first application and control the execution, update, or termination of the second application based on the device's resources.

[0017] Meanwhile, the vehicle orchestrator can control the microservices of the first application or the microservices of the second application based on the device resources and communication nodes when the first application and the second application are executed.

[0018] Meanwhile, the vehicle orchestrator can monitor vehicle speed data according to a vehicle speed-related application, and control the vehicle light application to terminate if the vehicle speed data is higher than a predetermined value while the vehicle light application is running.

[0019] Meanwhile, the vehicle orchestrator can control the lane keeping application to run while the driver monitoring application is running.

[0020] Meanwhile, the vehicle orchestrator can control the lane keeping application to be updated based on the device's resources while the driver monitoring application is running.

[0021] Meanwhile, the vehicle orchestrator can perform gateway management and node management of worker nodes and master nodes.

[0022] Meanwhile, the vehicle orchestrator can perform safety management and fail-safe mode management for vehicle safety.

[0023] Meanwhile, the vehicle orchestrator can perform fail-safe mode management, including monitoring and managing the main first microservice or the redundant second microservice.

[0024] Meanwhile, the vehicle orchestrator can control the transmission of a recovery command to the first microservice and an activity command to the second microservice when a failure occurs in the first microservice.

[0025] Meanwhile, the first microservice or the second microservice may be a safety monitor service.

[0026] Meanwhile, the vehicle orchestrator can deploy, run, or manage multiple microservices for the safety monitor service.

[0027] Meanwhile, the vehicle orchestrator can control the operation transition from the first microservice to the second microservice based on the monitoring score or analysis score.

[0028] A signal processing device and a vehicle display device including the same according to another embodiment of the present disclosure include a core orchestrator and a processor for executing the vehicle orchestrator, the processor for executing at least one application based on a container runtime, and the vehicle orchestrator for controlling, based on communication with the core orchestrator, a partial connection or partial sharing of heterogeneous applications or heterogeneous microservices.

[0029] Meanwhile, the vehicle orchestrator or core orchestrator can control the execution, update, or termination of the second application based on sensor data corresponding to the first application when the first application and the second application are executed.

[0030] A signal processing device and a vehicle display device including the same according to one embodiment of the present disclosure include a core orchestrator and a processor for executing the vehicle orchestrator, wherein the processor executes at least one application based on a container runtime, and the vehicle orchestrator monitors and manages applications, microservices, and device resources based on communication with the core orchestrator. Accordingly, mixed operation of microservices can be performed.

[0031] Meanwhile, the vehicle orchestrator can select and control target applications or microservices, enabling mixed microservice operations.

[0032] Meanwhile, the vehicle orchestrator can control partial connection or sharing of heterogeneous applications or heterogeneous microservices based on communication with the core orchestrator. This enables partial connection or sharing of heterogeneous applications or heterogeneous microservices.

[0033] Meanwhile, the vehicle orchestrator can set update timings for running applications based on communication with the core orchestrator. This enables mixed operation of microservices.

[0034] Meanwhile, the vehicle orchestrator can set update timings for applications running while the vehicle is in motion. This enables mixed operation of microservices.

[0035] Meanwhile, the vehicle orchestrator or core orchestrator can control the execution, update, or termination of the second application based on sensor data corresponding to the first application when the first and second applications are running. This allows for the management of associated applications or microservices during application execution.

[0036] Meanwhile, the vehicle orchestrator can monitor the microservices corresponding to the first application and control the execution, update, or termination of the second application based on the device's resources. This allows for the management of associated applications or microservices during application execution.

[0037] Meanwhile, the vehicle orchestrator can control the microservices of the first application or the microservices of the second application based on device resources and communication nodes when the first and second applications are running. This allows for the management of associated applications or microservices during application execution.

[0038] Meanwhile, the vehicle orchestrator can monitor vehicle speed data based on vehicle speed-related applications and, if the vehicle speed data exceeds a predetermined value while the vehicle light application is running, control the termination of the vehicle light application. This allows for the management of related applications or microservices while the application is running.

[0039] Meanwhile, the vehicle orchestrator can control the execution of the lane-keeping application while the driver monitoring application is running. This allows for the management of related applications or microservices while the application is running.

[0040] Meanwhile, the vehicle orchestrator can control the lane-keeping application to update based on the device's resources while the driver monitoring application is running. This allows for the management of related applications or microservices while the application is running.

[0041] Meanwhile, the vehicle orchestrator can manage gateways and node management for worker and master nodes. This enables mixed operation of microservices.

[0042] Meanwhile, the vehicle orchestrator can perform safety management and fail-safe mode management for vehicle safety. This can improve the stability of applications or microservices.

[0043] Meanwhile, the vehicle orchestrator can perform fail-safe mode management, including monitoring and managing the primary microservice or the redundant secondary microservice. This can improve the stability of the application or microservice.

[0044] Meanwhile, the vehicle orchestrator can control the system to send a recovery command to the first microservice and an activation command to the second microservice when a failure occurs in the first microservice. This can improve the stability of the application or microservice.

[0045] Meanwhile, either the first or second microservice may be a safety monitor service. This allows for mixed operation of microservices.

[0046] Meanwhile, the vehicle orchestrator can deploy, run, or manage multiple microservices for the safety monitoring service. This improves the stability of the application or microservice.

[0047] Meanwhile, the vehicle orchestrator can control the transition from the first microservice to the second microservice based on monitoring or analysis scores. This can improve the stability of the application or microservice.

[0048] A signal processing device and a vehicle display device including the same according to another embodiment of the present disclosure include a core orchestrator and a processor for executing the vehicle orchestrator, wherein the processor executes at least one application based on a container runtime, and the vehicle orchestrator controls, based on communication with the core orchestrator, partial connection or partial sharing of heterogeneous applications or heterogeneous microservices. Accordingly, mixed operation of microservices can be performed.

[0049] Meanwhile, the vehicle orchestrator or core orchestrator can control the execution, update, or termination of the second application based on sensor data corresponding to the first application when the first and second applications are running. This allows for the management of associated applications or microservices during application execution.

[0050] Figure 1 is a drawing showing an example of the exterior and interior of a vehicle.

[0051] Figures 2 to 2c are drawings illustrating various architectures of a vehicle communication gateway.

[0052] Figure 3a is a drawing showing an example of the arrangement of a vehicle display device inside a vehicle.

[0053] Figure 3b is a drawing showing another example of the arrangement of a vehicle display device inside a vehicle.

[0054] Fig. 4 is an example of an internal block diagram of the vehicle display device of Fig. 3b.

[0055] FIGS. 5A to 5D are drawings showing various examples of vehicle display devices.

[0056] FIG. 6 is an example of a block diagram of a vehicle display device according to an embodiment of the present disclosure.

[0057] FIGS. 7A and 7B are drawings for reference in the description of a signal processing device related to the present disclosure.

[0058] FIGS. 8A to 8E are diagrams illustrating various examples of execution of microservices according to embodiments of the present disclosure.

[0059] FIG. 9 is an example of an internal block diagram of a signal processing device according to an embodiment of the present disclosure.

[0060] FIG. 10 is an example of an internal block diagram of a signal processing device according to another embodiment of the present disclosure.

[0061] FIG. 11 is a diagram illustrating a vehicle signal processing system according to an embodiment of the present disclosure.

[0062] Fig. 12 is an example of an internal block diagram of the signal processing device of Fig. 11.

[0063] Figures 13a to 19d are drawings referenced in the description of Figure 12.

[0064] Hereinafter, the present disclosure will be described in more detail with reference to the drawings.

[0065] The suffixes "module" and "part" used in the following description are given solely for the convenience of writing this specification and do not impart any particularly significant meaning or role to the components themselves. Therefore, the terms "module" and "part" may be used interchangeably.

[0066] Figure 1 is a drawing showing an example of the exterior and interior of a vehicle.

[0067] Referring to the drawing, the vehicle (200) is operated by a plurality of wheels (103FR, 103FL, 103RL, etc.) that rotate by a power source and a steering wheel (150) for controlling the direction of travel of the vehicle (200).

[0068] Meanwhile, the vehicle (200) may further be equipped with a camera (195) for capturing images of the front of the vehicle.

[0069] Meanwhile, the vehicle (200) may be equipped with multiple displays (180a, 180b) for displaying images, information, etc. inside.

[0070] In Fig. 1, a cluster display (180a) and an AVN (Audio Video Navigation) display (180b) are exemplified as multiple displays (180a, 180b). In addition, a HUD (Head Up Display) is also possible.

[0071] Meanwhile, the AVN (Audio Video Navigation) display (180b) may also be named a center information display.

[0072] Meanwhile, the vehicle (200) described in this specification may be a concept that includes all of a vehicle equipped with an engine as a power source, a hybrid vehicle equipped with an engine and an electric motor as a power source, and an electric vehicle equipped with an electric motor as a power source.

[0073] Figures 2 to 2c are drawings illustrating various architectures of a vehicle communication gateway.

[0074] First, Fig. 2 is a drawing illustrating the first architecture of a vehicle communication gateway.

[0075] Referring to the drawing, the first architecture (300a) can correspond to a zone-based architecture.

[0076] Accordingly, sensor devices and processors inside the vehicle may be placed in each of the plurality of zones (Z1 to Z4), and a signal processing device (170a) including a vehicle communication gateway (GWDa) may be placed in the central area of ​​the plurality of zones (Z1 to Z4).

[0077] Meanwhile, the signal processing device (170a) may further include, in addition to the vehicle communication gateway (GWDa), an autonomous driving control module (ACC), a cockpit control module (CPG), etc.

[0078] The vehicle communication gateway (GWDa) within the signal processing device (170a) may be an HPC (High Performance Computing) gateway.

[0079] That is, the signal processing device (170a) of FIG. 2 is an integrated HPC and can exchange data with an external communication module (not shown) or a processor (not shown) within a plurality of zones (Z1 to Z4).

[0080] Figure 3a is a drawing showing an example of the arrangement of a vehicle display device inside a vehicle.

[0081] Referring to the drawing, the interior of the vehicle may be equipped with a cluster display (180a), an AVN (Audio Video Navigation) display (180b), a rear seat entertainment display (180c, 180d), a room mirror display (not shown), etc.

[0082] Figure 3b is a drawing showing another example of the arrangement of a vehicle display device inside a vehicle.

[0083] A vehicle display device (100) according to an embodiment of the present disclosure may include a plurality of displays (180a to 180b), and a signal processing device (170) that performs signal processing for displaying images, information, etc. on the plurality of displays (180a to 180b) and outputs an image signal to at least one display (180a to 180b).

[0084] Among the plurality of displays (180a to 180b), the first display (180a) may be a cluster display (180a) for displaying driving status, operation information, etc., and the second display (180b) may be an AVN (Audio Video Navigation) display (180b) for displaying vehicle driving information, a navigation map, various entertainment information, or images.

[0085] The signal processing device (170) has a processor (175) therein and can execute a first virtual machine to a third virtual machine (not shown) on a hypervisor (not shown) within the processor (175).

[0086] A second virtual machine (not shown) can operate for the first display (180a), and a third virtual machine (not shown) can operate for the second display (180b).

[0087] Meanwhile, the first virtual machine (not shown) within the processor (175) can control the shared memory (508) based on the hypervisor (505) to be set for the same data transmission to the second virtual machine (not shown) and the third virtual machine (not shown). Accordingly, the same information or the same image can be displayed in synchronization on the first display (180a) and the second display (180b) within the vehicle.

[0088] Meanwhile, the first virtual machine (not shown) within the processor (175) shares at least a portion of data with the second virtual machine (not shown) and the third virtual machine (not shown) for data sharing processing. Accordingly, data can be shared and processed among multiple virtual machines for multiple displays within the vehicle.

[0089] Meanwhile, a first virtual machine (not shown) within a processor (175) may receive and process vehicle wheel speed sensor data, and transmit the processed wheel speed sensor data to at least one of a second virtual machine (not shown) or a third virtual machine (not shown). Accordingly, the vehicle wheel speed sensor data may be shared with at least one virtual machine.

[0090] Meanwhile, the vehicle display device (100) according to the embodiment of the present disclosure may further include a rear seat entertainment display (180c) for displaying driving status information, simple navigation information, various entertainment information, or images.

[0091] The signal processing device (170) can control the RSE display (180c) by executing a fourth virtual machine (not shown) in addition to the first virtual machine to the third virtual machine (not shown) on a hypervisor (not shown) within the processor (175).

[0092] Accordingly, it is possible to control various displays (180a to 180c) using one signal processing device (170).

[0093] Meanwhile, some of the multiple displays (180a~180c) may operate under Linux OS, while others may operate under Web OS.

[0094] The signal processing device (170) according to the embodiment of the present disclosure can control the same information or the same image to be displayed in synchronization on displays (180a to 180c) operating under various operating systems (OS).

[0095] Meanwhile, in FIG. 3b, a vehicle speed indicator (212a) and a vehicle interior temperature indicator (213a) are displayed on a first display (180a), a home screen (222) including a plurality of applications and a vehicle speed indicator (212b) and a vehicle interior temperature indicator (213b) are displayed on a second display (180b), and a second home screen (222b) including a plurality of applications and a vehicle interior temperature indicator (213c) are displayed on a third display (180c).

[0096] Fig. 4 is an example of an internal block diagram of the vehicle display device of Fig. 3b.

[0097] Referring to the drawings, a vehicle display device (100) according to an embodiment of the present disclosure may include an input unit (110), a communication unit (120) for communication with an external device, a plurality of communication modules (EMa to EMd) for internal communication, a memory (140), a signal processing unit (170), a plurality of displays (180a to 180c), an audio output unit (185), and a power supply unit (190).

[0098] A plurality of communication modules (EMa to EMd) can be arranged, for example, in a plurality of zones (Z1 to Z4) of FIG. 2, respectively.

[0099] Meanwhile, the signal processing device (170) may have a communication switch (736b) for data communication with each communication module (EM1 to EM4) inside.

[0100] Each communication module (EM1 to EM4) can perform data communication with multiple sensor devices (SN) or ECUs (770) or area signal processing devices (170Z).

[0101] Meanwhile, the plurality of sensor devices (SN) may include a camera (195), a lidar (196), a radar (197), or a position sensor (198).

[0102] The input unit (110) may be equipped with physical buttons, pads, etc. for button input, touch input, etc.

[0103] Meanwhile, the input unit (110) may be equipped with a microphone (not shown) for user voice input.

[0104] The communication unit (120) can exchange data wirelessly with a mobile terminal (800) or a server (900).

[0105] In particular, the communication unit (120) can wirelessly exchange data with the vehicle driver's mobile terminal. Various data communication methods are possible, such as Bluetooth, WiFi, WiFi Direct, and APiX.

[0106] The communication unit (120) can receive weather information, road traffic information, for example, TPEG (Transport Protocol Expert Group) information, from a mobile terminal (800) or a server (900). To this end, the communication unit (120) may be equipped with a mobile communication module (not shown).

[0107] A plurality of communication modules (EM1 to EM4) can receive sensor data, etc. from an ECU (770), a sensor device (SN), or an area signal processing device (170Z), and transmit the received sensor data to the signal processing device (170).

[0108] Here, the sensor data may include at least one of vehicle direction data, vehicle location data (GPS data), vehicle angle data, vehicle speed data, vehicle acceleration data, vehicle inclination data, vehicle forward / backward data, battery data, fuel data, tire data, vehicle lamp data, vehicle interior temperature data, and vehicle interior humidity data.

[0109] Such sensor data can be obtained from a heading sensor, a yaw sensor, a gyro sensor, a position module, a vehicle forward / backward sensor, a wheel sensor, a vehicle speed sensor, a body tilt detection sensor, a battery sensor, a fuel sensor, a tire sensor, a steering sensor by steering wheel rotation, a vehicle interior temperature sensor, a vehicle interior humidity sensor, etc.

[0110] Meanwhile, the position module may include a GPS module or a position sensor (198) for receiving GPS information.

[0111] Meanwhile, at least one of the plurality of communication modules (EM1 to EM4) can transmit location information data sensed by a GPS module or location sensor (198) to a signal processing device (170).

[0112] Meanwhile, at least one of the plurality of communication modules (EM1 to EM4) can receive vehicle front image data, vehicle side image data, vehicle rear image data, vehicle surrounding obstacle distance information, etc. from a camera (195), lidar (196), radar (197), etc., and transmit the received information to a signal processing device (170).

[0113] The memory (140) can store various data for the overall operation of the vehicle display device (100), such as a program for processing or controlling the signal processing device (170).

[0114] For example, the memory (140) may store data regarding a hypervisor, a first virtual machine, a third virtual machine, or the like, for execution within the processor (175).

[0115] The audio output unit (185) converts an electric signal from the signal processing device (170) into an audio signal and outputs it. For this purpose, a speaker or the like may be provided.

[0116] The power supply unit (190) can supply power required for the operation of each component under the control of the signal processing device (170). In particular, the power supply unit (190) can receive power from a battery or the like inside the vehicle.

[0117] The signal processing device (170) controls the overall operation of each unit within the vehicle display device (100).

[0118] For example, the signal processing device (170) may include a processor (175) that performs signal processing for a vehicle display (180a, 180b).

[0119] The processor (175) can execute a first virtual machine to a third virtual machine (not shown) on a hypervisor (not shown) within the processor (175).

[0120] Among the first virtual machine to the third virtual machine (not shown), the first virtual machine (not shown) may be named a server virtual machine (Server Virtual Maschine), and the second virtual machine to the third virtual machine (not shown) may be named a guest virtual machine (Guest Virtual Maschine).

[0121] For example, a first virtual machine (not shown) within a processor (175) may receive, process, or output sensor data from a plurality of sensor devices, such as vehicle sensor data, location information data, camera image data, audio data, or touch input data.

[0122] In this way, by performing most of the data processing in the first virtual machine (not shown), data sharing in a 1:N manner becomes possible.

[0123] As another example, a first virtual machine (not shown) can directly receive and process CAN data, Ethernet data, audio data, radio data, USB data, and wireless communication data for a second virtual machine or a third virtual machine (not shown).

[0124] And, the first virtual machine (not shown) can transmit processed data to the second virtual machine or the third virtual machine (not shown).

[0125] Accordingly, among the first virtual machine to the third virtual machine (not shown), only the first virtual machine (not shown) receives sensor data, communication data, or external input data from multiple sensor devices and performs signal processing, thereby reducing the signal processing burden on other virtual machines, enabling 1:N data communication, and enabling synchronization when sharing data.

[0126] Meanwhile, the first virtual machine (not shown) can control the second virtual machine (not shown) and the third virtual machine (not shown) to share the same data by writing data to the shared memory (508).

[0127] For example, a first virtual machine (not shown) can record vehicle sensor data, the location information data, the camera image data, or the touch input data in shared memory (508) and control the same data to be shared with a second virtual machine (not shown) and a third virtual machine (not shown). Accordingly, data sharing in a 1:N manner becomes possible.

[0128] Ultimately, by performing most of the data processing on the first virtual machine (not shown), data sharing in a 1:N manner becomes possible.

[0129] Meanwhile, the first virtual machine (not shown) within the processor (175) can control the shared memory (508) based on the hypervisor (505) to be set for the same data transmission to the second virtual machine (not shown) and the third virtual machine (not shown).

[0130] Meanwhile, the signal processing device (170) can process various signals such as audio signals, video signals, and data signals. To this end, the signal processing device (170) can be implemented in the form of a system on chip (SOC).

[0131] Meanwhile, the signal processing device (170) in the display device (100) of FIG. 4 may be the same as the signal processing device (170, 170a1, 170a2) of the vehicle display device of FIG. 5a or lower.

[0132] FIGS. 5A to 5D are drawings showing various examples of vehicle display devices.

[0133] FIG. 5A illustrates an example of a vehicle display device according to an embodiment of the present disclosure.

[0134] Referring to the drawings, a vehicle display device (800a) according to an embodiment of the present disclosure includes a signal processing device (170a1, 170a2) and a plurality of area signal processing devices (170Z1 to 170Z4).

[0135] Meanwhile, in the drawing, two signal processing devices (170a1, 170a2) are exemplified, but this is for backup purposes, etc., and one is also possible.

[0136] Meanwhile, the signal processing device (170a1, 170a2) may also be named an HPC (High Performance Computing) signal processing device.

[0137] Multiple area signal processing devices (170Z1 to 170Z4) are arranged in each area (Z1 to Z4) and can transmit sensor data to signal processing devices (170a1, 170a2).

[0138] The signal processing device (170a1, 170a2) receives data via a wire from multiple area signal processing devices (170Z1 to 170Z4) or a communication device (120).

[0139] In the drawing, data is exchanged based on wired communication between a signal processing device (170a1, 170a2) and multiple area signal processing devices (170Z1 to 170Z4), and the signal processing device (170a1, 170a2) and the server (400) exchange data based on wireless communication. However, data may be exchanged based on wireless communication between a communication device (120) and a server (400), and the signal processing device (170a1, 170a2) and the communication device (120) may exchange data based on wired communication.

[0140] Meanwhile, data received by the signal processing device (170a1, 170a2) may include camera data or sensor data.

[0141] For example, sensor data within a vehicle may include at least one of vehicle wheel speed data, vehicle direction data, vehicle location data (GPS data), vehicle angle data, vehicle speed data, vehicle acceleration data, vehicle inclination data, vehicle forward / backward data, battery data, fuel data, tire data, vehicle lamp data, vehicle interior temperature data, vehicle interior humidity data, vehicle exterior radar data, and vehicle exterior lidar data.

[0142] Meanwhile, camera data may include vehicle exterior camera data and vehicle interior camera data.

[0143] Meanwhile, the signal processing device (170a1, 170a2) can execute multiple virtual machines (820, 830, 840) based on safety standards.

[0144] In the drawing, it is illustrated that a processor (175) within a signal processing device (170a) executes a hypervisor (505) and, on the hypervisor (505), executes first to third virtual machines (820 to 840) according to an automotive safety integrity level (Automotive SIL; ASIL).

[0145] The first virtual machine (820) may be a virtual machine corresponding to Quality Management (QM), which is the lowest safety level in the Automotive Safety Integrity Level (ASIL) and is a non-enforceable grade.

[0146] The first virtual machine (820) can execute an operating system (822), a container runtime (824) on the operating system (822), and containers (827, 829) on the container runtime (824).

[0147] The second virtual machine (820) may be a virtual machine corresponding to ASIL A or ASIL B, where the sum of severity, exposure, and controllability is 7 or 8 in the automotive safety integrity level (ASIL).

[0148] The second virtual machine (820) can execute an operating system (832), a container runtime (834) on the operating system (832), and containers (837, 839) on the container runtime (834).

[0149] The third virtual machine (840) may be a virtual machine corresponding to ASIL C or ASIL D, in which the sum of severity, exposure, and controllability is 9 or 10 in the automotive safety integrity level (ASIL).

[0150] Meanwhile, ASIL D can correspond to the grade that requires the highest safety level.

[0151] The third virtual machine (840) can run a safety operating system (842) and an application (845) on the operating system (842).

[0152] Meanwhile, the third virtual machine (840) may also execute a safety operating system (842), a container runtime (844) on the safety operating system (842), and a container (847) on the container runtime (844).

[0153] Meanwhile, unlike the drawing, the third virtual machine (840) can also be executed through a separate core rather than the processor (175). This will be described later with reference to FIG. 5b.

[0154] FIG. 5b illustrates another example of a vehicle display device according to an embodiment of the present disclosure.

[0155] Referring to the drawings, a vehicle display device (800b) according to an embodiment of the present disclosure includes a signal processing device (170a1, 170a2) and a plurality of area signal processing devices (170Z1 to 170Z4).

[0156] The vehicle display device (800b) of FIG. 5b is similar to the vehicle display device (800a) of FIG. 5a, but the signal processing device (170a1) has some differences from the signal processing device (170a1) of FIG. 5a.

[0157] To describe the difference, the signal processing device (170a1) may include a processor (175) and a second processor (177).

[0158] The processor (175) within the signal processing unit (170a1) executes a hypervisor (505), and executes first and second virtual machines (820 to 830) on the hypervisor (505) according to the automotive safety integrity level (Automotive SIL; ASIL).

[0159] The first virtual machine (820) can execute an operating system (822), a container runtime (824) on the operating system (822), and containers (827, 829) on the container runtime (824).

[0160] The second virtual machine (820) can execute an operating system (832), a container runtime (834) on the operating system (832), and containers (837, 839) on the container runtime (834).

[0161] Meanwhile, the second processor (177) within the signal processing device (170a1) can execute a third virtual machine (840).

[0162] The third virtual machine (840) can execute a safety operating system (842), an auto-execution (845) on the operating system (842), and an application (845) on the auto-execution (845). That is, unlike FIG. 5A, an auto-execution (846) on the operating system (842) can be executed.

[0163] Meanwhile, the third virtual machine (840) may, similarly to FIG. 5a, execute a safety operating system (842), a container runtime (844) on the safety operating system (842), and a container (847) on the container runtime (844).

[0164] Meanwhile, the third virtual machine (840) requiring a high level of security is preferably executed on a second processor (177), which is a different core or different processor, unlike the first and second virtual machines (820 to 830).

[0165] Meanwhile, in the signal processing devices (170a1, 170a2) of FIGS. 5a and 5b, when the first signal processing device (170a) malfunctions, the second signal processing device (170a2), which is a backup device, can operate.

[0166] Alternatively, it is also possible for the signal processing devices (170a1, 170a2) to operate simultaneously, with the first signal processing device (170a) operating as the main device and the second signal processing device (170a2) operating as the sub device. This will be described with reference to FIGS. 5c and 5d.

[0167] FIG. 5c illustrates another example of a vehicle display device according to an embodiment of the present disclosure.

[0168] Referring to the drawings, a vehicle display device (800c) according to an embodiment of the present disclosure includes a signal processing device (170a1, 170a2) and a plurality of area signal processing devices (170Z1 to 170Z4).

[0169] Meanwhile, in the drawing, two signal processing devices (170a1, 170a2) are exemplified, but this is for backup purposes, etc., and one is also possible.

[0170] Meanwhile, the signal processing device (170a1, 170a2) may also be named an HPC (High Performance Computing) signal processing device.

[0171] Multiple area signal processing devices (170Z1 to 170Z4) are arranged in each area (Z1 to Z4) and can transmit sensor data to signal processing devices (170a1, 170a2).

[0172] The signal processing device (170a1, 170a2) receives data via a wire from multiple area signal processing devices (170Z1 to 170Z4) or a communication device (120).

[0173] In the drawing, data is exchanged based on wired communication between a signal processing device (170a1, 170a2) and multiple area signal processing devices (170Z1 to 170Z4), and the signal processing device (170a1, 170a2) and the server (400) exchange data based on wireless communication. However, data may be exchanged based on wireless communication between a communication device (120) and a server (400), and the signal processing device (170a1, 170a2) and the communication device (120) may exchange data based on wired communication.

[0174] Meanwhile, data received by the signal processing device (170a1, 170a2) may include camera data or sensor data.

[0175] Meanwhile, among the signal processing devices (170a1, 170a2), the processor (175) in the first signal processing device (170a1) executes a hypervisor (505) and can execute a safety virtualization machine (860) and a non-safety virtualization machine (870) on the hypervisor (505), respectively.

[0176] Meanwhile, among the signal processing devices (170a1, 170a2), the processor (175b) in the second signal processing device (170a2) executes the hypervisor (505b) and can execute only the safety virtualization machine (880) on the hypervisor (505).

[0177] In this way, since the processing for safety is separated between the first signal processing device (170a1) and the second signal processing device (170a2), it is possible to improve stability and processing speed.

[0178] Meanwhile, high-speed network communication can be performed between the first signal processing device (170a1) and the second signal processing device (170a2).

[0179] FIG. 5d illustrates another example of a vehicle display device according to an embodiment of the present disclosure.

[0180] Referring to the drawings, a vehicle display device (800d) according to an embodiment of the present disclosure includes a signal processing device (170a1, 170a2) and a plurality of area signal processing devices (170Z1 to 170Z4).

[0181] The vehicle display device (800d) of FIG. 5d is similar to the vehicle display device (800c) of FIG. 5c, but the second signal processing device (170a2) has some differences from the second signal processing device (170a2) of FIG. 5c.

[0182] The processor (175b) in the second signal processing device (170a2) of FIG. 5d executes a hypervisor (505b) and can execute a safety virtualization machine (880) and a non-safety virtualization machine (890) on the hypervisor (505).

[0183] That is, unlike FIG. 5c, the difference is that the processor (175b) within the second signal processing device (170a2) further executes a non-safety virtualization machine (890).

[0184] In this way, since the processing for safety and non-safety is separated into the first signal processing device (170a1) and the second signal processing device (170a2), it is possible to improve stability and processing speed.

[0185] FIG. 6 is an example of a block diagram of a vehicle display device according to an embodiment of the present disclosure.

[0186] Referring to the drawings, a vehicle display device (900) according to an embodiment of the present disclosure includes a signal processing device (170) and at least one display.

[0187] In the drawing, at least one display is illustrated, a cluster display (180a) and an AVN display (180b).

[0188] Meanwhile, the vehicle display device (900) may further include a plurality of area signal processing devices (170Z1 to 170Z4).

[0189] The signal processing device (170) at this time is a high-performance centralized signal processing and control device having multiple CPUs (175), GPUs (178), NPUs (179), etc., and may be called an HPC (High Performance Computing) signal processing device or a central signal processing device.

[0190] A plurality of area signal processing devices (170Z1 to 170Z4) and a signal processing device (170) are connected by wired cables (CB1 to CB4).

[0191] Meanwhile, multiple area signal processing devices (170Z1 to 170Z4) can be connected to each other with wired cables (CBa to CBd).

[0192] The wired cable (CBa~CBd) at this time may include a CAN communication cable, an Ethernet communication cable, or a PCI Express cable.

[0193] Meanwhile, a signal processing device (170) according to an embodiment of the present disclosure may be equipped with at least one processor (175, 178, 177) and a large-capacity storage device (925).

[0194] For example, a signal processing device (170) according to an embodiment of the present disclosure may include a central processor (175, 177), a graphics processor (178), and a neural processor (179).

[0195] Meanwhile, sensor data may be transmitted from at least one of the multiple area signal processing devices (170Z1 to 170Z4) to the signal processing device (170). In particular, the sensor data may be stored in a storage device (925) within the signal processing device (170).

[0196] The sensor data at this time may include at least one of camera data, lidar data, radar data, vehicle direction data, vehicle location data (GPS data), vehicle angle data, vehicle speed data, vehicle acceleration data, vehicle inclination data, vehicle forward / backward data, battery data, fuel data, tire data, vehicle lamp data, vehicle interior temperature data, and vehicle interior humidity data.

[0197] In the drawing, it is exemplified that camera data from a camera (195a) and lidar data from a lidar sensor (196) are input to a first area signal processing device (170Z1), and the camera data and lidar data are transmitted to a signal processing device (170) via a second area signal processing device (170Z2), a third area signal processing device (170Z3), etc.

[0198] Meanwhile, since the data read speed or write speed to the storage device (925) is faster than the network speed when sensor data is transmitted from at least one of the plurality of area signal processing devices (170Z1 to 170Z4) to the signal processing device (170), it is preferable that multi-path routing be performed so that a network bottleneck does not occur.

[0199] To this end, the signal processing device (170) according to the embodiment of the present disclosure can perform multi-path routing based on a Software Defined Network (SDN). Accordingly, a stable network environment can be secured when reading or writing data from the storage device (925). Furthermore, since data can be transmitted to the storage device (925) using multiple paths, the network configuration can be dynamically changed to transmit data.

[0200] Data communication between a plurality of area signal processing devices (170Z1 to 170Z4) and a signal processing device (170) in a vehicle display device (900) according to an embodiment of the present disclosure is preferably Peripheral Component Interconnect Express communication for high-bandwidth, low-latency communication.

[0201] FIGS. 7A and 7B are drawings for reference in the description of a signal processing device related to the present disclosure.

[0202] Figure 7a illustrates an application based on camera data, etc., running on a signal processing device.

[0203] Referring to the drawing, a signal processing device (170x) related to the present disclosure can execute a Driver Monitoring Systems (DMS) application (785) based on camera data from an in-vehicle camera (195i), sensor data from a pressure sensor (SNp), and sensor data from a gas sensor (SNc), and can control a warning sound to be output to an audio output unit (185) based on the result data.

[0204] Figure 7b is a drawing referenced in the operation description of Figure 7a.

[0205] Referring to the drawing, a signal processing device (170x) related to the present disclosure includes a processor (175x), and the processor (175x) can execute a hypervisor (505).

[0206] Meanwhile, a processor (175x) related to the present disclosure executes a plurality of virtual machines (520x, 530x, 540x) on a hypervisor (505), and a second virtual machine (530x) among the plurality of virtual machines (520x, 530x, 540x) executes a driver monitoring system (DMS) application (785) based on camera data from an internal vehicle camera (195i), sensor data from a pressure sensor (SNp), and sensor data from a gas sensor (SNc), and executes a lane keeping assist system (LKAS) application (787) based on camera data outside the vehicle.

[0207] Meanwhile, among the multiple virtual machines (520x, 530x, 540x), a third virtual machine (540x) can execute a forward collision warning (FCW) application (789) based on vehicle external camera data.

[0208] Meanwhile, as shown in FIGS. 7a and 7b, when a driver monitoring system (DMS) application (785) and a lane keeping assist system (787) are executed within a second virtual machine (530x), there is a problem that the workload of the second virtual machine (530x) is significant.

[0209] In particular, in order to execute the driver monitoring system (DMS) application (785), camera data from an in-vehicle camera (195i), sensor data from a pressure sensor (SNp), and sensor data from a gas sensor (SNc) must be received and processed, so there is a problem that the workload of the second virtual machine (530x) is considerable.

[0210] Meanwhile, the third virtual machine (540x) runs on a separate virtual machine from the lane keeping assist system (787), which is based on vehicle external camera data when executing the forward collision warning (FCW) application (789), so there is a problem that the workload is performed inefficiently.

[0211] Accordingly, in this disclosure, a method for sharing intermediate result data of an application, etc., when executing a similar application is proposed.

[0212] To this end, the signal processing device (170) according to the embodiment of the present disclosure divides the application into a plurality of micro services, and executes other micro services based on the results of the micro services, etc., thereby efficiently distributing the workload.

[0213] For example, a signal processing device (170) according to an embodiment of the present disclosure may control a plurality of micro services, wherein a first micro service is performed in a first virtual machine and a second micro service is performed in a second virtual machine, and the result of the first micro service is shared using a shared memory (508) or the like, so that the second micro service is executed based on the result data of the first virtual machine. Accordingly, data processing can be performed efficiently.

[0214] Meanwhile, the signal processing device (170) according to the embodiment of the present disclosure can execute multiple virtual machines by distinguishing them according to the safety level.

[0215] Meanwhile, the signal processing device (170) according to the embodiment of the present disclosure can execute applications or microservices by distinguishing them according to the safety level.

[0216] Accordingly, data processing can be performed reliably according to the Automotive Safety Integrity Level (ASIL) in relation to advanced driver assistance systems (ADAS) or autonomous driving.

[0217] FIGS. 8A to 8E are diagrams illustrating various examples of execution of microservices according to embodiments of the present disclosure.

[0218] Figure 8a illustrates that multiple microservices corresponding to ASIL B are executed based on camera data from an internal camera (195i).

[0219] Referring to the drawing, a signal processing device (170) according to an embodiment of the present disclosure can execute a driver monitoring system (DMS) application (905) corresponding to ASIL B.

[0220] For example, a signal processing device (170) according to an embodiment of the present disclosure can execute a driver monitoring system (DMS) application (905) by separating it into multiple microservices.

[0221] The drawing illustrates multiple microservices for a driver monitoring system (DMS) application (905), including a face detection microservice (910b), an eye movement microservice (915b), an eye tracking microservice (920b), and an alert microservice (930b).

[0222] That is, the signal processing device (170) according to the embodiment of the present disclosure can execute a face detection microservice (910b), an eye movement microservice (915b), a gaze tracking microservice (920b), and a warning microservice (930b) as a plurality of microservices for a driver monitoring system (DMS) application (905) corresponding to ASIL B.

[0223] Meanwhile, the face detection microservice (910b) is executed based on camera data from the internal camera (195i), and the result data of the face detection microservice (910b) is transmitted to the eye movement microservice (915b).

[0224] Meanwhile, the eye movement microservice (915b) is executed based on the result data of the face detection microservice (910b), and the result data of the eye movement microservice (915b) is transmitted to the gaze tracking microservice (920b).

[0225] Meanwhile, the gaze tracking microservice (920b) is executed based on the result data of the eye movement microservice (915b), and the result data of the gaze tracking microservice (920b) is transmitted to the alert microservice (930b).

[0226] Meanwhile, the warning microservice (930b) is executed based on the result data of the gaze tracking microservice (920b), and the result data is input to the audio output unit (185), so that a warning sound can be output from the audio output unit (185).

[0227] Meanwhile, the signal processing device (170) according to the embodiment of the present disclosure can further execute a second face detection microservice (910c) and a head movement microservice (915c) for a driver monitoring system (DMS) application (905) corresponding to ASIL B.

[0228] The second face detection microservice (910c) is executed based on camera data from the internal camera (195i), and the result data of the second face detection microservice (910c) is transmitted to the head movement microservice (915c).

[0229] Meanwhile, the head movement microservice (915c) is executed based on the result data of the second face detection microservice (910c), and the result data of the head movement microservice (915c) is transmitted to the gaze tracking microservice (920b).

[0230] Meanwhile, the gaze tracking microservice (920b) is executed based on the result data of the head movement microservice (915c) and the result data of the eye movement microservice (915b), and the result data of the gaze tracking microservice (920b) can be transmitted to the warning microservice (930b).

[0231] Meanwhile, the signal processing device (170) according to the embodiment of the present disclosure can further execute a plurality of micro-services that are not related to ASIL B, for example, corresponding to QM (Quality Management).

[0232] In the drawing, a third face detection microservice (910a), a face recognition microservice (915a), and a personal microservice (920a) can each be executed as multiple microservices, not related to ASIL B.

[0233] Figure 8b illustrates an example of execution of multiple microservices corresponding to ASIL B and microservices corresponding to QM based on camera data from an internal camera (195i).

[0234] Referring to the drawing, the signal processing device (170) according to the embodiment of the present disclosure can execute a driver monitoring system (DMS) application (905) corresponding to ASIL B, similar to FIG. 8A.

[0235] For example, a signal processing device (170) according to an embodiment of the present disclosure can execute a face detection microservice (910b), an eye movement microservice (915b), a gaze tracking microservice (920b), and a warning microservice (930b) as a plurality of microservices for a driver monitoring system (DMS) application (905) corresponding to ASIL B.

[0236] Meanwhile, the signal processing device (170) according to the embodiment of the present disclosure can further execute a second face detection microservice (910c) and a head movement microservice (915c) for a driver monitoring system (DMS) application (905) corresponding to ASIL B.

[0237] Meanwhile, the signal processing device (170) according to the embodiment of the present disclosure can execute a third face detection microservice (910a), a face recognition microservice (915a), and a personal microservice (920a) as a plurality of microservices, which are not related to ASIL B.

[0238] Meanwhile, the signal processing device (170) according to the embodiment of the present disclosure can execute an augmented reality microservice (930c), which is an example of a graphic provision microservice, as a microservice corresponding to QM.

[0239] At this time, the signal processing device (170) according to the embodiment of the present disclosure can transmit the result data of the eye tracking microservice (920b) among the microservices in the application (905) corresponding to ASIL B to the augmented reality microservice (930c) corresponding to QM, which has a lower safety level.

[0240] Accordingly, the augmented reality microservice (930c) corresponding to QM is executed based on the result data of the gaze tracking microservice (920b), and the result data of the augmented reality microservice (930c) can be transmitted to the display (180) and displayed.

[0241] Figure 8c illustrates the execution of multiple microservices corresponding to QM.

[0242] Referring to the drawing, a signal processing device (170) according to an embodiment of the present disclosure can execute a passenger monitoring application (940) corresponding to QM.

[0243] For example, a signal processing device (170) according to an embodiment of the present disclosure can execute a passenger monitoring application (940) by separating it into multiple micro-services.

[0244] The drawing illustrates multiple microservices for a passenger monitoring application (940), including a passenger press detection microservice (950b), a passenger movement microservice (955b), a passenger detection microservice (960b), and a graphics provision microservice (965b).

[0245] That is, the signal processing device (170) according to the embodiment of the present disclosure can execute a plurality of microservices, namely, a passenger seating microservice (950b), a passenger movement microservice (955b), a passenger detection microservice (960b), and a graphics provision microservice (965b), for a passenger monitoring application (940) corresponding to QM.

[0246] Meanwhile, the passenger seating microservice (950b) is executed based on sensor data from the pressure sensor (SNp), and the result data of the passenger seating microservice (950b) is transmitted to the passenger movement microservice (955b).

[0247] Meanwhile, the passenger movement microservice (955b) is executed based on the result data of the passenger seating microservice (950b), and the result data of the passenger movement microservice (955b) is transmitted to the passenger detection microservice (960b).

[0248] Meanwhile, the passenger detection microservice (960b) is executed based on the result data of the passenger movement microservice (955b), and the result data of the passenger detection microservice (960b) is transmitted to the graphics provision microservice (965b).

[0249] Meanwhile, the graphic provision microservice (965b) is executed based on the result data of the passenger detection microservice (960b), and the result data of the graphic provision microservice (965b) can be transmitted to the display (180) and displayed.

[0250] Meanwhile, the signal processing device (170) according to the embodiment of the present disclosure can further execute a gas detection (CO2 detection) microservice (950c) for a passenger monitoring application (940) corresponding to QM.

[0251] The gas detection (CO2 detection) microservice (950c) is executed based on sensor data from the gas sensor (SNc), and the result data of the gas detection (CO2 detection) microservice (950c) is transmitted to the passenger movement microservice (955b).

[0252] Meanwhile, the passenger movement microservice (955b) is executed based on the result data of the gas detection (CO2 detection) microservice (950c), and the result data of the passenger movement microservice (955b) is transmitted to the passenger detection microservice (960b).

[0253] Meanwhile, the signal processing device (170) according to the embodiment of the present disclosure can further execute a plurality of micro-services unrelated to the passenger monitoring application (940).

[0254] In the drawing, the signal processing unit (170) is illustrated as executing a plurality of microservices, each of which is unrelated to the passenger monitoring application (940), including a face detection microservice (910a), a face recognition microservice (915a), and a personal microservice (920a) based on camera data from an in-vehicle camera (195i).

[0255] Figure 8d illustrates the execution of multiple microservices corresponding to ASIL B and microservices corresponding to QM.

[0256] Referring to the drawing, the signal processing device (170) according to the embodiment of the present disclosure can execute a passenger monitoring application (940) corresponding to QM, similar to FIG. 8c.

[0257] For example, a signal processing device (170) according to an embodiment of the present disclosure can execute a plurality of microservices, for a passenger monitoring application (940) corresponding to QM, including a passenger seating microservice (950b), a passenger movement microservice (955b), a passenger detection microservice (960b), and a graphics provision microservice (965b).

[0258] Meanwhile, the signal processing device (170) according to the embodiment of the present disclosure can further execute a gas detection (CO2 detection) microservice (950c) for a passenger monitoring application (940) corresponding to QM.

[0259] Meanwhile, the signal processing device (170) according to the embodiment of the present disclosure can execute a plurality of micro-services, including a face detection micro-service (910a), a face recognition micro-service (915a), and a personal micro-service (920a), which are not related to the passenger monitoring application (940).

[0260] Meanwhile, the signal processing device (170) according to the embodiment of the present disclosure can further execute a passenger monitoring application (945) corresponding to ASIL B.

[0261] In the drawing, multiple microservices for an occupant monitoring application (945) corresponding to ASIL B are illustrated, including an occupant seating microservice (950d), an occupant movement microservice (955d), an occupant detection microservice (960d), and an alert microservice (965d).

[0262] Meanwhile, the passenger seating microservice (950d) is executed based on sensor data from the pressure sensor (SNp), and the result data of the passenger seating microservice (950d) is transmitted to the passenger movement microservice (955d).

[0263] Meanwhile, the passenger movement microservice (955d) is executed based on the result data of the passenger seating microservice (950d), and the result data of the passenger movement microservice (955d) is transmitted to the passenger detection microservice (960d).

[0264] Meanwhile, the passenger detection microservice (960d) is executed based on the result data of the passenger movement microservice (955d), and the result data of the passenger detection microservice (960d) is transmitted to the warning microservice (965d).

[0265] Meanwhile, the warning microservice (965d) is executed based on the result data of the passenger detection microservice (960d), and the result data of the warning microservice (965d) can be transmitted to and output by the audio output unit (185).

[0266] Meanwhile, the signal processing device (170) according to the embodiment of the present disclosure can further execute a gas detection (CO2 detection) microservice (950e) for a passenger monitoring application (945) corresponding to ASIL B.

[0267] Meanwhile, the gas detection (CO2 detection) microservice (950e) is executed based on sensor data from the gas sensor (SNc), and the result data of the gas detection (CO2 detection) microservice (950e) can be transmitted to the passenger movement microservice (955d).

[0268] Meanwhile, the signal processing device (170) according to the embodiment of the present disclosure may not transmit the result data of the passenger detection microservice (960b) among the microservices in the passenger monitoring application (940) corresponding to QM to the warning microservice (965d) in the passenger monitoring application (945) corresponding to ASIL B.

[0269] That is, since the safety level of the passenger detection microservice (960b) within the passenger monitoring application (940) corresponding to QM is lower than the safety level of the warning microservice (965d) within the passenger monitoring application (945) corresponding to ASIL B, the signal processing device (170) according to the embodiment of the present disclosure cannot transmit the result data of the passenger detection microservice (960b) to the warning microservice (965d) within the passenger monitoring application (945) corresponding to ASIL B. Accordingly, each safety level can be maintained.

[0270] Figure 8e illustrates another example of execution of multiple microservices corresponding to ASIL B and microservices corresponding to QM based on camera data from an internal camera (195i).

[0271] Referring to the drawing, the signal processing device (170) according to the embodiment of the present disclosure can execute a driver monitoring system (DMS) application (985) corresponding to ASIL B, similarly to FIG. 8b.

[0272] The Driver Monitoring System (DMS) application (985) is similar to the Driver Monitoring System (DMS) application (905) of FIG. 8b, but differs in that the second face detection microservice (910c) is not performed.

[0273] For example, a signal processing device (170) according to an embodiment of the present disclosure can execute a plurality of microservices, including a face detection microservice (910b), an eye movement microservice (915b), a head movement microservice (915c), a gaze tracking microservice (920b), and a warning microservice (930b), for a driver monitoring system (DMS) application (985) corresponding to ASIL B.

[0274] Meanwhile, the head movement microservice (915c) is executed based on the result data of the face detection microservice (910b), and the result data of the head movement microservice (915c) is transmitted to the gaze tracking microservice (920b).

[0275] Meanwhile, the signal processing device (170) according to the embodiment of the present disclosure can execute a third face detection microservice (910a), a face recognition microservice (915a), and a personal microservice (920a) as a plurality of microservices, which are not related to ASIL B.

[0276] Meanwhile, the signal processing device (170) according to the embodiment of the present disclosure can execute an augmented reality microservice (930c) as a microservice corresponding to QM.

[0277] At this time, the signal processing device (170) according to the embodiment of the present disclosure can transmit the result data of the eye tracking microservice (920b) among the microservices in the application (985) corresponding to ASIL B to the augmented reality microservice (930c) corresponding to QM, which has a lower safety level.

[0278] Accordingly, the augmented reality microservice (930c) corresponding to QM is executed based on the result data of the gaze tracking microservice (920b), and the result data of the augmented reality microservice (930c) can be transmitted to the display (180) and displayed.

[0279] FIG. 9 is an example of an internal block diagram of a signal processing device according to an embodiment of the present disclosure.

[0280] Referring to the drawing, a signal processing device (170) in a system (1000) according to one embodiment of the present disclosure includes a processor (175) that executes a hypervisor (505).

[0281] Meanwhile, the processor (175) can correspond to the central processor (CPU) of FIG. 6.

[0282] Meanwhile, the processor (175) may have multiple processor cores.

[0283] The drawing illustrates multiple processor cores each operating based on a safety level of ASIL B, but various variations are possible.

[0284] For example, some of the multiple processor cores may operate based on a safety level of ASIL B, while others may operate based on a safety level of QM.

[0285] Meanwhile, a signal processing device (170) according to one embodiment of the present disclosure may further include a second processor (177) including an M core or MCU (micom nuit) for executing applications of the highest safety level, ASIL D.

[0286] Meanwhile, the processor (175) executes multiple virtual machines (810 to 830) on the hypervisor (505).

[0287] Meanwhile, among the multiple virtualization machines (810 to 830), the first virtualization machine (830) executes multiple microservices (910b, 915b, 920b, 930b) corresponding to the first safety level such as ASIL B.

[0288] Meanwhile, among the plurality of virtual machines (810 to 830), the first virtual machine (830) transmits the result data of the first microservice (920b) among the plurality of microservices to the second virtual machine (820) among the plurality of virtual machines (810 to 830) corresponding to the second security level lower than the first security level or to the virtual machine within the second signal processing device (170Z). Accordingly, mixed operation of microservices can be performed. Furthermore, data processing can be performed efficiently using microservices.

[0289] Meanwhile, the first virtual machine (830) can execute multiple microservices separately to execute the first application corresponding to the first security level.

[0290] That is, among the plurality of virtual machines (810 to 830), the first virtual machine (830) can separately execute a face detection microservice (910b), an eye movement microservice (915b), a gaze tracking microservice (920b), and a warning microservice (930b) as a plurality of microservices for a driver monitoring system (DMS) application (905) corresponding to a first safety level such as ASIL B.

[0291] Meanwhile, the second virtual machine (820) executes a second application corresponding to the second security level, and the second application can be executed based on the result data of the first microservice (920b).

[0292] For example, among multiple virtualization machines (810 to 830), the second virtualization machine (820) can execute an augmented reality microservice (930c) as a microservice corresponding to the second safety level, QM, as shown in FIG. 8b.

[0293] Meanwhile, among the multiple virtualization machines (810 to 830), the second virtualization machine (820) can further execute a facial recognition microservice (915a) as a microservice corresponding to the second security level, QM, as shown in FIG. 8b.

[0294] Meanwhile, among the multiple virtualization machines (810 to 830), the second virtualization machine (820) can execute an augmented reality microservice (930c) as a microservice corresponding to QM.

[0295] Meanwhile, the first virtual machine (830) can transmit the result data of the eye tracking microservice (920b) among the microservices in the application (905) corresponding to ASIL B to the augmented reality microservice (930c) corresponding to QM, which has a lower safety level.

[0296] That is, the first virtual machine (830) can transmit the result data of the eye tracking microservice (920b) among the microservices within the application (905) corresponding to ASIL B to the augmented reality microservice (930c) within the second virtual machine (820), which has a lower safety level, as shown in FIG. 8b.

[0297] Specifically, the first virtual machine (830) may execute a face detection microservice (910b), an eye movement microservice (915b), a gaze tracking microservice (920b), and a warning microservice (930b) based on the received camera data, and transmit the result data of the gaze tracking microservice (920b) to the second virtual machine (820) corresponding to the second safety level.

[0298] Meanwhile, the second virtual machine (820) can be controlled to execute the augmented reality microservice (930c) based on the result data of the eye tracking microservice (920b), as shown in FIG. 8b, and display the result data of the augmented reality microservice (930c) on the display (180).

[0299] In this way, since there is no need to run a separate eye tracking microservice (920b) within the second virtual machine (820), mixed operation of microservices can be performed.

[0300] Meanwhile, the first virtual machine (830) can transmit the result data of the first microservice (920b) to at least one virtual machine corresponding to a second safety level lower than the first safety level using the shared memory (508).

[0301] Meanwhile, the first virtual machine (830) can transmit the result data of the first micro service (920b) to the second virtual machine (820) corresponding to the second safety level lower than the first safety level using the shared memory (508).

[0302] Meanwhile, the first virtual machine (830) can transmit the result data of the eye tracking microservice (920b) to the augmented reality microservice (930c) within the second virtual machine (820) using the shared memory within the hypervisor (505).

[0303] In this way, when using shared memory (508) to transmit result data, 1:n result data transmission becomes possible.

[0304] Meanwhile, the second virtual machine (820) may not transmit the result data of the microservice being executed or the result data of the application to the first virtual machine (830).

[0305] That is, the second virtual machine (820) may not transmit data to the first virtual machine (830) with a higher security level.

[0306] For example, the result data of the facial recognition microservice (915a) within the second virtual machine (820) is not transmitted to the first virtual machine (830). Accordingly, the security level of each virtual machine can be maintained.

[0307] Meanwhile, among the plurality of virtual machines (810 to 830), the first virtual machine (830) may not transmit the result data of the first micro service (920b) among the plurality of micro services to a virtual machine of a third safety level higher than the first safety level.

[0308] For example, if the third virtual machine (810) among the plurality of virtual machines (810 to 830) has a third safety level higher than the first safety level, such as ASIL D, the first virtual machine (830) may not transmit the result data of the first microservice (920b) among the plurality of microservices to the third virtual machine (810) or the fourth virtual machine (840) at the third safety level higher than the first safety level. Accordingly, the safety level of each virtual machine can be maintained.

[0309] Meanwhile, some cores of the processor (175) may execute a first virtual machine (830), and other cores of the processor (175) may execute a second virtual machine (820).

[0310] In the drawing, some cores of the processor (175) execute a first virtual machine (830) corresponding to the first safety level, ASIL B, and other cores of the processor (175) execute a second virtual machine (820) corresponding to the second safety level, QM. Accordingly, mixed operation of microservices can be performed.

[0311] Meanwhile, the second processor (177) can execute an application or virtual machine of ASIL D, the highest safety level.

[0312] Meanwhile, the security level of the application or virtual machine running on the second processor (177) may be higher than the first security level. Accordingly, mixed operation of microservices can be performed.

[0313] Meanwhile, the second virtual machine (820) can execute a face recognition microservice (915a) based on the received camera data, as shown in FIG. 8d, and execute a personal microservice (920a), which is an additional microservice, based on the result data of the face recognition microservice.

[0314] Meanwhile, the second virtual machine (820) may execute the passenger seating microservice (950b), the passenger movement microservice (955b), the passenger detection microservice (960b), and the graphics provision microservice (965b) for the passenger monitoring application (940) corresponding to QM, as shown in FIG. 8d, based on the received sensor data or camera data, and may not transmit the service result data of the passenger detection microservice (960b) to the first virtual machine (830).

[0315] Meanwhile, if the service result data of the passenger detection microservice (960b) is not received from the second virtual machine (820), the first virtual machine (830) can execute the passenger seating microservice (950d), the passenger movement microservice (955d), the passenger detection microservice (960d), and the warning microservice (965d) for the passenger monitoring application (945) corresponding to ASIL B, based on the received sensor data or camera data, as shown in FIG. 8d. Accordingly, mixed operation of microservices can be performed.

[0316] Meanwhile, the first virtual machine (830) within the signal processing device (170) according to another embodiment of the present disclosure executes the first application and transmits the result data or intermediate result data of the first application to the second virtual machine (820) or the virtual machine within the second signal processing device (170Z) corresponding to the second security level. Accordingly, mixed operation of microservices can be performed. Furthermore, data processing can be performed efficiently using microservices.

[0317] Meanwhile, the first virtual machine (830) may execute a first application including a plurality of microservices and transmit result data of at least some of the plurality of microservices to the second virtual machine (820) or the second signal processing device (170Z).

[0318] FIG. 10 is an example of an internal block diagram of a signal processing device according to another embodiment of the present disclosure.

[0319] Referring to the drawing, a signal processing device (170) in a system (1000b) according to another embodiment of the present disclosure can transmit data to a second signal processing device (170) or receive data from the second signal processing device (170).

[0320] Describing the difference from Fig. 9, the second signal processing device (170z) may be an area signal processing device.

[0321] The second signal processing device (170z) has a processor (175z) that executes a hypervisor (505z).

[0322] Meanwhile, the processor (175z) in the second signal processing device (170z) may have multiple processor cores.

[0323] Meanwhile, the second signal processing device (170z) may further include a separate processor (177z) including an M core or MCU (micom nuit) for executing applications of the highest safety level, ASIL D.

[0324] Meanwhile, the processor (175z) can execute at least one virtualization machine (830z) on the hypervisor (505).

[0325] Meanwhile, a separate processor (177z) can run a virtual machine (840z) corresponding to ASIL D, the highest safety level on the M core.

[0326] Meanwhile, camera data from the internal camera (195i) can be transmitted to the signal processing device (170) or the second signal processing device (170z).

[0327] In the drawing, a virtual machine (830z) within a processor (175z) is illustrated executing a camera data-based video stream application (9993).

[0328] Meanwhile, the processor (175) in the signal processing device (170) can receive sensor data or camera data from the second signal processing device (170Z).

[0329] Meanwhile, the first virtual machine (830) within the signal processing device (170) may execute a plurality of micro-services corresponding to the first safety level based on sensor data or camera data, and, as shown in FIG. 8b, may transmit the result data of the first micro-service (920b) among the plurality of micro-services to the second virtual machine (820) corresponding to the second safety level lower than the first safety level.

[0330] Meanwhile, the first virtual machine (830) within the signal processing device (170) may execute a plurality of microservices corresponding to the first safety level based on sensor data or camera data, and transmit the result data of the first microservice (920b) among the plurality of microservices to the virtual machine within the second signal processing device (170Z) corresponding to the second safety level lower than the first safety level.

[0331] For example, a first virtual machine (830) within a signal processing device (170) can transmit the result data of the gaze tracking microservice (920b) to a virtual machine (830z) within a second signal processing device (170Z) corresponding to the same safety level, ASIL B.

[0332] Accordingly, the virtual machine (830z) within the second signal processing device (170Z) can perform mixed operation of microservices without having to separately execute the eye tracking microservice (920b).

[0333] FIG. 11 is a diagram illustrating a vehicle signal processing system according to an embodiment of the present disclosure.

[0334] Referring to the drawing, a vehicle signal processing system (1100) according to an embodiment of the present disclosure includes a server (400) and a signal processing device (170) within a vehicle (200).

[0335] A signal processing device (170) according to an embodiment of the present disclosure includes a processor (175) that executes a core orchestrator (2000) and a vehicle orchestrator (2010).

[0336] For example, a processor (175) according to an embodiment of the present disclosure can execute firmware (1115) based on standard-based firmware (1110).

[0337] Meanwhile, the processor (175) according to the embodiment of the present disclosure can execute the framework (1120) on the firmware (1115).

[0338] Meanwhile, the framework (1120) may include an operating system (1122), a container runtime (2133), a core orchestrator (2000), a vehicle orchestrator (2010), a hardware abstraction layer (HAL) (1128), and an application (1134).

[0339] For example, the processor (175) may execute an operating system (1122) on firmware (1115), execute a container runtime (2133) on the operating system (1122), execute a core orchestrator (2000) and a vehicle orchestrator (2010) on the container runtime (2133), and execute an application (1134) on the core orchestrator (2000) or the vehicle orchestrator (2010).

[0340] Meanwhile, the server (400) can execute the framework (1180).

[0341] Meanwhile, the framework (1180) may include a standard-based virtual compute environment (1182), an operating system (1184), a container runtime (1185), an orchestrator (1186), a hardware abstraction layer (HAL) (1188), and an application (1194).

[0342] For example, the server (400) may run a container runtime (1185) on an operating system (1184), run an orchestrator (1186) and a hardware abstraction layer (HAL) (1188) on the container runtime (1185), and run an application (1194) on the orchestrator (1186) or the hardware abstraction layer (HAL) (1188).

[0343] Meanwhile, the signal processing device (170) in the vehicle can download and install or update an application received from the outside through a server (400) or the like, in addition to the pre-installed application.

[0344] In this disclosure, a vehicle orchestrator (2010) is executed separately from a core orchestrator (2000) to enable mixed operation of heterogeneous applications or heterogeneous microservices corresponding to heterogeneous applications.

[0345] To this end, the vehicle orchestrator (2010) can execute a resource monitor (2014), a controller (2020), a gateway (2005), and a node agent (2114).

[0346] For example, a resource monitor (2014) within a vehicle orchestrator (2010) may monitor or manage resources such as an operating action (2007), a device (2008), a network or communication (2009), a vehicle application (Vehicle Application Pod) (2011), a vehicle application deployment (Vehicle Application Deployment) (2016), a signal processing unit (SOC) (2017), permissions (2018), and real-time or safety (2019) resources.

[0347] Meanwhile, the controller (2020) within the vehicle orchestrator (2010) can execute a vehicle application controller (2012), a device access controller (2013), a deployment policy (2022), etc.

[0348] Meanwhile, the gateway (2005) within the vehicle orchestrator (2010) can perform functions such as receiving service-oriented architecture (SOA) data (2002) and converting vehicle data (2003).

[0349] Meanwhile, the node agent (2114) within the vehicle orchestrator (2010) can execute a node monitor (2032), a node daemon (2033), etc.

[0350] Fig. 12 is an example of an internal block diagram of the signal processing device of Fig. 11.

[0351] Referring to the drawings, a signal processing device (170) according to one embodiment of the present disclosure includes a processor (CPU) (175) that executes a core orchestrator (2000) and a vehicle orchestrator (2010).

[0352] Meanwhile, a processor (175) according to one embodiment of the present disclosure executes at least one application based on a container runtime (2133), and a vehicle orchestrator (2010) monitors and manages applications, microservices, and device resources based on communication with a core orchestrator (2000). Accordingly, mixed operation of microservices can be performed.

[0353] Meanwhile, a signal processing device (170) according to one embodiment of the present disclosure may further include a GPU (178), an NPU (179), etc.

[0354] Meanwhile, the signal processing device (170) can execute a worker node (2110) including a middleware (2112) and a node agent (2114) through a processor (175), etc., and execute an application (2132, 2134, 2136), a container runtime (2133), and an orchestrator client (2135) on the worker node (2110).

[0355] Meanwhile, the signal processing device (170) executes a master node (2005) including a gateway (2005) through a processor (175), and executes a core orchestrator (2000) including an application programming interface (API) server (2005) on the master node (2005).

[0356] Meanwhile, a processor (175) according to one embodiment of the present disclosure executes a vehicle orchestrator (2010) separately from a core orchestrator (2000).

[0357] Meanwhile, a vehicle orchestrator (2010) according to one embodiment of the present disclosure may execute a resource monitor (2014) for resource monitoring, a vehicle application controller (2012) for vehicle application control or microservice control, a device access controller (2013) for device access control, a controller (2020), etc.

[0358] Meanwhile, the gateway (2005) within the master node (2105) and the node agent (2114) within the worker node (2114) can operate based on the operation or control of the vehicle orchestrator (2010).

[0359] For example, the gateway (2005) receives vehicle data, including sensor data, and transmits the vehicle data to the API server (2107) within the core orchestrator (2000).

[0360] Meanwhile, the vehicle orchestrator (2010) receives vehicle data, etc. from the core orchestrator (2000), and can control at least some of the applications, etc. being executed based on device resources, etc.

[0361] For example, a vehicle orchestrator (2010) can select and control a target application or target microservice, thereby enabling mixed operation of microservices.

[0362] Specifically, the vehicle orchestrator (2010) can select and control a target application or target microservice based on vehicle data, device resources, etc. from the core orchestrator (2000).

[0363] Meanwhile, a processor (175) according to another embodiment of the present disclosure executes at least one application based on a container runtime (2133), and a vehicle orchestrator (2010) controls, based on communication with a core orchestrator (2000), partial connection or partial sharing of heterogeneous applications or heterogeneous microservices. Accordingly, mixed operation of microservices can be performed.

[0364] For example, the vehicle orchestrator (2010) may output a control signal for some connection or some sharing to the orchestrator client (2135) based on communication with the core orchestrator (2000), and the orchestrator client (2135) may transmit the control signal to the container runtime (2133).

[0365] Meanwhile, the container runtime (2133) can control at least one of the multiple applications (2132, 2134, 2136) being executed.

[0366] For example, the container runtime (2133) may control partial connection or partial sharing of heterogeneous applications or heterogeneous microservices based on a control signal received from the orchestrator client (2135). Accordingly, partial connection or partial sharing of heterogeneous applications or heterogeneous microservices may be performed.

[0367] The vehicle orchestrator (2010) can set update timings for running applications, enabling mixed operation of microservices.

[0368] For example, a vehicle orchestrator (2010) can set an update point for an application running while the vehicle is in motion and perform the update at that point through communication with the core orchestrator (2000). This enables mixed operation of microservices.

[0369] Meanwhile, the vehicle orchestrator (2010) or core orchestrator (2000) can control the execution, update, or termination of the second application based on sensor data corresponding to the first application when the first application and the second application are executed.

[0370] At this time, the first application may be an application that runs within the vehicle, and the second application may be an application that is downloaded and run through an external server (400), etc.

[0371] For example, the vehicle orchestrator (2010) may monitor vehicle speed data according to a vehicle speed-related application, and control the vehicle light application to terminate if the vehicle speed data is higher than a predetermined value while the vehicle light application is running.

[0372] Specifically, the vehicle orchestrator (2010) can execute a vehicle light application for controlling vehicle lights while the vehicle is stopped. The vehicle light application at this time can be received and installed through an external server (400), etc.

[0373] Meanwhile, the vehicle orchestrator (2010) monitors vehicle speed data based on vehicle speed-related applications while the vehicle light application is running, and when the vehicle is turned on and moving, it can control the termination of the vehicle light application for reasons such as vehicle safety. Accordingly, it is possible to manage related applications or microservices while the application is running.

[0374] Meanwhile, the vehicle orchestrator (2010) can monitor the microservice corresponding to the first application and control the execution, update, or termination of the second application based on the device's resources.

[0375] For example, a vehicle orchestrator (2010) can monitor a microservice corresponding to a driver monitoring application, which is an example of a first application, and control a lane keeping application, which is an example of a second application, to be updated based on device resources.

[0376] Specifically, the vehicle orchestrator (2010) can control the lane keeping application, an example of a second application, to be updated when device resources are available and the device resource utilization rate is below a predetermined percentage while monitoring a microservice corresponding to a driver monitoring application, an example of a first application. This allows for the management of related applications or microservices while the application is running.

[0377] As another example, a vehicle orchestrator (2010) may monitor a microservice corresponding to a driver monitoring application, which is an example of a first application, and control the execution of a lane keeping application, which is an example of a second application, based on device resources.

[0378] That is, the vehicle orchestrator (2010) can control the execution of the lane keeping application while the driver monitoring application is running. This allows for the management of related applications or microservices while the application is running.

[0379] Meanwhile, the vehicle orchestrator (2010) can control the microservices of the first application or the microservices of the second application based on device resources and communication nodes when the first and second applications are executed. This allows for the management of related applications or microservices during application execution.

[0380] Meanwhile, the vehicle orchestrator (2010) can perform gateway (2005) management and node management of worker nodes (2110) and master nodes (2105).

[0381] To this end, the vehicle orchestrator (2010) can execute a node agent (2114) for managing the gateway (2005), node management of the worker node (2110) and the master node (2105), as shown in FIG. 11.

[0382] Figures 13a to 19d are drawings referenced in the description of Figure 12.

[0383] Figure 13a is a diagram illustrating a core orchestrator service and a vehicle orchestrator service.

[0384] Referring to the drawing, the core orchestrator service can perform real-time services such as fast execution or immediate response, and the vehicle orchestrator service can perform functional safety services of the executing microservices such as fail-safe, monitoring, and authentication.

[0385] Meanwhile, the core orchestrator (2000) may execute or include a core orchestrator service, and the vehicle orchestrator (2010) may execute or include a vehicle orchestrator service.

[0386] Meanwhile, the core orchestrator service may have fixed components, etc., and the vehicle orchestrator service may have variable components, etc.

[0387] Meanwhile, the core orchestrator (2000) may run or include an API server (2017), a scheduler (1332), and a state manager (1334).

[0388] Meanwhile, the vehicle orchestrator (2010) may execute or include an SOA gateway (2005), a network interface (2004), a device interface (2013), a node agent (2114), a deployment policy (1325), a vehicle resource (2014), a controller (2020), a manifest (1310), a global scheduler, etc.

[0389] FIG. 13b illustrates an example of the operation of a processor according to one embodiment of the present disclosure.

[0390] Referring to the drawing, the processor (175) can execute infotainment services (1322), advanced driver assistance systems (ADAS) and autonomous driving (AD) services (1324), connectivity services (1326), power and sea control services (1328), etc.

[0391] For example, the processor (175) can read or reference the manifest (1310) and execute multiple applications (1332, 1334, 1336), etc., on the manifest (1310).

[0392] The manifest (1310) at this time may include elements or information required for execution of a network or communication service (1312), an infrastructure service (1314) such as a hypervisor or virtual machine, a service manifest (1315), a real-time or security requirement service (1319), an access permission service (1318) related to a device or resource, or a configuration service (1317).

[0393] Figure 13c illustrates an example of the operation of the area signal processing unit and the central signal processing unit.

[0394] Referring to the drawing, the area signal processing device (170z) can receive sensor data from a sensor or actuator (SDm) and transmit the sensor data, etc. to the central signal processing device (170).

[0395] Meanwhile, the area signal processing device (170z) can execute control (1341), logic (1343), user interface (1345), etc., for executing applications or services.

[0396] Meanwhile, the central signal processing unit (170) receives sensor data from the area signal processing unit (170z), executes an operating system (1122), a container runtime (2133), a core orchestrator (2000), and a vehicle orchestrator (2010) on the processor (175), and, based on the result of communication between the core orchestrator (2000) and the vehicle orchestrator (2010), executes a plurality of microservices (1351, 1355, 1358), etc., through the container runtime (2133).

[0397] At this time, multiple microservices (1351, 1355, 1358) can execute control (1359), logic (1356), and user interface (1352) corresponding to control (1341), logic (1343), and user interface (1345) within the area signal processing device (170z).

[0398] Meanwhile, among the first to third microservices (1351, 1355, 1358), the third microservice (1358) can control the operation of the processor (175).

[0399] Figure 13d illustrates an example of the operation of the signal processing device.

[0400] Referring to the drawing, the signal processing device (170) can execute a vehicle application on a TSN (Time Sensitive Networking) (1371), SOA (1372), device (1374), or neural network processor (179).

[0401] For example, the signal processing device (170) can execute an application (1367) through a processor (175), memory (140), storage device (925), or port (1369).

[0402] Meanwhile, the signal processing device (170) may also execute an application (1367) based on additional resources (1362).

[0403] Figure 13e is a diagram illustrating various modes of the application.

[0404] Referring to the drawing, an application running on a processor (175) within a signal processing device (175) can perform normal mode (1380), degradation mode (1384), and emergency mode (1288).

[0405] For example, a processor (175) according to one embodiment of the present disclosure may operate in normal mode (238) and then limit some operations of the processor (175) when operating in degradation mode (1384).

[0406] Meanwhile, the processor (175) may operate in degradation mode (1384) and then perform emergency mode (1288), at which time it may be restarted.

[0407] Meanwhile, the processor (175) can be restarted when operating in normal mode (238) and then in emergency mode (1288).

[0408] Meanwhile, the vehicle orchestrator (2010) within the processor (175) can perform safety management and fail-safe mode management for vehicle safety.

[0409] For example, the vehicle orchestrator (2010) can perform fail-safe mode management, including monitoring and managing the primary microservice or the redundant secondary microservice. This can improve the stability of the application or microservice.

[0410] Meanwhile, the vehicle orchestrator (2010) can control the transmission of a recovery command to the first microservice and an activity command to the second microservice when a failure occurs in the first microservice. This can improve the stability of the application or microservice.

[0411] Meanwhile, either the first or second microservice may be a safety monitor service. This allows for mixed operation of microservices.

[0412] Meanwhile, the vehicle orchestrator (2010) can deploy, execute, or manage multiple microservices for the safety monitoring service. This can improve the stability of the application or microservice.

[0413] Meanwhile, the vehicle orchestrator (2010) can control the transition from the first microservice to the second microservice based on the monitoring score or analysis score. This can improve the stability of the application or microservice.

[0414] Figure 14a illustrates an example of a core orchestrator.

[0415] Referring to the drawing, the core orchestrator (2000) may include an application part (1400) and a cluster part (1440).

[0416] The application part (1400) may include a deployment policy (1400), a deployment (1420), an ingress (1432), a service (1433), a persistent volume claim (1435), a configuration map (1437), etc.

[0417] The Deployment Policy (1400) may include a Horizontal Pod Autoscaler (1412), a Stateful Set (1413), a Daemon Set (1414), a Jon (1415), a Cron Job (1417), a Replica Set (1418), etc.

[0418] Deployment (1420) may include volumes (1427), microservices (1422) related to containers, etc.

[0419] Meanwhile, a microservice (1422) may include a container based on environment variables, resource limits, etc.

[0420] Meanwhile, the cluster part (1440) may include a service account (1441), a secret service (1443), a storage class (1445), a limit range (1447), a resource quota (1449), etc.

[0421] Meanwhile, unlike Fig. 14a, some components within the core orchestrator (2000) may also be provided within the vehicle orchestrator (2010).

[0422] For example, Deployment Policy (1400), Persistent Volume Claim (1435), Config Map (1437), Ingress (1432), Service (1433), etc. may be provided within the Vehicle Orchestrator (2010) rather than the Core Orchestrator (2000).

[0423] Figure 14b illustrates an example of a vehicle orchestrator.

[0424] Referring to the drawing, the vehicle orchestrator (2010) may include a deployment policy (1325) and a vehicle application deployment (1460).

[0425] Meanwhile, the Deployment Policy (1325) may include fail-safe mode (1451), monitor set (1453), passive redundant, active redundant, shadow mode, cron job, daemon set, stateful set, replica set, etc.

[0426] Meanwhile, the vehicle application deployment (1460) may include an execution manifest (1464), a machine manifest (1465), a service manifest (1467), etc.

[0427] Figure 15a is a diagram illustrating an example of execution of an application by device resources.

[0428] Referring to the drawing, if the first application (1510) is operating based on data from the first device (1505), and the first application (1510) wants to operate based on data from another device, the second device (1507), a device change or application porting must be performed.

[0429] At this time, the first device (1505) may be a first camera (195m1), and the second device (1507) may be a second camera (195m2).

[0430] At this time, if the devices are different types, it may not be easy to change devices.

[0431] FIG. 15b is a diagram illustrating an example of execution of an application by a device resource according to an embodiment of the present disclosure.

[0432] Referring to the drawing, a first application (1520) may operate based on data from a first device (1505), and a second application (1522) may operate based on data from another device, a second device (1509).

[0433] At this time, the first device (1505) may be a first camera (195m1), and the second device (1509) may be a second camera (195m2).

[0434] Meanwhile, the first device (1505) and the second device (150) may be real devices (1515, 1517).

[0435] Meanwhile, the processor (175) according to the embodiment of the present disclosure may execute a first virtual device (1516) corresponding to a first device (1505) when executing a first application (1520), and may execute a second virtual device (1518) corresponding to a second device (150) when executing a second application (1522).

[0436] Meanwhile, a camera application (1534) within a vehicle application (1530) running on a processor (175) controls the operation of a second device (1509) based on set or allocated resource (1536) information, and can receive camera data (1524), etc., from a second virtual device (1518).

[0437] That is, the vehicle orchestrator (2010) can control the camera application (1534) within the vehicle application (1530) to operate the second device (1509) based on the set or allocated resource (1536) information, and to receive camera data (1524), etc., from the second virtual device (1518).

[0438] Accordingly, it becomes possible to stably receive data from the first device (1505) or the second device (1509).

[0439] Figure 16a illustrates an example of downloading an application.

[0440] Referring to the drawing, the signal processing device (170) can receive an application, such as a driver monitoring application (1605), from an external server, etc.

[0441] Additionally, the processor or memory (1612) within the signal processing device (170) can store the received application and execute the received application.

[0442] Accordingly, the signal processing device (170) can execute the driver monitoring application (1610).

[0443] However, the vehicle middleware (1614) running within the processor (175) has a disadvantage in that it cannot access the running driver monitoring application (1610) due to a conflict, etc.

[0444] Figure 16b illustrates another example of downloading an application.

[0445] Referring to the drawing, the application type (1621) based on the namespace can be divided into a system application (1623) and an external application (1625).

[0446] Meanwhile, the vehicle application controller (2012) within the signal processing device (170) can receive a system application (1623) and an external application (1625) from the outside.

[0447] Meanwhile, the resource manager (1640) within the processor (175) manages the resources of the processor (175) or memory (140).

[0448] In addition, the vehicle orchestrator (2010) within the processor (175) can control the execution of a driver monitoring application (1642), which is an example of a system application (1623) or an external application (1625), based on the resources of the processor (175) or memory (140).

[0449] At this time, unlike in Fig. 16a, access to other applications or microservices is enabled for the driver monitoring application (1642), which is an example of an external application (1625). Accordingly, mixed operation of microservices can be performed.

[0450] Figure 17 illustrates an example of execution of a vehicle light application.

[0451] Referring to the drawing, the gateway (2005) within the processor (175) can receive speed data from a vehicle speed sensor, etc.

[0452] The vehicle state controller (1709) running on the processor (175) can transmit the received speed data to the device access controller (1711), and the device access controller (1711) can transmit the speed data to the running vehicle light application (1713).

[0453] Meanwhile, the vehicle light application (1713) can control the vehicle lamp (1743), which is a real device (1717), through a virtual device (1715).

[0454] Meanwhile, the device manager (174) can control the actual device (1717).

[0455] For example, the vehicle light application (1713) can control the vehicle lamp (1743) to operate the vehicle light based on the operation information (1702) when the vehicle is stopped and the vehicle speed is 0.

[0456] As another example, the vehicle orchestrator (2010) can control the operation of the vehicle lights to be turned off and the vehicle light application (1713) to be terminated based on the operation information (1704) when the vehicle is moving and the vehicle speed data is above a predetermined value.

[0457] Meanwhile, the operation information (1702) when the vehicle speed is 0 and the operation information (1704) when the vehicle speed data is greater than a predetermined value can be stored in the signal processing device (175) as a separate file (1705).

[0458] Meanwhile, the vehicle light application (1713) can be managed as a vehicle application (1719).

[0459] Figure 18a illustrates an example of resources of multiple central signal processing units and area signal processing units.

[0460] Referring to the drawing, (a) of FIG. 18a illustrates that the area signal processing unit (1802), the first central signal processing unit (HPC1) (1804), the first central signal processing unit (HPC1) (1806), and the second central signal processing unit (HPC2) (1808) each execute an IVI virtualization machine, a virtualization machine for autonomous driving, a virtualization machine, and operate with a Linux operating system, a QNX operating system, etc.

[0461] In addition, the processor capacity, memory capacity, etc. of the first central signal processing unit (HPC1) (1804), the first central signal processing unit (HPC1) (1806), and the second central signal processing unit (HPC2) (1808) are exemplified.

[0462] Figure 18a (b) illustrates a graphical representation of the items in Figure 18a (a).

[0463] That is, the IVI virtualization machine item (IVI VM1) (1812) and the virtualization machine item for autonomous driving (1813) are executed in the first central signal processing unit (HPC1) (1817), and the virtualization machine item (VM1) is executed in the second central signal processing unit (HPC2) (1819).

[0464] Figure 18b illustrates the protocol, container, API, etc. of each signal processing device.

[0465] Referring to the drawing, as in (a) of FIG. 18b, the area signal processing device (1802) may be exemplified by a SOME / IP protocol, container-free operation, and an IVI virtualization machine of the first central signal processing device (HPC1) (1804) may be exemplified by a tail sensor, DDS protocol, container operation, and API version 1.2, and the virtualization machine for autonomous driving of the first central signal processing device (HPC1) (1804) may be exemplified by a tail gate motor, DDS protocol, container operation, and API version 1.0.

[0466] According to (a) of Fig. 18b, (b) of Fig. 18b illustrates an SOA selection list.

[0467] (b) of FIG. 18B illustrates a C1 item (1832) corresponding to an IVI virtualization machine of a first central signal processing unit (HPC1) (1804), a C2 item (1834) corresponding to a virtualization machine for autonomous driving of the first central signal processing unit (HPC1) (1804), and an S1 item (1836) corresponding to a virtualization machine of a second central signal processing unit (HPC2) (1808).

[0468] Figure 18c illustrates a graphical user interface according to the settings of Figures 18a and 18b.

[0469] Referring to the drawing, the processor (175) can control the placement of the IVI virtualization machine item (IVI VM1) (1812) and the virtualization machine item (1813) for autonomous driving on the first central signal processing unit (HPC1) item (1817) according to the settings of FIGS. 18a and 18b, and the placement of the C1 item (1832) and the C2 item (1834) on the IVI virtualization machine item (IVI VM1) (1812) and the virtualization machine item (1813) for autonomous driving.

[0470] Meanwhile, the processor (175) may place the IVI virtualization machine item (IVI VM1) (1812) on the second central signal processing unit (HPC2) item (1819) and place the S1 item (1836) on the IVI virtualization machine item (IVI VM1) (1812) according to the settings of FIGS. 18a and 18b.

[0471] Meanwhile, the processor (175) can select a shared memory item (1843) among the Ethernet item (1841) and the shared memory item (1843) as a communication method between the C1 item (1832) and the C2 item (1834).

[0472] Meanwhile, the processor (175) can select the Ethernet item (1846) among the Ethernet item (1846) and the PCI-E item (1848) as a communication method between the C2 item (1834) and the S1 item (1836).

[0473] Figure 19a is a diagram illustrating a setting between multiple signal processing devices.

[0474] Referring to the drawing, the first central signal processing unit (1914) can execute an autonomous driving virtual machine, a vehicle service, Android, a service operating system, etc.

[0475] The second central signal processing unit (1912) can run a Linux operating system for autonomous driving.

[0476] The first central signal processing unit (1914) and the second central signal processing unit (1912) are connected through an Ethernet switch (1913) and can perform Ethernet communication.

[0477] The first central signal processing unit (1914) and multiple area signal processing units (1922, 1925) are connected through switches (1915, 1916, 1918), respectively, and can perform TSN communication.

[0478] Meanwhile, the first area signal processing device (1922) can be connected to the first base device (1921) via the switch (1916).

[0479] Meanwhile, the second area signal processing device (1925) can be connected to the second base device (1926) via the switch (1918).

[0480] Meanwhile, the graph (1930) of FIG. 19a can illustrate the utilization rate of the processor within each signal processing device.

[0481] Meanwhile, a vehicle orchestrator (2010) within a processor (175) according to one embodiment of the present disclosure can monitor the inter-node communication status between a plurality of signal processing devices such as FIG. 19A and monitor resources of the entire system.

[0482] Meanwhile, the vehicle orchestrator (2010) within the processor (175) according to one embodiment of the present disclosure can establish inter-node communication paths or resource allocations according to settings. Accordingly, mixed operation of microservices can be performed.

[0483] Figure 19b illustrates a graphical user interface corresponding to the operation of the signal processing device.

[0484] Referring to the drawing, the processor (175) can control the placement of the IVI virtualization machine item (1951) and the autonomous driving virtualization machine item (1953) on the signal processing device item (1955).

[0485] Meanwhile, the processor (175) can place a cluster item (1942), a home item (1943), and an air conditioning item (1945) on the IVI virtualization machine item (1951), and can control the placement of a traffic signal service item (1947) on the autonomous driving virtualization machine item (1953).

[0486] At this time, for the IVI virtualization machine item (1951), the processor utilization may be allocated at 60% and the memory may be allocated at 124 MB, and for the autonomous driving virtualization machine item (1953), the processor utilization may be allocated at 40% and the memory may be allocated at 528 MB.

[0487] Figure 19c illustrates a graphical user interface related to the traffic signal service item of Figure 19b.

[0488] Referring to the drawing, the processor (175) can control the display of a detailed item (1964) as a traffic signal service item (1962).

[0489] Detailed items (1964) may include processor utilization, memory capacity, authentication method, device, API items, etc.

[0490] Meanwhile, the processor (175) can control the display of a server request graph (1986), a load graph (1968), etc. in relation to a traffic signal service item (1962).

[0491] FIG. 19d illustrates a graphical user interface related to a signal processing system according to an embodiment of the present disclosure.

[0492] Referring to the drawing, the processor (175) can control the display of battery pack information (1971), signal processing device information (1980), and server information (1970) within the vehicle.

[0493] Battery pack information within the vehicle (1971) may include temperature management information (1972), location management information (1973), sensor data management information (1974), etc.

[0494] Signal processing device information (1980) may include battery charge control information (1981), data storage device information (1982), device monitoring information (1983), fault monitoring information (1984), safety information (1985), update agent information (1986), IDS agent information (87), diagnostics information (1988), subscription agent information (1989), state estimation algorithm information (1991), connectivity information (1992), display interface information (1993), or authentication service information (1995).

[0495] Sensor data management information (1974) may include algorithm update information (1975), battery tracking information (1976), battery subscription information (1977), battery tracking information (1978), or authentication service information (1979).

[0496] This graphical user interface allows you to understand and perform mixed operations of applications or microservices in complex environments.

[0497] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person skilled in the art to which the present invention pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. Equipped with a processor that runs the core orchestrator and the vehicle orchestrator, The above processor, Run at least one application based on the container runtime, The above vehicle orchestrator, A signal processing device that performs monitoring and management of applications, microservices, and device resources based on communication with the above core orchestrator.

2. In paragraph 1, The above vehicle orchestrator, A signal processing unit that selects and controls a target application or target microservice.

3. In paragraph 1, The above vehicle orchestrator, A signal processing device that controls some connection or some sharing of heterogeneous applications or heterogeneous microservices based on communication with the above core orchestrator.

4. In paragraph 1, The above vehicle orchestrator, A signal processing device that sets an update point for an application to be executed based on communication with the core orchestrator.

5. In paragraph 1, The above vehicle orchestrator, A signal processing device that sets the update point of an application being executed while the vehicle is driving.

6. In paragraph 1, The above vehicle orchestrator or the above core orchestrator, When running the first and second applications, A signal processing device that controls execution, update, or termination of the second application based on sensor data corresponding to the first application.

7. In paragraph 6, The above vehicle orchestrator, A signal processing device that monitors a microservice corresponding to the first application and controls execution, update, or termination of the second application based on the resources of the device.

8. In paragraph 6, The above vehicle orchestrator, A signal processing device that controls a microservice of the first application or a microservice of the second application based on the device resource and communication node when the first application and the second application are executed.

9. In paragraph 1, The above vehicle orchestrator, A signal processing device that monitors vehicle speed data according to a vehicle speed-related application, and controls the vehicle light application to terminate the vehicle speed data when the vehicle speed data is higher than a predetermined value while the vehicle light application is running.

10. In paragraph 1, The above vehicle orchestrator, A signal processing device that controls the execution of a lane keeping application while a driver monitoring application is running.

11. In paragraph 1, The above vehicle orchestrator, A signal processing device that controls the lane keeping application to be updated based on the resources of the device while the driver monitoring application is running.

12. In paragraph 1, The above vehicle orchestrator, A signal processing device that performs gateway management, node management of worker nodes and master nodes.

13. In paragraph 1, The above vehicle orchestrator, A signal processing device that performs safety management and fail-safe mode management for vehicle safety.

14. In paragraph 1, The above vehicle orchestrator, A signal processing device that performs fail-safe mode management including monitoring and managing a main first microservice or a redundant second microservice.

15. In paragraph 14, The above vehicle orchestrator, A signal processing device that controls, when a failure occurs in the first microservice, to transmit a recovery command to the first microservice and to transmit an activity command to the second microservice.

16. In paragraph 1, The above vehicle orchestrator, A signal processing device that deploys, executes, or manages multiple microservices for a safety monitor service.

17. In paragraph 14, The above vehicle orchestrator, A signal processing device that controls an operation transition from the first microservice to the second microservice based on a monitoring score or analysis score.

18. Equipped with a processor that runs the core orchestrator and the vehicle orchestrator, The above processor, Run at least one application based on the container runtime, The above core orchestrator is, A signal processing device that controls some connection or some sharing of heterogeneous applications or heterogeneous microservices based on communication with the above core orchestrator.

19. In paragraph 18, The above vehicle orchestrator or the above core orchestrator, When running the first and second applications, A signal processing device that controls execution, update, or termination of the second application based on sensor data corresponding to the first application.

20. At least one display; A signal processing device for outputting a video signal to the display; The above signal processing device, A vehicle display device comprising a signal processing device according to any one of claims 1 to 19.

Images