Signal processing device and vehicle display device having the signal processing device

Abstract

An embodiment of the signal processing apparatus and a vehicle display device having the signal processing apparatus according to the present invention includes: at least one neural processor; and a central processing unit that runs an application for driving the vehicle; in the event that the operation of the application is determined to have failed, the central processing unit controls the operation of a second application corresponding to the application to run on another central processing unit or another signal processing device, and the central processing unit changes the baseline rollback assurance time for the failure of the application based on the security level of the application. Thus, the application for driving the vehicle can be run stably.

Description

Technical Field

[0001] The present invention relates to a signal processing apparatus and a display device for a vehicle having the signal processing apparatus, and more specifically, to a signal processing apparatus capable of stably operating an application for driving a vehicle and a display device for a vehicle having the signal processing apparatus. Background Technology

[0002] A vehicle is a device that allows its passengers to move in a desired direction. A car is a representative example.

[0003] On the other hand, for the convenience of users of the vehicle, a vehicle signal processing device is installed inside the vehicle.

[0004] On the other hand, vehicle signal processing devices can run various applications for vehicle operation.

[0005] For example, vehicle signal processing devices can run vehicle driver assistance (ADAS) applications or automatic driving (AD) applications.

[0006] On the other hand, when running applications for vehicle operation, various malfunctions may occur in the vehicle, and solutions for stable vehicle control in the event of malfunctions are being studied. Summary of the Invention

[0007] The technical problem to be solved

[0008] The problem to be solved by the present invention is to provide a signal processing device capable of stably operating applications for vehicle driving, and a vehicle display device having the signal processing device.

[0009] Another problem to be solved by the present invention is to provide a signal processing device capable of stably operating applications for vehicle driving based on a safety level, and a vehicle display device having the signal processing device.

[0010] Technical solutions to the problem

[0011] A signal processing apparatus according to an embodiment of the present invention and a display device for a vehicle having the signal processing apparatus include: at least one neural processor; and a central processor for running an application for driving the vehicle; in the event that the operation of the application is determined to have failed, the central processor controls the operation of a second application corresponding to the application to run on another central processor or other signal processing apparatus; the central processor changes the baseline rollback assurance time for the failure of the application operation based on the security level of the application.

[0012] On the other hand, when the application's security level is the first security level, the CPU can set the baseline rollback assurance time to the first time; when the application's security level is the second security level, which is higher than the first security level, the CPU can set the baseline rollback assurance time to the second time, which is greater than the first time.

[0013] On the other hand, when the security level of the application is the third security level, which is lower than the first security level, the central processing unit can set the baseline backoff assurance time to the third time, which is less than the first time.

[0014] On the other hand, if the backoff guarantee time of the operation is greater than or equal to the baseline backoff guarantee time, the central processing unit can control the execution of a pre-set action.

[0015] On the other hand, if the backoff assurance time of the operation is less than the baseline backoff assurance time, the central processing unit can be controlled to run a second application corresponding to the application on other central processing units or other signal processing devices.

[0016] On the other hand, if the fallback assurance time of the operation is less than the baseline fallback assurance time, the central processing unit can control the execution of the fallback action.

[0017] On the other hand, if the backoff assurance time of the operation is less than the baseline backoff assurance time, the central processing unit can control the execution of any one of the plurality of backoff actions.

[0018] On the other hand, multiple reversal actions can include vehicle takeover requests, blind braking, lane keeping braking, lane changing braking, stopping on the shoulder, or route-based stopping.

[0019] On the other hand, the central processing unit can be controlled such that, when the fallback assurance time of the operation is greater than the baseline fallback assurance time, as the fallback assurance time of the operation decreases, the difference between the estimated baseline fallback assurance time and the baseline fallback assurance time becomes smaller, thereby restricting the operation of applications with higher security levels.

[0020] On the other hand, if the difference between the estimated baseline backoff assurance time and the baseline backoff assurance time is a first time, the CPU can be controlled to run an application corresponding to the first security level; if the difference between the estimated baseline backoff assurance time and the baseline backoff assurance time is a second time greater than the first time, the CPU can be controlled to run an application corresponding to a second security level higher than the first security level.

[0021] On the other hand, the central processing unit can adjust the baseline rollback assurance time according to the vehicle's driving environment.

[0022] On the other hand, the central processing unit can be controlled to calculate the rollback assurance time based on at least one of the following: system temperature, performance based on security level, system error logs, application latency, or system load.

[0023] On the other hand, the neural processor can perform neural processing based on at least one of the following: system temperature, performance based on security level, system error logs, application latency, or system load, and calculate backoff assurance time based on neural processing.

[0024] On the other hand, neural processors can perform neural processing by feeding back power or temperature data from the results of neural processing.

[0025] On the other hand, the central processing unit can run a hypervisor and run multiple virtual machines on the hypervisor; the first virtual machine of the multiple virtual machines can run neural system services for the control of the neural processor.

[0026] On the other hand, the second virtual machine among the plurality of virtual machines can run vehicle driver assistance (ADAS) applications; the third virtual machine among the plurality of virtual machines can run driver monitoring system (DMS) applications or augmented reality applications.

[0027] On the other hand, if the application running on the second virtual machine in a plurality of virtual machines has a higher security level than the application running on the third virtual machine, the central processing unit can control the application running on the second virtual machine to have a longer baseline rollback assurance time than the application running on the third virtual machine.

[0028] Another embodiment of the signal processing apparatus and the vehicle display device having the signal processing apparatus have a central processing unit that runs an application for driving a vehicle; the central processing unit changes the baseline rollback assurance time for the failure of the application's action based on the security level of the application; if the application's action is determined to have failed based on the baseline rollback assurance time and the calculated rollback assurance time, the central processing unit controls the execution of a rollback action or a failure action.

[0029] Invention Effects

[0030] An embodiment of the signal processing apparatus and a vehicle display device having the signal processing apparatus according to the present invention includes: at least one neural processor; and a central processing unit (CPU) for running an application for driving the vehicle; in the event that the operation of the application is determined to have failed, the CPU controls the operation of a second application corresponding to the application to run on another CPU or other signal processing device, and the CPU changes the baseline backoff assurance time for the failure of the application operation based on the security level of the application. Thus, the application for driving the vehicle can be run stably. In particular, the application for driving the vehicle can be run stably based on the security level.

[0031] On the other hand, when the application's security level is the first security level, the central processing unit (CPU) can set the baseline rollback assurance time to a first time; when the application's security level is the second security level, which is higher than the first security level, the CPU can set the baseline rollback assurance time to a second time that is greater than the first time. This allows for the stable operation of applications used for vehicle operation.

[0032] On the other hand, when the application's security level is the third security level, which is lower than the first security level, the central processing unit can set the baseline backoff assurance time to a third time that is less than the first time. This allows for the stable operation of applications used for vehicle operation.

[0033] On the other hand, if the backoff guarantee time of the operation is greater than or equal to the baseline backoff guarantee time, the central processing unit can control the execution of a pre-set action. This enables the stable operation of applications used for vehicle operation.

[0034] On the other hand, if the backoff guarantee time of the operation is less than the baseline backoff guarantee time, the central processing unit can be controlled to run a second application corresponding to the application on another central processing unit or other signal processing device. This enables the stable operation of the application for vehicle operation.

[0035] On the other hand, if the fallback assurance time of the operation is less than the baseline fallback assurance time, the central processing unit can control the execution of a fallback operation. This enables stable operation of applications used for vehicle driving.

[0036] On the other hand, if the backoff assurance time of the operation is less than the baseline backoff assurance time, the central processing unit can control the execution of any one of the plurality of backoff actions. This enables the stable operation of applications used for vehicle driving.

[0037] On the other hand, multiple reversal actions can include vehicle takeover requests, blind braking, lane keeping braking, lane changing braking, stopping on the shoulder, or path-based parking. This enables stable operation of applications used for vehicle movement.

[0038] On the other hand, the central processing unit can be controlled such that, when the fallback assurance time of the operation is greater than the baseline fallback assurance time, as the fallback assurance time of the operation decreases, the difference between the estimated baseline fallback assurance time and the baseline fallback assurance time becomes smaller, thus restricting the operation of applications with higher security levels. This enables the stable operation of applications used for vehicle operation.

[0039] On the other hand, when the difference between the calculated estimated baseline backoff assurance time and the baseline backoff assurance time is a first time, the CPU can control the operation to run an application corresponding to the first security level; when the difference between the calculated estimated baseline backoff assurance time and the baseline backoff assurance time is a second time greater than the first time, the CPU can control the operation to run an application corresponding to a second security level higher than the first security level. Thus, applications for vehicle operation can be run stably.

[0040] On the other hand, the central processing unit can adjust the baseline backoff assurance time according to the vehicle's driving environment. This enables stable operation of applications used for vehicle operation.

[0041] On the other hand, the central processing unit can calculate the rollback assurance time based on at least one of the following: system temperature, performance based on safety level, system error logs, application latency, or system load. This enables stable operation of applications used for vehicle operation.

[0042] On the other hand, the neural processor can perform neural processing based on at least one of the following: system temperature, performance based on safety level, system error logs, application latency, or system load, and calculate backoff assurance time based on the neural processing. This enables stable operation of applications used for vehicle driving.

[0043] On the other hand, neural processors can perform neural processing by feeding back power or temperature data from the results of neural processing. This enables the stable operation of applications used for vehicle operation.

[0044] On the other hand, the central processing unit can run a supervisor program, and run multiple virtual machines on the supervisor program; the first virtual machine among the multiple virtual machines can run neural system services for the control of the neural processor. Thus, applications for vehicle operation can run stably.

[0045] On the other hand, the second virtual machine among the plurality of virtual machines can run vehicle driver assistance systems (ADAS) applications; the third virtual machine among the plurality of virtual machines can run driver monitoring system (DMS) applications or augmented reality applications. Thus, applications for vehicle operation can run stably.

[0046] On the other hand, if the application running on the second virtual machine among multiple virtual machines has a higher security level than the application running on the third virtual machine, the central processing unit can control the application to have a longer baseline rollback assurance time than the application running on the third virtual machine. This allows for the stable operation of applications used for vehicle operation.

[0047] Another embodiment of the signal processing apparatus and a vehicle display device having the signal processing apparatus includes a central processing unit (CPU) that runs an application for driving a vehicle. The CPU adjusts the baseline rollback guarantee time for application failure based on the application's security level. If, based on the baseline rollback guarantee time and a calculated rollback guarantee time, the CPU determines that the application has failed, it controls the execution of a rollback action or a failure action. This enables stable operation of the application for driving the vehicle. In particular, it enables stable operation of the application for driving the vehicle based on the security level. Attached Figure Description

[0048] Figure 1 This is a diagram showing an example of the exterior and interior of a vehicle.

[0049] Figure 2 This is a diagram illustrating an example of a communication gateway for a vehicle.

[0050] Figure 3a This is a diagram showing an example of the configuration of a vehicle display device inside a vehicle.

[0051] Figure 3b This is another example of the configuration of a vehicle display device inside a vehicle.

[0052] Figure 4 yes Figure 3b An example of an internal block diagram of a vehicle display device.

[0053] Figures 5a to 5d This is a diagram showing various examples of display devices for vehicles.

[0054] Figure 6 This is an example of a block diagram of a vehicle display device according to an embodiment of the present invention.

[0055] Figure 7This is an example of an internal block diagram illustrating a signal processing apparatus according to an embodiment of the present invention.

[0056] Figure 8 This is a diagram illustrating an example of a system driven in a signal processing apparatus according to an embodiment of the present invention.

[0057] Figure 9 This is another example of a block diagram of a vehicle display device according to an embodiment of the present invention.

[0058] Figure 10 This is a flowchart illustrating the operation method of a vehicle display device according to an embodiment of the present invention.

[0059] Figures 11 to 13 It is explaining Figure 9 or Figure 10 The diagram referenced in the action. Detailed Implementation

[0060] The present invention will now be described in detail with reference to the accompanying drawings.

[0061] The suffixes “module” and “section” used in the following description for constituent elements are merely for the convenience of writing the specification and do not have any particularly important meaning or function in themselves. Therefore, “module” and “section” can be used interchangeably.

[0062] Figure 1 This is a diagram showing an example of the exterior and interior of a vehicle.

[0063] Referring to the accompanying drawings, the vehicle 200 is driven by a plurality of wheels 103FR, 103FL, 103RL and a steering wheel 150. The plurality of wheels 103FR, 103FL and 103RL are rotated by a power source, and the steering wheel 150 is used to adjust the direction of travel of the vehicle 200.

[0064] On the other hand, the vehicle 200 may also have a camera 195 for acquiring images of the front of the vehicle.

[0065] On the other hand, multiple displays 180a and 180b for displaying images, information, etc. can be installed inside the vehicle 200.

[0066] exist Figure 1 Examples of multiple displays 180a and 180b include instrument cluster display 180a and AVN (Audio Video Navigation) display 180b. Other examples could be HUD (Head-Up Display), etc.

[0067] On the other hand, the AVN (Audio Video Navigation) display 180b can also be named the Central Information Display.

[0068] On the other hand, the concept of vehicle 200 described in this specification may encompass vehicles that have an engine as a power source, hybrid vehicles that have both an engine and an electric motor as a power source, electric vehicles that have an electric motor as a power source, etc.

[0069] Figure 2 This is a diagram illustrating an example of a communication gateway for a vehicle.

[0070] Referring to the attached diagram, the architecture 300a of the vehicle communication gateway can correspond to a zone-based architecture.

[0071] Therefore, each of the multiple zones Z1 to Z4 can be equipped with a sensor device and a processor inside the vehicle, and a signal processing device 170a including a vehicle communication gateway GWDa can be configured in the central area of ​​the multiple zones Z1 to Z4.

[0072] On the other hand, in addition to the vehicle communication gateway GWDa, the signal processing device 170a may also include an autonomous driving control module ACC and a cockpit control module CPG.

[0073] In this case, the vehicle communication gateway GWDa within the signal processing device 170a can be an HPC (High Performance Computing) gateway.

[0074] Right now, Figure 2 The signal processing device 170a can be used as an integrated HPC gateway to exchange data with external communication modules (not shown) or processors (not shown) in multiple zones Z1 to Z4.

[0075] Figure 3a This is a diagram showing an example of the configuration of a vehicle display device inside a vehicle.

[0076] Referring to the attached diagram, the vehicle interior may be equipped with an instrument cluster display 180a, an AVN (Audio Video Navigation) display 180b, a rear seat entertainment display 180c, 180d, and a rearview mirror display (not shown).

[0077] Figure 3b This is another example of the configuration of a vehicle display device inside a vehicle.

[0078] The vehicle display device 100 of this embodiment may have a plurality of displays 180a to 180b and a signal processing device 170. The signal processing device 170 performs signal processing for displaying images, information, etc. on the plurality of displays 180a to 180b and outputs image signals to at least one display 180a to 180b.

[0079] The first display 180a among the plurality of displays 180a to 180b may be an instrument cluster display 180a for displaying driving status, action information, etc., and the second display 180b may be an AVN (Audio Video Navigation) display 180b for displaying vehicle driving information, navigation maps, various entertainment information or images.

[0080] The signal processing device 170 may have a processor 175 internally configured, and may run a first virtual machine to a third virtual machine (not shown) on a hypervisor (not shown) within the processor 175.

[0081] The second virtual machine (not shown) can operate for the first display 180a, and the third virtual machine (not shown) can operate for the second display 180b.

[0082] On the other hand, the first virtual machine (not shown) within the processor 175 can be controlled to configure the shared memory 508 based on the hypervisor 505 to transmit the same data to the second virtual machine (not shown) and the third virtual machine (not shown). Thus, the same information or the same image can be displayed synchronously on the first display 180a and the second display 180b within the vehicle.

[0083] On the other hand, for data sharing, a first virtual machine (not shown) within the processor 175 shares at least a portion of the data with a second virtual machine (not shown) and a third virtual machine (not shown). Thus, data processing can be shared among multiple virtual machines used for multiple displays within the vehicle.

[0084] On the other hand, the first virtual machine (not shown) within the processor 175 can receive and process the vehicle's wheel speed sensor data, and transmit the processed wheel speed sensor data to at least one of the second virtual machine (not shown) and the third virtual machine (not shown). Thus, the vehicle's wheel speed sensor data can be shared with at least one virtual machine, etc.

[0085] On the other hand, the vehicle display device 100 of the present invention may also include a rear seat entertainment (RSE) display 180c for displaying driving status information, brief navigation information, various entertainment information or images.

[0086] In addition to the first to third virtual machines (not shown), the signal processing device 170 can also control the RSE display 180c by running a fourth virtual machine (not shown) on a hypervisor (not shown) within the processor 175.

[0087] Therefore, a signal processing device 170 can be used to control various displays 180a to 180c.

[0088] On the other hand, some of the multiple displays 180a-180c can operate based on Linux OS (operating system), while others can operate based on Internet OS.

[0089] The signal processing device 170 of this embodiment can be controlled to synchronously display the same information or the same image even when the displays 180a to 180c operate under various operating systems (OS).

[0090] on the other hand, Figure 3b Example: A vehicle speed indicator 212a and a vehicle interior temperature indicator 213a are displayed on a first display 180a; a main screen 222 including a plurality of applications and the vehicle speed indicator 212b and the vehicle interior temperature indicator 213b is displayed on a second display 180b; and a second main screen 222b including a plurality of applications and the vehicle interior temperature indicator 213c is displayed on a third display 180c.

[0091] Figure 4 yes Figure 3b An example of an internal block diagram of a vehicle display device.

[0092] Referring to the accompanying drawings, the vehicle display device 100 of the present invention may include an input unit 110, a communication unit 120 for communication with external devices, a plurality of communication modules EMa to EMD for internal communication, a memory 140, a signal processing device 170, a plurality of displays 180a to 180c, an audio output unit 185, and a power supply unit 190.

[0093] Multiple communication modules EMa to EMD can be configured individually, for example. Figure 2 The multiple zones Z1 to Z4.

[0094] On the other hand, a communication switch 736b for data communication with each communication module EM1 to EM4 may be provided inside the signal processing device 170.

[0095] Each communication module EM1 to EM4 can communicate with multiple sensor devices SN, ECU (Electronic Control Unit) 770, or area signal processing device 170Z.

[0096] On the other hand, the plurality of sensor devices SN may include a camera 195, a lidar 196, a radar 197, or a position sensor 198.

[0097] The input unit 110 may be equipped with physical buttons, tablets, etc., for key input, touch input, etc.

[0098] On the other hand, the input unit 110 may be equipped with a microphone (not shown) for user voice input.

[0099] The communication unit 120 can exchange data wirelessly with the mobile terminal 800 or the server 900.

[0100] In particular, the communication unit 120 can exchange data wirelessly with the vehicle driver's mobile terminal. Various wireless data communication methods can be used, such as Bluetooth, WiFi, WiFi Direct, and APiX.

[0101] The communication unit 120 can receive weather information, road traffic information, such as TPEG (Transport Protocol Expert Group) information, from the mobile terminal 800 or the server 900. For this purpose, the communication unit 120 may include a mobile communication module (not shown).

[0102] Multiple communication modules EM1 to EM4 can receive sensor data from ECU 770, sensor device SN, or area signal processing device 170Z, and transmit the received sensor data to signal processing device 170.

[0103] Here, sensor data may include at least one of the following: vehicle orientation data, vehicle position data (GPS data), vehicle angle data, vehicle speed data, vehicle acceleration data, vehicle tilt data, vehicle forward / reverse data, battery data, fuel data, tire data, headlight data, vehicle interior temperature data, and vehicle interior humidity data.

[0104] This sensor data can be acquired from heading sensors, yaw sensors, gyroscope sensors, position modules, vehicle forward / reverse sensors, wheel sensors, vehicle speed sensors, vehicle tilt sensors, battery sensors, fuel sensors, tire sensors, steering sensors based on steering wheel rotation, vehicle interior temperature sensors, and vehicle interior humidity sensors.

[0105] On the other hand, the positioning module may include a GPS module or a position sensor 198 for receiving GPS (Global Positioning System) information.

[0106] On the other hand, at least one of the plurality of communication modules EM1 to EM4 can transmit the location information data detected by the GPS module or the location sensor 198 to the signal processing device 170.

[0107] On the other hand, at least one of the plurality of communication modules EM1 to EM4 can receive frontal image data, side image data, rear image data, and distance information of obstacles around the vehicle from the camera 195, lidar 196, or radar 197, and transmit the received information to the signal processing device 170.

[0108] The memory 140 may store programs for processing or controlling the signal processing device 170, or for the vehicle display device 100. For example, the memory 140 may store management programs for running on the processor 175, and data about the first virtual machine to the third virtual machine.

[0109] The audio output unit 185 converts the electrical signal from the signal processing device 170 into an audio signal and outputs it. For this purpose, a speaker or the like can be provided.

[0110] The power supply unit 190 can supply the power required for the operation of each component according to the control of the signal processing device 170. In particular, the power supply unit 190 can receive power from the battery or the like inside the vehicle.

[0111] The signal processing device 170 controls the overall operation of each unit within the vehicle display device 100.

[0112] For example, signal processing device 170 may include processor 175 that performs signal processing for vehicle displays 180a, 180b.

[0113] Processor 175 can run a first virtual machine to a third virtual machine (not shown) on a hypervisor (not shown) within processor 175.

[0114] The first virtual machine (not shown) among the first to third virtual machines (not shown) can be named the Server Virtual Machine, and the second to third virtual machines (not shown) can be named the Guest Virtual Machine.

[0115] For example, a first virtual machine (not shown) within processor 175 can receive sensor data from a plurality of sensor devices, such as vehicle sensor data, location information data, camera image data, audio data, or touch input data, and process or manipulate the data before outputting it.

[0116] As described above, data sharing in a 1:N manner can be achieved by performing most of the data processing in the first virtual machine (not shown).

[0117] As another example, the first virtual machine (not shown) can directly receive and process CAN (Controller Area Network) data, Ethernet data, audio data, radio data, USB (Universal Serial Bus) data, and wireless communication data for the second to third virtual machines (not shown).

[0118] In addition, the first virtual machine (not shown) can transfer the processed data to the second virtual machine to the third virtual machine (not shown).

[0119] Therefore, by having only the first virtual machine (not shown) among the first to third virtual machines (not shown) receive sensor data, communication data, or external input data from multiple sensor devices and perform signal processing, the signal processing burden in other virtual machines can be reduced, 1:N data communication can be achieved, and thus synchronization can be achieved when sharing data.

[0120] On the other hand, the first virtual machine (not shown) can be controlled to store data in shared memory 508 and share the same data with the second virtual machine (not shown) and the third virtual machine (not shown).

[0121] For example, the first virtual machine (not shown) can be controlled to store vehicle sensor data, location information data, camera image data, or touch input data in shared memory 508, and share the same data with the second virtual machine (not shown) and the third virtual machine (not shown). This enables 1:N data sharing.

[0122] Ultimately, by performing most of the data processing in the first virtual machine (not shown), 1:N data sharing can be achieved.

[0123] On the other hand, the first virtual machine (not shown) within the processor 175 can be controlled to set up a shared memory 508 based on the hypervisor 505 to transfer the same data to the second virtual machine (not shown) and the third virtual machine (not shown).

[0124] On the other hand, the signal processing device 170 can process various signals such as audio signals, video signals, and data signals. Therefore, the signal processing device 170 can be implemented as a system on chip (SOC).

[0125] on the other hand, Figure 4 The signal processing device 170 within the display device 100 can interact with Figure 5a The signal processing devices 170, 170a1, and 170a2 for the vehicle display device shown in the following figures are the same.

[0126] Figures 5a to 5d This is a diagram showing various examples of display devices for vehicles.

[0127] Figure 5a This is an example of a vehicle display device according to an embodiment of the present invention.

[0128] Referring to the accompanying drawings, the vehicle display device 800a of this embodiment of the invention includes signal processing devices 170a1, 170a2, and a plurality of area signal processing devices 170Z1 to 170Z4.

[0129] On the other hand, the accompanying drawings illustrate two signal processing devices 170a1 and 170a2, but this is for backup purposes, etc., and could also be a single signal processing device.

[0130] On the other hand, signal processing devices 170a1 and 170a2 can also be named HPC (High Performance Computing) signal processing devices.

[0131] Multiple signal processing devices 170Z1 to 170Z4 can be configured in each region Z1 to Z4 and transmit sensor data to signal processing devices 170a1 and 170a2.

[0132] Signal processing devices 170a1 and 170a2 receive data from a plurality of regional signal processing devices 170Z1 to 170Z4 or communication device 120 via wired connection.

[0133] Although the accompanying drawings illustrate the exchange of data between signal processing devices 170a1, 170a2 and multiple regional signal processing devices 170Z1 to 170Z4 based on wired communication, and the exchange of data between signal processing devices 170a1, 170a2 and server 400 based on wireless communication, it is also possible for communication device 120 and server 400 to exchange data based on wireless communication, while signal processing devices 170a1, 170a2 and communication device 120 can exchange data based on wired communication.

[0134] On the other hand, the data received by the signal processing devices 170a1 and 170a2 may include camera data or sensor data.

[0135] For example, sensor data inside the vehicle may include at least one of the following: wheel speed data, vehicle direction data, vehicle position data (GPS data), vehicle angle data, vehicle speed data, vehicle acceleration data, vehicle tilt data, vehicle forward / reverse data, battery data, fuel data, tire data, headlight data, vehicle interior temperature data, vehicle interior humidity data, vehicle exterior radar data, and vehicle exterior lidar data.

[0136] On the other hand, camera data can include data from both external and internal vehicle cameras.

[0137] On the other hand, signal processing devices 170a1 and 170a2 can run multiple virtual machines 820, 830, and 840 according to the safety level.

[0138] The accompanying drawings illustrate a scenario where the processor 175 within the signal processing device 170a runs a management program 505, and on the management program 505, the first virtual machine to the third virtual machine 820 to 840 are run according to the Automotive Safety Integrity Level (ASIL).

[0139] The first virtual machine 820 may be a virtual machine corresponding to QM (Quality Management), which is the lowest safety level in the Automotive Safety Integrity Level (ASIL) and is not mandatory.

[0140] The first virtual machine 820 can run operating system 822, container runtime 824 on operating system 822, and containers 827 and 829 on container runtime 824.

[0141] The second virtual machine 820 can be a virtual machine corresponding to ASIL A or ASIL B, which are the sum of Severity, Exposure, and Controllability in the Automotive Safety Integrity Level (ASIL) of 7 or 8.

[0142] The second virtual machine 820 can run operating system 832, container runtime 834 on operating system 832, and containers 837 and 839 on container runtime 834.

[0143] The third virtual machine 840 can be a virtual machine corresponding to ASIL C or ASIL D, which are automotive safety integrity levels (ASIL) with a sum of 9 or 10 in severity, exposure, and controllability.

[0144] On the other hand, ASIL D can correspond to the level that requires the highest level of security.

[0145] The third virtual machine 840 can run the secure operating system 842 and the application 845 on the operating system 842.

[0146] On the other hand, the third virtual machine 840 can also run a secure operating system 842, a container runtime 844 on the secure operating system 842, and a container 847 on the container runtime 844.

[0147] On the other hand, unlike the attached diagram, the third virtual machine 840 can also run using an additional core instead of processor 175. This will be discussed later. Figure 5b Please provide an explanation.

[0148] Figure 5b Another example of a vehicle display device according to an embodiment of the present invention is shown.

[0149] Referring to the accompanying drawings, the vehicle display device 800b of this embodiment includes signal processing devices 170a1, 170a2 and a plurality of area signal processing devices 170Z1 to 170Z4.

[0150] Although Figure 5b Vehicle display device 800b and Figure 5a The vehicle display device 800a is similar, but the signal processing device 170a1 is... Figure 5a There are some differences between the signal processing device 170a1 and the signal processing device 170a1.

[0151] Focusing on this difference, the signal processing device 170a1 may include a processor 175 and a second processor 177.

[0152] The processor 175 within the signal processing device 170a1 runs a management program 505, and on the management program 505 runs the first virtual machine to the second virtual machine 820 to 830 according to the Automotive Safety Integrity Level (ASIL).

[0153] The first virtual machine 820 can run operating system 822, container runtime 824 on operating system 822, and containers 827 and 829 on container runtime 824.

[0154] The second virtual machine 820 can run operating system 832, container runtime 834 on operating system 832, and containers 837 and 839 on container runtime 834.

[0155] On the other hand, the second processor 177 within the signal processing device 170a1 can run the third virtual machine 840.

[0156] The third virtual machine 840 can run the secure operating system 842, the AUTOSAR (Automotive Open System Architecture) 845 on the operating system 842, and applications 845 on the AUTOSAR 845. That is, it is compatible with... Figure 5a In contrast, it can also run AUTOSAR 846 on the 842 operating system.

[0157] On the other hand, the third virtual machine 840 can also be used with Figure 5a Similarly, secure operating system 842, container runtime 844 on secure operating system 842, and container 847 on container runtime 844 are run.

[0158] On the other hand, preferably, unlike the first to second virtual machines 820 to 830, the third virtual machine 840 with a high security level is required to run on other cores or as a second processor 177 of other processors.

[0159] On the other hand, Figure 5a and Figure 5b In the signal processing devices 170a1 and 170a2, when the first signal processing device 170a malfunctions, the second signal processing device 170a2, which is used as a backup, can operate.

[0160] Alternatively, signal processing devices 170a1 and 170a2 may operate simultaneously, with the first signal processing device 170a acting as the primary operator and the second signal processing device 170a2 acting as an auxiliary operator. For this, refer to... Figure 5c and Figure 5d Please provide an explanation.

[0161] Figure 5c This illustrates yet another example of a vehicle display device according to an embodiment of the present invention.

[0162] Referring to the accompanying drawings, the vehicle display device 800c of this embodiment includes signal processing devices 170a1, 170a2 and a plurality of area signal processing devices 170Z1 to 170Z4.

[0163] On the other hand, the accompanying drawings illustrate two signal processing devices 170a1 and 170a2, but this is for backup purposes, etc., and it could also be a single signal processing device.

[0164] On the other hand, signal processing devices 170a1 and 170a2 can also be named HPC (High Performance Computing) signal processing devices.

[0165] Multiple signal processing devices 170Z1 to 170Z4 can be configured in each region Z1 to Z4 and transmit sensor data to signal processing devices 170a1 and 170a2.

[0166] Signal processing devices 170a1 and 170a2 receive data from a plurality of regional signal processing devices 170Z1 to 170Z4 or communication device 120 via wired connection.

[0167] Although the accompanying drawings illustrate the exchange of data between signal processing devices 170a1, 170a2 and multiple regional signal processing devices 170Z1 to 170Z4 based on wired communication, and the exchange of data between signal processing devices 170a1, 170a2 and server 400 based on wireless communication, it is also possible for communication device 120 and server 400 to exchange data based on wireless communication, while signal processing devices 170a1, 170a2 and communication device 120 can exchange data based on wired communication.

[0168] On the other hand, the data received by the signal processing devices 170a1 and 170a2 may include camera data or sensor data.

[0169] On the other hand, the processor 175 in the first signal processing device 170a1 of the signal processing devices 170a1 and 170a2 can run the management program 505, and a safe virtual machine 860 and a non-safe virtual machine 870 can be run on the management program 505 respectively.

[0170] On the other hand, the processor 175b in the second signal processing device 170a2 of the signal processing devices 170a1 and 170a2 can run the supervisor program 505b, and the safety virtual machine 880 can run only on the supervisor program 505b.

[0171] In this way, the safety processing is divided into a first signal processing device 170a1 and a second signal processing device 170a2, thereby improving stability and processing speed.

[0172] On the other hand, high-speed network communication can be performed between the first signal processing device 170a1 and the second signal processing device 170a2.

[0173] Figure 5d This illustrates yet another example of a vehicle display device according to an embodiment of the present invention.

[0174] Referring to the accompanying drawings, the vehicle display device 800d of this embodiment includes signal processing devices 170a1, 170a2 and a plurality of area signal processing devices 170Z1 to 170Z4.

[0175] Figure 5d Vehicle display device 800d and Figure 5c The vehicle display device 800c is similar, but the second signal processing device 170a2 is... Figure 5c There are some differences in the second signal processing device 170a2.

[0176] Figure 5d The processor 175b within the second signal processing device 170a2 can run a hypervisor 505b, and run a safe virtual machine 880 and a non-safe virtual machine 890 on the hypervisor 505b respectively.

[0177] That is, the difference lies in, with Figure 5c In contrast, the processor 175b within the second signal processing device 170a2 also runs a non-safety virtual machine 890.

[0178] According to this method, the processing for safety and non-safety is divided into a first signal processing device 170a1 and a second signal processing device 170a2, thereby improving stability and processing speed.

[0179] Figure 6 This is an example of a block diagram of a vehicle display device according to an embodiment of the present invention.

[0180] Referring to the accompanying drawings, the vehicle display device 900 of this embodiment of the invention includes a signal processing device 170 and at least one display.

[0181] In the accompanying drawings, instrument cluster display 180a, AVN display 180b, network display 180c, and 180d are exemplified as at least one display.

[0182] On the other hand, the instrument cluster display 180a and the AVN display 180b can be connected to the display port respectively.

[0183] On the other hand, network displays 180c and 180d can each connect to the vehicle's internal network via network ports. This network can be an Ethernet network based on Ethernet communication.

[0184] The accompanying drawings illustrate the connection of network displays 180c and 180d to the third area signal processing device 170Z3 and the fourth area signal processing device 170Z4, respectively. However, unlike this, they can also be connected to other area signal processing devices or directly to the signal processing device 170.

[0185] On the other hand, the vehicle display device 900 may also be equipped with a plurality of area signal processing devices 170Z1 to 170Z4.

[0186] At this time, the signal processing device 170 can be used as a high-performance centralized signal processing and control device with multiple CPUs 175, GPUs 178, NPUs 179, etc., and is named HPC (High Performance Computing) signal processing device or central signal processing device.

[0187] The multiple area signal processing devices 170Z1 to 170Z4 and the signal processing device 170 are connected by wired cables CB1 to CB4.

[0188] On the other hand, the multiple regional signal processing devices 170Z1 to 170Z4 can be connected to each other using wired cables CBa to CBd.

[0189] At this time, the wired cables CBa to CBd may include CAN communication cables, Ethernet communication cables, or PCI Express (peripheral component interconnect express) cables.

[0190] On the other hand, the signal processing apparatus 170 of the present invention may be provided with at least one processor 175, 178, 177 and a large-capacity storage device 925.

[0191] For example, the signal processing device 170 of this embodiment may include a central processing unit 175, 177, a graphics processor 178, and a neural processor 179.

[0192] On the other hand, sensor data can be transmitted from at least one of the multiple region signal processing devices 170Z1 to 170Z4 directional signal processing devices 170. In particular, the sensor data can be stored in the storage device 925 within the signal processing device 170.

[0193] The sensor data at this time may include at least one of the following: camera data, lidar data, radar data, vehicle direction data, vehicle position data (GPS data), vehicle angle data, vehicle speed data, vehicle acceleration data, vehicle tilt data, vehicle forward / reverse data, battery data, fuel data, tire data, headlight data, vehicle interior temperature data, and vehicle interior humidity data.

[0194] The accompanying drawings illustrate a scenario where camera data from camera 195a and lidar data from lidar sensor 196 are input to a first area signal processing unit 170Z1, and the camera data and lidar data are transmitted to a signal processing unit 170 via a second area signal processing unit 170Z2 and a third area signal processing unit 170Z3, etc.

[0195] On the other hand, since the speed at which the storage device 925 reads and writes data is faster than the network speed when transmitting sensor data from at least one direction signal processing device 170 among the plurality of area signal processing devices 170Z1 to 170Z4, it is preferable to perform multi-path routing to prevent network bottlenecks.

[0196] Therefore, the signal processing apparatus 170 of this embodiment can perform multi-path routing based on Software-Defined Network (SDN). This ensures a stable network environment for data reading and writing to the storage device 925. Furthermore, since multiple paths can be used to transmit data to the storage device 925, data can be transmitted by dynamically changing the network configuration.

[0197] For high-frequency, low-latency communication, the data communication between the plurality of area signal processing devices 170Z1 to 170Z4 within the vehicle display device 900 of this embodiment and the signal processing device 170 is preferably Peripheral Component Interconnect Express (PCI Express) communication.

[0198] Figure 7This is an example of an internal block diagram of a signal processing device according to an embodiment of the present invention.

[0199] Referring to the accompanying drawings, a signal processing system 1000 according to an embodiment of the present invention may have a central signal processing device 170 and a regional signal processing device 170z.

[0200] On the other hand, the signal processing device 170 in the system 1000 of one embodiment of the present invention has a plurality of processor cores CR1 to CRn and MR.

[0201] On the other hand, a subset of the multiple processor cores CR1 to CRn, and a portion of CR1 to CRn in the MR, can interact with... Figure 6 The processor cores within the central processing unit (CPU) correspond to these cores.

[0202] For example, a plurality of processor cores CR1 to CRn, and a subset of CR1 to CRn in MR, can be coupled with... Figure 6 The application processor core within the central processing unit (CPU) corresponds to this.

[0203] On the other hand, a plurality of processor cores CR1 to CRn, and a portion of CR1 to CRn in MR, can be operated based on hypervisor 505, and the hypervisor can run a plurality of virtual machines 820 to 850.

[0204] On the other hand, another part of the plurality of processor cores CR1 to CRn and MR can correspond to M core or MCU (microcontroller unit).

[0205] On the other hand, another part of the plurality of processor cores CR1 to CRn and MR can run an operating system 805a corresponding to the second security level such as ASIL D without running the hypervisor 505, and run a fourth virtual machine 840 on the operating system 805a.

[0206] On the other hand, the fourth virtual machine 840 can run applications corresponding to the second security level, such as ASIL D, or microservices 843 corresponding to applications corresponding to the second security level. Therefore, applications or microservices 843 corresponding to the second security level can run stably.

[0207] On the other hand, the first processor core CR1 among the plurality of processor cores CR1 to CRn and MR can run a hypervisor 505 and run an operating system 805b corresponding to a second security level such as ASIL D on the hypervisor 505, and run a first virtual machine 850 on the operating system 805b.

[0208] On the other hand, the first virtual machine 850 can run applications corresponding to a first security level such as ASIL B, or microservices 853a and 853b corresponding to applications corresponding to the first security level. Therefore, applications or microservices 853a and 853b corresponding to the first security level can run stably.

[0209] On the other hand, unlike the attached figure, the first processor core CR1 among the plurality of processor cores CR1 to CRn and MR can also run an operating system corresponding to a first security level such as ASIL B on the hypervisor 505.

[0210] On the other hand, the plurality of processor cores CR1 to CRn, the second processor core CR2 and the third processor core CR3 in MR can run a hypervisor 505, and run an operating system 805c corresponding to a first security level such as ASIL B on the hypervisor 505, and run a second virtual machine 850 on the operating system 805c.

[0211] On the other hand, the second virtual machine 850 can run a third application corresponding to the first security level, such as ASIL B, or microservices 833a to 833d corresponding to the third application corresponding to the first security level on an operating system 805c corresponding to the first security level. Thus, the application or microservices 833a to 833d corresponding to the first security level can run stably.

[0212] On the other hand, the plurality of processor cores CR1 to CRn and the remaining processor cores CR4 to CRn in MR can run the hypervisor 505, and run the operating system 805d corresponding to the third security level such as QM on the hypervisor 505, and run the third virtual machine 820 on the operating system 805d.

[0213] On the other hand, the third virtual machine 820 can run a fourth application corresponding to the third security level, such as QM, or microservices 823a to 823d corresponding to the fourth application corresponding to the third security level on an operating system 805d that corresponds to a third security level lower than the first security level. Therefore, applications or microservices 823a to 823d corresponding to the third security level can run stably.

[0214] On the other hand, the regional signal processing device 170z may have a plurality of application processor cores CRR1 to CRRm and an M core MRb, wherein the M core MRb is used to run applications such as ASIL D, which is the second security level, which is the highest security level.

[0215] On the other hand, a portion of the plurality of processor cores CRR1 to CRRm and MRb in the regional signal processing device 170z, RR1 to CRRm, can run an operating system 806b corresponding to a first security level such as ASIL B, and run a virtual machine 830b corresponding to the first security level on the operating system 806a.

[0216] On the other hand, the virtual machine 830b corresponding to the first security level can run applications corresponding to the first security level, such as ASIL B, or microservices 830ba to 830bd corresponding to applications corresponding to the first security level. Therefore, applications or microservices 830ba to 830bd corresponding to the first security level can run stably.

[0217] On the other hand, another part of the multiple processor cores CRR1 to CRRm and MRb in the regional signal processing device 170z can run an operating system 806a corresponding to the second security level such as ASIL D, and run a virtual machine 840b corresponding to the second security level such as ASIL D on the operating system 806a.

[0218] On the other hand, the virtual machine 840b corresponding to the second security level can run applications corresponding to the second security level, such as ASIL D, or microservices 843b corresponding to applications corresponding to the second security level. Therefore, applications or microservices 843b corresponding to the second security level can run stably.

[0219] Figure 8 This is a diagram illustrating an example of a system driven in a signal processing apparatus according to an embodiment of the present invention.

[0220] Referring to the accompanying drawings, a signal processing device 170 within a signal processing system 1000 according to an embodiment of the present invention has a central processing unit 175 and at least one neural processor 179a to 179c.

[0221] On the other hand, the signal processing apparatus 170 of this embodiment may also have a graphics processor 178.

[0222] On the other hand, the central processing unit 175 of this embodiment of the invention runs the management program 505.

[0223] On the other hand, the system 1100 driven in the signal processing apparatus 170 of the present invention runs a plurality of virtual machines 810 to 850 on the hypervisor 505.

[0224] Specifically, the central processing unit 175 in the signal processing device 170 of this embodiment of the invention runs a management program 505 and runs a plurality of virtual machines 810 to 850 on the management program 505.

[0225] On the other hand, the central processing unit 175 within the signal processing device 170 of this embodiment of the invention runs an application for driving the vehicle.

[0226] On the other hand, the central processing unit 175 can be controlled to run a second application corresponding to the application on another central processing unit or other signal processing device when the application is determined to have failed, and to change the baseline rollback assurance time for the application's failed action based on the application's security level.

[0227] This enables applications used for vehicle operation to run stably. In particular, it enables applications used for vehicle operation to run stably based on safety levels.

[0228] On the other hand, the signal processing apparatus 170 of one embodiment of the present invention may also include a shared memory 508.

[0229] The accompanying drawings illustrate a scenario where a hypervisor 505 runs on a central processing unit 175, and a shared memory 508 runs within the hypervisor 505.

[0230] On the other hand, the signal processing apparatus 170 of the present invention can receive data from the camera device 195, the sensor device 700, the communication device 120 or the lidar device (not shown), and perform signal processing using at least one of the central processing unit 175, the graphics processor 178 and a plurality of neural processors 179a to 179c.

[0231] On the other hand, the sensor device 700 can continuously output sensor data to the signal processing device 170 during the vehicle's movement.

[0232] The sensor data at this time can be data from various sensor devices 700 of the vehicle, including at least one of the following: vehicle direction data, vehicle position data (GPS data), vehicle angle data, vehicle speed data, vehicle acceleration data, vehicle tilt data, vehicle forward / reverse data, battery data, fuel data, tire data, headlight data, vehicle interior temperature data, and vehicle interior humidity data.

[0233] On the other hand, the camera device 195 can continuously output camera data to the signal processing device 170 during the movement of the vehicle.

[0234] On the other hand, the lidar (not shown) can continuously output lidar data to the signal processing unit 170 during the vehicle's movement.

[0235] On the other hand, the neural processor 179 can detect objects based on camera data and act on or output result data including the objects at a variable frame rate based on the objects.

[0236] On the other hand, the neural processor 179 can receive camera data at a fixed frame rate and detect objects based on the camera data, and act on or output result data including the objects at a variable frame rate based on the objects.

[0237] On the other hand, the first virtual machine 810, which is a server virtual machine among the plurality of virtual machines 810 to 850, controls the operation of the neural processor 179.

[0238] On the other hand, among the plurality of virtual machines 810 to 850, the second virtual machine 850 and the third virtual machine 830, which serve as guest virtual machines, can run applications separately.

[0239] The accompanying diagram illustrates a scenario where a second virtual machine 850 runs a vehicle driver assistance system (ADAS) application (Nad) or an autonomous driving application, and a third virtual machine 830 runs a driver monitoring system (DMS) application (Ndm) and an augmented reality (AR) application (Nar).

[0240] The first virtual machine 810 can be controlled such that, when it sequentially receives requests for a first operation, a second operation, and a third operation from at least one of the plurality of virtual machines 810 to 850, and the first and third operations can be processed in parallel, the first neural processor 179a processes the first and third operations in parallel, and processes the second operation after completing the first and third operations. This allows the neural processor to operate efficiently. Furthermore, it reduces power consumption.

[0241] On the other hand, the first virtual machine 810 can be controlled such that, in a state where a request for a fourth operation is received after a request for a third operation, and given that the operation layers used for the second and fourth operations can be shared, the first neural processor 179a can continuously process the second and fourth operations after completing the first and third operations. This allows the neural processor to operate effectively.

[0242] On the other hand, the first virtual machine 810 can be controlled to change the arrangement of data for the multiple operations in the internal memory 1805 within the first neural processor 179a when receiving multiple operation requests from multiple applications. This enables the neural processor to operate effectively.

[0243] On the other hand, the first virtual machine 810 can run neural system services 1110 for the control of at least one neural processor 179a to 179c.

[0244] On the other hand, the neural system service 1110 can be controlled to change the arrangement of data for the multiple operations in the internal memory 1805 within the first neural processor 179a when it receives requests for multiple operations from multiple applications. This allows the neural processor to operate effectively.

[0245] On the other hand, the neural system service 1110 may run or have at least one neural manager 1113 for managing the neural processors 179a to 179c, at least one neural controller 1115 for determining or controlling the inference mode of the neural processors 179a to 179c, and at least one neural interface 1118 for interface connection with the neural processors 179a to 179c.

[0246] On the other hand, the neural system service 1110 may also run or have a model container 509, which manages the interface of model parameters related to the actions of the neural processor 179 and the version of the learning file.

[0247] The Neural Manager 1113 can perform artificial intelligence model management, learning model management, camera data management, sensor data management, or command queue management.

[0248] The neural controller 1115 can determine the optimal inference method for at least one neural processor 179a-179c, or execute queues, partitions, caches, scalable codes, or control at least one neural processor 179a-179c.

[0249] The neural interface 1118 can run an application programming interface (API) associated with an accelerator of at least one neural processor 179a-179c.

[0250] On the other hand, the interface 522 within the first virtual machine 810 can perform interface connections between the neural system service 1110 and the model container 509 or between the neural system service 1110 and the shared memory 508.

[0251] On the other hand, interface 522 within the first virtual machine 810 can perform interface connections for the first virtual machine 810.

[0252] On the other hand, interface 522 within the first virtual machine 810 can also perform interface connections for the vehicle driving assistance application Nad running within the second virtual machine 850, or the driver monitoring system application Ndm or the augmented reality application Nar running within the third virtual machine 830.

[0253] For example, the interface 522 within the first virtual machine 810 can be controlled to use shared memory 508 to transmit camera data, sensor data, or voice data to the neural processor 179.

[0254] On the other hand, the interface 522 within the first virtual machine 810 can be controlled to transmit the result data output from the neural processor 179 and stored in the shared memory 508 to the neural system service 1110.

[0255] On the other hand, the interface 522 within the first virtual machine 810 can be controlled to transmit the result data output from the neural processor 179 and stored in the shared memory 508 to the vehicle driving assistance application Nad running in the second virtual machine 850, or the driver monitoring system application Ndm or the augmented reality application Nar running in the third virtual machine 830.

[0256] On the other hand, the first virtual machine 810 can run on the first operating system 805, the second virtual machine 850 can run on the second operating system 805b with a high security level, and the third virtual machine 830 can run on the third operating system 805c.

[0257] That is, multiple virtual machines 810 to 850 can each run on different operating systems, or they can run on at least two operating systems.

[0258] On the other hand, the neural manager 1113 can manage the driving requirements of applications based on artificial neural networks, control neural network weight data, and process the required input data.

[0259] On the other hand, the neural manager 1113 can process the optimized command queue sequentially through a hardware accelerator and send the computation results to the application.

[0260] Driving requirements can include the computational priority, dependencies, and accuracy of the neural network. Computational priority refers to the requirement that the first operation must always be processed before the second, or, in the case of a safety-critical neural network, that the network be processed first in the command queue compared to other candidate neural networks. This priority is a pre-defined value.

[0261] On the other hand, neural network weight data can refer to the file in which the values ​​of each matrix element are structured and stored during the inference process of the neural network results calculated through a series of matrix operations.

[0262] The neural network weight data can be pre-stored in the model container 509 within the neural system service 1110 through API calls during application installation.

[0263] On the other hand, the basic weight data loaded into the model container 509 can be automatically converted and stored at various discretization levels during system initialization. For example, if the basic weights are defined as FP32, they can be discretized into INT8, INT16, and FP16, thus providing four weight files for storage.

[0264] The required input data can refer to vehicle speed, current position, radar, lidar, camera images, and intermediate to final calculation results from the previous neural network, as well as other input signals required for the current neural network's action.

[0265] Input data can be transmitted in real time from the server virtual machine to the shared memory 508 within the hypervisor 505 via an interface that utilizes the actions of the central processing unit 175.

[0266] Command queues are storage buffers with sequential FIFO data structures, which can be defined by hardware accelerators to provide a series of sequences for processing artificial neural networks.

[0267] A neural network computation request entering the command queue can be transmitted along with metadata such as the application name, the location of the application virtual machine, the storage destination of the computation results, hardware accelerator control settings, the memory location of the input data, and the memory location information of the discretization level of the weighted data.

[0268] The hardware accelerator control settings may include the inherent number of the hardware accelerator responsible for the computation, the current object discretization level of the weight data (INT8, INT16, FP16, FP32, etc.), and a mapping table that maps the internal memory addresses of each hardware accelerator to the weight positions of the object neural network.

[0269] On the other hand, the neural controller 1115 can schedule an optimized command queue based on the requested artificial neural network operation instructions and the availability of current hardware resources, and control the matching of the actual hardware accelerator with the expected actions of the command queue.

[0270] The neural controller 1115 can receive neural network driving requirements from the neural manager 1113 and optimize the instruction queue.

[0271] In other words, the optimization process involves determining the priority, dependency, and accuracy metadata of all slots in the current command queue. When applying various queue optimization techniques (partition, caching, accuracy coding, etc.) to all candidate instructions in the current command queue, it can simulate scheduling to compute combinations of directions that maximize hardware utilization and minimize the latency of individual computation requests per unit time.

[0272] Based on the optimal slot position obtained in this way, the weight file (learning model) can be requested from the neural manager 1113 and loaded into the internal hardware memory.

[0273] If we use partitioning techniques in optimization to manage two different neural networks like a virtual neural network and use them as inputs for hardware computation requests, then the start and end positions of the weights of the first operation, corresponding to the address numbers of the internal hardware memory, are recorded in a mapping table. Then, the start and end positions of the weights of the second operation can be recorded in the mapping table.

[0274] Thus, although the hardware accelerator appears to perform parallel processing on a virtual neural network, the neural controller 1115 can use a mapping table to divide the result of the operation into the result of the first operation and the result of the second operation and send them to the respective applications.

[0275] After the initialization process is completed, the neural controller 1115 can receive sequential processing requests from the command queue from the neural manager 1113.

[0276] At this time, the neural controller 1115 can be controlled to retrieve the input data prepared in advance by the neural manager 1113 from the input data queue, make a pair between the neural network weights and their corresponding input data, and perform computation processing through the hardware accelerator API.

[0277] If, unlike the initial driving requirements, the discretization level of the current neural network changes according to specific conditions, the neural controller 1115 can perform a bitwise concanate that connects the weight conversion difference (Delta) of the hardware internal memory bitwise to the basic weights of the current internal memory in real time, thereby changing the discretization level of the basic weights of the internal memory.

[0278] On the other hand, the central processing unit 175 controls itself to run a second application corresponding to the application in another central processing unit 175 or other signal processing device 170 when the application for driving the vehicle is running and it is determined that the application has failed, and to change the baseline rollback guarantee time for the failure of the application based on the security level of the application.

[0279] On the other hand, the rollback guarantee time can refer to the time from the rollback start point to the rollback end point.

[0280] Alternatively, the rollback assurance time can refer to the period from the rollback start time to the rollback end time, starting from the point when the application's action is judged to have failed or a fault is judged.

[0281] On the other hand, safety level can refer to Automotive Safety Integrity Level (ASIL) or Autonomous Driving Level, or a combination of Automotive Safety Integrity Level and Autonomous Driving Level.

[0282] This enables applications used for vehicle operation to run stably. In particular, it enables applications used for vehicle operation to run stably based on safety levels.

[0283] On the other hand, the central processing unit 175 can set the base rollback assurance time to a first time when the application's security level is the corresponding first security level, and set the base rollback assurance time to a second time that is greater than the first time when the application's security level is a second security level higher than the first security level. This enables stable operation of applications used for vehicle operation.

[0284] For example, in the autonomous driving level 3 state, the central processing unit 175 can set the reference backoff assurance time to approximately 10 seconds as the first time for the first application corresponding to ASIL D, and in the autonomous driving level 4 state, the central processing unit 175 can set the reference backoff assurance time to approximately 30 seconds as the second time for the second application corresponding to ASIL D. Thus, applications for vehicle driving can be operated stably based on the safety level.

[0285] As another example, in the autonomous driving level 3 state, the central processing unit 175 can set the baseline rollback assurance time to approximately 7 seconds for the third application corresponding to ASIL B, and in the autonomous driving level 3 state, the central processing unit 175 can set the baseline rollback assurance time to approximately 10 seconds for the fourth application corresponding to ASIL D. Thus, applications for vehicle driving can be operated stably based on the safety level.

[0286] As another example, when the safety level of the driver monitoring system application Ndm is the first safety level corresponding to ASIL B, the central processing unit 175 can set the baseline backoff assurance time to approximately 10 seconds as the first time. When the safety level of the vehicle driving assistance application Nad is the second safety level corresponding to ASIL D, which is higher than ASIL B, the central processing unit 175 can set the baseline backoff assurance time to approximately 30 seconds as the second time. Thus, applications for vehicle operation can operate stably based on the safety level.

[0287] On the other hand, if the application's security level is the third security level, which is lower than the first security level, the central processing unit 175 can set the baseline rollback assurance time to a third time that is less than the first time. This allows the application for vehicle operation to run stably.

[0288] For example, in the state of autonomous driving level 2, the central processing unit 175 can set the reference backoff assurance time to approximately 1 second as the third time for the fifth application corresponding to ASIL D or ASIL B.

[0289] As another example, in the state of autonomous driving level 2, the central processing unit 175 can set the baseline rollback assurance time to approximately 0.7 seconds for the sixth application corresponding to QM.

[0290] As another example, when the augmented reality application Nar has a safety level of the third level, corresponding to QM below ASIL B, the central processing unit 175 can set the baseline backoff assurance time to approximately 0.5 seconds as the third time. This allows the application for vehicle operation to run stably based on the safety level.

[0291] On the other hand, the central processing unit 175 can be controlled such that, when the second virtual machine 850 among the plurality of virtual machines 810, 830, and 850 is running an application with a higher security level than the third virtual machine 830, the baseline rollback assurance time of the application running on the second virtual machine 850 is greater than the baseline rollback assurance time of the application running on the third virtual machine.

[0292] For example, when the second virtual machine 850 is running a first application with an autonomous driving level of 4, the central processing unit 175 can set the baseline rollback assurance time to approximately 30 seconds. When the third virtual machine 830 is running a second application with an autonomous driving level of 3, the central processing unit 175 can set the baseline rollback assurance time to approximately 10 seconds. Thus, applications for vehicle driving can be run stably based on safety levels.

[0293] As another example, when the second virtual machine 850 runs the vehicle driver assistance application Nad corresponding to ASIL D, the central processing unit 175 can set the baseline rollback assurance time of the vehicle driver assistance application Nad to approximately 30 seconds. When the third virtual machine 830 runs the driver monitoring system application Ndm corresponding to ASIL B, the central processing unit 175 can set the baseline rollback assurance time of the driver monitoring system application Ndm to approximately 10 seconds. Thus, applications for vehicle operation can run stably based on safety levels.

[0294] Figure 9This is another example of a block diagram of a vehicle display device according to an embodiment of the present invention.

[0295] Referring to the accompanying drawings, the signal processing system 1200 in the vehicle display device of this embodiment may have a first central signal processing device 170a1 and a second central signal processing device 170a2.

[0296] On the other hand, the first central signal processing device 170a1 can be a primary signal processing device, and the second central signal processing device 170a2 can be a secondary signal processing device.

[0297] That is, when the first central signal processing unit 170a1 is determined to be faulty, the second central signal processing unit 170a2 can take over (fallback) the operation.

[0298] At this time, the first central signal processing unit 170a1 can communicate with... Figure 8 The central signal processing unit 170 corresponds to this.

[0299] On the other hand, with Figure 8 Similarly, the second central signal processing device 170a2 has a central processing unit 175b and at least one neural processor 179b.

[0300] On the other hand, the second central signal processing device 170a2 in this embodiment of the invention may also have a graphics processor 178b.

[0301] On the other hand, the central processing unit 175b in this embodiment of the invention can run the management program 505b.

[0302] On the other hand, in the system 1100b driven by the second central signal processing device 170a2 in this embodiment of the invention, a plurality of virtual machines 810b to 850b can run on the management program 505b.

[0303] Specifically, the central processing unit 175b in the second central signal processing device 170a2 of this embodiment can run a management program 505b and run a plurality of virtual machines 810 to 850b on the management program 505b.

[0304] On the other hand, among the plurality of virtual machines 810b to 850b, the first virtual machine 810b, which serves as the server virtual machine, can control the operation of the neural processor 179b.

[0305] On the other hand, among the plurality of virtual machines 810b to 850b, the second virtual machine 850b and the third virtual machine 830b, which serve as guest virtual machines, can run applications separately.

[0306] The accompanying diagram illustrates a scenario where the second virtual machine 850b runs the vehicle driver assistance system (ADAS) application Nadb, and the third virtual machine 830b runs the driver monitoring system (DMS) application Ndmb and the augmented reality (AR) application Narb.

[0307] On the other hand, the first virtual machine 810b can run neural system services 1110b under the control of at least one neural processor 179b.

[0308] On the other hand, the neural system service 1110b may run or have a neural manager 1113b for managing at least one neural processor 179b, a neural controller 1115b for determining or controlling at least one neural processor 179b in an inferential manner, and a neural interface 1118b for interface connection with at least one neural processor 179b.

[0309] On the other hand, the neural system service 1110b may also run or have a model container 509b, which manages the interface for model parameters related to the actions of the neural processor 179b and the version of the learning files.

[0310] For example, if the first central signal processing unit 170a1 determines that the vehicle driving assistance application Nad has failed or malfunctioned during operation, the second central signal processing unit 170a2 can run the second vehicle driving assistance application Nadb corresponding to the vehicle driving assistance application Nad as a substitute. This enables the stable operation of the application used for vehicle driving.

[0311] As another example, if the first central signal processing unit 170a1 determines that the autonomous driving application has failed or malfunctioned during operation, the second central signal processing unit 170a2 can operate the second autonomous driving application corresponding to the autonomous driving application as a substitute. This enables the stable operation of the application used for vehicle driving.

[0312] As another example, if the first central signal processing unit 170a1 determines that the driver monitoring system application Ndm has failed or malfunctioned during operation, the second central signal processing unit 170a2 can operate the second driver monitoring system application Ndmb corresponding to the driver monitoring system application Ndm as a substitute. This allows for the stable operation of the application used for vehicle driving.

[0313] On the other hand, preferably, the reference backoff assurance time when the first central signal processing device 170a1 operates the vehicle driving assistance application Nad is greater than the reference backoff assurance time when the first central signal processing device 170a1 operates the driver monitoring system application Ndm.

[0314] In particular, preferably, when the safety level of the vehicle driver assistance application (Nad) is greater than that of the driver monitoring system application (Ndm), the baseline backoff assurance time when operating the vehicle driver assistance application (Nad) is greater than that when operating the driver monitoring system application (Ndm). Therefore, applications for vehicle operation can be operated stably based on the safety level.

[0315] Figure 10 This is a flowchart illustrating the operation method of a vehicle display device according to an embodiment of the present invention.

[0316] Referring to the accompanying drawings, the central processing unit 175 within the signal processing device 170 can run applications for vehicle operation (S1010).

[0317] On the other hand, the central processing unit 175 within the signal processing device 170 can be controlled to calculate a backoff guarantee time corresponding to a generally expected time based on the vehicle's sensor data or camera data or the operating state of the signal processing device (S1015).

[0318] For example, the central processing unit 175 can be controlled to calculate the rollback assurance time based on at least one of the following: system temperature, performance based on the security level, system error logs, application latency, and system load.

[0319] Specifically, the neural processor 179 can perform neural processing based on at least one of the following: system temperature, performance based on safety level, system error logs, application latency, and system load, and calculate the rollback assurance time based on the neural processing. This enables stable operation of applications used for vehicle operation.

[0320] On the other hand, the central processing unit 175 within the signal processing device 170 can be controlled to determine that the application's operation has failed (S1020), and in the event of failure, compare the fallback guarantee time of the operation with the reference fallback guarantee time (S1025), and if the fallback guarantee time of the operation is less than the reference fallback guarantee time, perform a fallback operation (S1030).

[0321] For example, if the backoff assurance time of the operation is less than the reference backoff assurance time, the central processing unit 175 in the signal processing device 170 can be controlled to perform any one of the plurality of backoff actions.

[0322] At this time, multiple back-off actions can include driver takeover request, blind braking, lane keeping and blind braking, emergency lane change and braking, pull over on the shoulder, or long distance route planning.

[0323] On the other hand, in step S1025, if the backoff assurance time of the operation is greater than or equal to the reference backoff assurance time, the central processing unit 175 can control the execution of a preset action (S1035).

[0324] On the other hand, the central processing unit 175 can be controlled to execute a pre-set failure action when the back-off assurance time of the operation is greater than or equal to the baseline back-off assurance time. In this case, it is preferable to execute an appropriate failure action.

[0325] For example, the central processing unit 175 can be controlled to run a second application corresponding to the application in another central processing unit 175 or other signal processing device 170 when the backoff assurance time of the operation is less than the reference backoff assurance time. This enables stable operation of the application for vehicle driving.

[0326] On the other hand, the central processing unit 175 can be controlled such that, when the fallback assurance time of the operation is greater than the baseline fallback assurance time, as the fallback assurance time of the operation decreases, the difference between the estimated baseline fallback assurance time and the baseline fallback assurance time becomes smaller, thereby restricting the operation of applications with higher security levels.

[0327] For example, the central processing unit 175 can be controlled to run an application corresponding to a first security level when the difference between the estimated baseline backoff guarantee time and the baseline backoff guarantee time is a first time, and to run an application corresponding to a second security level higher than the first security level when the difference between the estimated baseline backoff guarantee time and the baseline backoff guarantee time is a second time greater than the first time.

[0328] That is, the central processing unit 175 can be controlled to run an application corresponding to a second security level that is higher than the first security level when the backoff assurance time of the operation is greater than the baseline backoff assurance time, and when the difference between the estimated baseline backoff assurance time and the baseline backoff assurance time is greater than a first time. When the difference between the estimated baseline backoff assurance time and the baseline backoff assurance time is the first time, the operation of the application corresponding to the second security level is restricted, and the application corresponding to the first security level is run instead.

[0329] Specifically, the central processing unit 175 can be controlled to run the driver monitoring system application Ndm corresponding to the first safety level when the difference between the calculated estimated baseline backoff assurance time and the baseline backoff assurance time is a first time, and to run the vehicle driving assistance application Nad with a higher safety level than the driver monitoring system application Ndm when the difference between the baseline backoff assurance time and the baseline backoff assurance time is a second time greater than the first time. Thus, applications for vehicle operation can be stably run based on the safety level.

[0330] On the other hand, the central processing unit 175 can change the baseline rollback assurance time according to the vehicle's driving environment.

[0331] For example, the central processing unit 175 can control the baseline backoff assurance time to increase as the vehicle travels a greater distance.

[0332] As another example, the central processing unit 175 can control the base rollback assurance time to increase as the number of vehicle component repairs increases. This allows the application to operate stably in accordance with the vehicle's condition.

[0333] On the other hand, unlike the accompanying drawings, the central processing unit 175 within the signal processing device 170 can also be controlled to learn correctable faults based on vehicle sensor data or camera data or the operating state of the signal processing device between steps S1010 and S1015, and to perform active control based on the learning.

[0334] That is, the central processing unit 175 within the signal processing device 170 can learn correctable faults during the operation of the application and perform active control based on the learning before the application fails or malfunctions.

[0335] Active control at this time may include changing the drivable path by considering safe areas, or displaying autonomous driving level limits and warning symptoms on the display based on the warning symptoms corresponding to correctable faults.

[0336] On the other hand, the central processing unit 175 within the signal processing device 170 can be controlled to change the intensity of active control according to the security level when performing active control.

[0337] For example, the central processing unit 175 within the signal processing device 170 can be controlled such that, when performing active control, the intensity of active control is greater when the autonomous driving level is level 4 than when the autonomous driving level is level 3. This enables stable control.

[0338] On the other hand, the central processing unit 175 within the signal processing device 170 can be controlled to perform a rollback action based on the comparison result of the calculated rollback entry time and the reference rollback entry time, even in active control or the like, if it is determined in step S1020 that an action failure or malfunction has occurred during the operation of the application. This enables stable operation of the application for vehicle driving.

[0339] Figures 11 to 13 It is explaining Figure 9 or Figure 10 The diagram referenced in the action.

[0340] Figure 11 This is a diagram illustrating how backtracking of neural networks ensures the computation of time.

[0341] Referring to the attached diagram, the neural processor 179 can use the neural network 1220 to calculate the backoff assurance time.

[0342] On the other hand, the neural processor 179 can set the hardware monitoring information 1203, vehicle status information 1207, and system information 1209 of the signal processing device 170 as the input layer, and use a neural network to calculate the backoff assurance time.

[0343] On the other hand, the hardware monitoring information 1203 of the signal processing device 170 may include the system temperature information and the performance information based on the safety level of the signal processing device 170.

[0344] On the other hand, vehicle status information 1207 may include vehicle erroneous repair record information or system error records, etc.

[0345] On the other hand, the system information 1209 of the signal processing device may include system error records, application delay information, or system load information, etc.

[0346] That is, the neural processor 179 can perform neural processing based on at least one of the following: system temperature, performance based on security level, system error logs, application latency, and system load, and output result data based on the neural processing.

[0347] The resulting data at this point may include the rollback assurance time 1223 and power or temperature data 1224.

[0348] On the other hand, the neural processor 179 can perform neural processing by feeding back power data or temperature data from the results of neural processing. This enables stable operation of applications used for vehicle operation.

[0349] On the other hand, neural networks can include state-of-the-art neural networks for effective learning.

[0350] For example, neural networks can include deep neural networks (DNN), convolutional neural networks (CNN), recurrent neural networks (RNN), federated learning, or neural networks based on transformers that use self-attention mechanisms.

[0351] On the other hand, neural network models can change based on vehicle driving distance, driving records, repair or component replacement records, or application updates, hardware updates, etc.

[0352] Alternatively, neural processor 179 can calculate the rollback assurance time based on vehicle mileage, driving records, repair or component replacement records, or application updates, hardware updates, etc.

[0353] Figure 12 It is used to illustrate based on Figure 11 The diagram illustrates the actions of the failoperation manager, which ensures the rollback of operations.

[0354] Referring to the attached diagram, the fault action manager 819 can ensure the action of time 1223 based on the calculated rollback.

[0355] The fault action manager 819 can control the execution of either a rollback action Sfa or a fault action Sop, based on the comparison between the calculated rollback assurance time and the baseline rollback assurance time.

[0356] For example, the fault action manager 819 can be controlled to execute any one of a plurality of rollback actions if the calculated rollback assurance time is less than the baseline rollback assurance time.

[0357] As another example, the fault action manager 819 can be controlled to execute a pre-set fault action Sop when the calculated rollback assurance time is greater than or equal to the baseline rollback assurance time.

[0358] On the other hand, the fault action manager 819 can control the execution of fault actions to include, in addition to the calculated rollback assurance time and the base rollback assurance time, any one of a plurality of fault actions (Sop) based on non-corrected fault (Non-corrected Failure) 1302, SAE Level 1305, vehicle driving state 1307, or driving application 1304, in order to execute fault actions. This enables the stable operation of applications used for vehicle driving.

[0359] Figure 13 This is a diagram illustrating multiple back-off actions.

[0360] Referring to the attached diagram, the central processing unit 175 can be controlled to execute any one of a plurality of backoff actions when the backoff assurance time of the operation is less than the baseline backoff assurance time.

[0361] As shown in the figure, the multiple reversal actions at this time can include vehicle takeover request, blind braking S0, lane keeping braking S1, lane changing braking S2, stopping on the shoulder, or path-based parking, etc. This enables stable operation of applications used for vehicle driving.

[0362] On the other hand, the central processing unit 175 within the signal processing apparatus 170 of this embodiment controls the application to change the baseline backoff guarantee time for application operation failure based on the application's security level, and performs a backoff action or a failure action if the application operation is determined to have failed based on the baseline backoff guarantee time and the calculated backoff guarantee time. This enables the stable operation of applications for vehicle operation. In particular, it enables the stable operation of applications for vehicle operation based on the security level.

[0363] The preferred embodiments of the present invention have been illustrated and described above. However, the present invention is not limited to the specific embodiments described above. Various modifications can be made by those skilled in the art without departing from the spirit of the present invention as claimed in the claims. Such modifications should not be understood separately from the technical concept or prospect of the present invention.

Claims

1. A signal processing apparatus, wherein, have: At least one neural processor; as well as Central processing unit, which runs applications used for driving the vehicle; If the operation of the application is determined to be a failure, the central processing unit controls the operation to run a second application corresponding to the application on another central processing unit or other signal processing device; The central processing unit adjusts the baseline rollback assurance time for failed actions of the application based on the application's security level.

2. The signal processing apparatus according to claim 1, wherein, When the security level of the application is the first security level, the central processing unit sets the baseline rollback assurance time to the first time. When the security level of the application is a second security level that is higher than the first security level, the central processing unit sets the baseline rollback assurance time to a second time that is greater than the first time.

3. The signal processing apparatus according to claim 2, wherein, When the security level of the application is a third security level lower than the first security level, the central processing unit sets the baseline rollback assurance time to a third time that is less than the first time.

4. The signal processing apparatus according to claim 1, wherein, If the backoff assurance time of the operation is greater than or equal to the baseline backoff assurance time, the central processing unit controls the execution of a preset action.

5. The signal processing apparatus according to claim 1, wherein, If the rollback assurance time of the operation is less than the baseline rollback assurance time, the central processing unit controls the operation to run a second application corresponding to the application on another central processing unit or other signal processing device.

6. The signal processing apparatus according to claim 1, wherein, If the rollback assurance time is less than the baseline rollback assurance time, the central processing unit controls the execution of the rollback action.

7. The signal processing apparatus according to claim 1, wherein, If the rollback assurance time of the operation is less than the baseline rollback assurance time, the central processing unit controls the execution of any one of the plurality of rollback actions.

8. The signal processing apparatus according to claim 7, wherein, The plurality of reversal actions include vehicle takeover request, blind braking, lane keeping braking, lane changing braking, stopping on the shoulder, or path-based stopping.

9. The signal processing apparatus according to claim 1, wherein, The central processing unit controls the operation such that, when the rollback assurance time of the operation is greater than the baseline rollback assurance time, as the rollback assurance time of the operation decreases, the difference between the estimated baseline rollback assurance time and the baseline rollback assurance time becomes smaller, thereby restricting the operation of applications with higher security levels.

10. The signal processing apparatus according to claim 1, wherein, If the difference between the estimated baseline backoff assurance time and the baseline backoff assurance time is a first time, the central processing unit controls the operation to run the application corresponding to the first security level. If the difference between the estimated baseline rollback assurance time and the baseline rollback assurance time is greater than the first time, the central processing unit controls the operation to run an application corresponding to a second security level higher than the first security level.

11. The signal processing apparatus according to claim 1, wherein, The central processing unit adjusts the baseline rollback assurance time based on the vehicle's driving environment.

12. The signal processing apparatus according to claim 1, wherein, The central processing unit controls the calculation of the rollback assurance time based on at least one of the following: system temperature, performance based on the security level, system error logs, application latency, or system load.

13. The signal processing apparatus according to claim 1, wherein, The neural processor performs neural processing based on at least one of the system's temperature, performance based on the security level, the system's error logs, the application's latency, or the system's load, and calculates the rollback assurance time based on the neural processing.

14. The signal processing apparatus according to claim 13, wherein, The neural processor performs the neural processing by feeding back power data or temperature data from the results of the neural processing.

15. The signal processing apparatus according to claim 1, wherein, The central processing unit runs a management program and runs a plurality of virtual machines on the management program; The first virtual machine among the plurality of said virtual machines runs neural system services for the control of the neural processor.

16. The signal processing apparatus according to claim 15, wherein, The second virtual machine among the plurality of said virtual machines runs the vehicle driving assistance application; A third virtual machine among the plurality of virtual machines runs a driver monitoring system application or an augmented reality application.

17. The signal processing apparatus according to claim 15, wherein, If the application running on the second virtual machine among the plurality of virtual machines has a higher security level than the application running on the third virtual machine, the central processing unit controls the application running on the second virtual machine to have a longer baseline rollback assurance time than the application running on the third virtual machine.

18. A signal processing apparatus, wherein, It has a central processing unit that runs applications for driving the vehicle; The central processing unit adjusts the baseline fallback assurance time for failed actions of the application based on the application's security level. If the application's action is determined to have failed based on the baseline rollback assurance time and the calculated rollback assurance time, the central processing unit controls the execution of either a rollback action or a fault action.

19. A display device for a vehicle, wherein, include: At least one display; as well as The signal processing device outputs image signals to the display. The signal processing apparatus includes the signal processing apparatus according to claims 1 to 18.

Images