Signal processing device and vehicle display device comprising same

Abstract

A signal processing device and a vehicle display device comprising same, according to one embodiment of the present disclosure, comprise: at least one neural processor; and a central processor for controlling the neural processor, wherein the central processor divides an artificial intelligence model to be executed in the neural processor into a plurality of groups and controls execution or suspension of the respective groups. Therefore, the neural processor can be efficiently operated.

Description

Signal processing device and vehicle display device equipped with the same

[0001] The present disclosure relates to a server, and more specifically, to a signal processing device capable of efficiently operating a neural processor and a vehicle display device equipped with the same.

[0002] A vehicle is a device that moves the user in the desired direction. A typical example is an automobile.

[0003] Meanwhile, for the convenience of users of the vehicle, a vehicle signal processing device is installed inside the vehicle.

[0004] The signal processing unit inside the vehicle receives and processes sensor data from various internal sensor devices.

[0005] Meanwhile, due to Advanced Driver Assistance Systems (ADAS) or autonomous driving, the types and number of sensors installed in vehicles are increasing, leading to a trend of increasing data that needs to be processed.

[0006] Meanwhile, methods using neural networks for processing camera data, such as object extraction, are being researched.

[0007] The prior art Korean Patent No. 10-2463175 relates to an object recognition method and apparatus, wherein features are extracted from an input image in a neural network to generate a feature map, and in parallel with the generation of the feature map, a region of interest corresponding to at least one object of interest is extracted from the input image, and the number of object candidate regions used for detecting the object of interest is determined based on the size of the region of interest, and the object of interest is recognized from the region of interest based on the number of object candidate regions in the neural network.

[0008] However, according to prior art, since signal processing is performed on camera data based on a fixed frame, there is a disadvantage that signal processing is not efficient.

[0009] The problem that the present disclosure aims to solve is to provide a signal processing device capable of efficiently operating a neural processor and a vehicle display device equipped with the same.

[0010] Another problem that the present disclosure aims to solve is to provide a signal processing device capable of efficiently operating a plurality of neural processors and a vehicle display device equipped with the same.

[0011] A signal processing device and a vehicle display device equipped with the same according to one embodiment of the present disclosure comprises at least one neural processor and a central processor that controls the neural processor, and the central processor divides an artificial intelligence model executed in the neural processor into a plurality of groups and controls the execution or suspension of each of the plurality of groups.

[0012] Meanwhile, the central processor can determine the division of the artificial intelligence model or the number of divisions of the artificial intelligence model based on the size of the data input to the neural processor or the estimated execution time in the neural processor.

[0013] Meanwhile, the central processor can separate the first artificial intelligence model into a first number of groups when the first artificial intelligence model is executed in the neural processor, and separate the second artificial intelligence model into a second number of groups when the second artificial intelligence model is executed in the neural processor.

[0014] Meanwhile, the central processor can divide an artificial intelligence model for object detection running in a neural processor into a first artificial intelligence model for lane detection and a second artificial intelligence model for person detection.

[0015] Meanwhile, the neural processor can perform human detection by executing a second artificial intelligence model during lane detection based on a first artificial intelligence model.

[0016] Meanwhile, the central processor can control some of the multiple groups to be executed in the neural processor and control other parts of the multiple groups not to be executed in the neural processor.

[0017] Meanwhile, the central processor can divide the first artificial intelligence model by object within the camera data when the artificial intelligence model executed in the neural processor is a first artificial intelligence model for processing based on camera data.

[0018] Meanwhile, the central processor can separate a first artificial intelligence model running on a neural processor into multiple binaries and set the processing order of the separated multiple binaries based on priority or processing frequency.

[0019] Meanwhile, the central processor sets the processing order of multiple artificial intelligence models executed in the neural processor based on priority or processing frequency, and the neural processor can execute multiple artificial intelligence models according to the set processing order.

[0020] Meanwhile, if the first artificial intelligence model is scheduled to run for a first period in the first neural processor and the second artificial intelligence model is scheduled to run repeatedly for a period shorter than the first period in the second neural processor, the central processor can control the second artificial intelligence model to run repeatedly during the execution of the first artificial intelligence model in the first neural processor.

[0021] Meanwhile, the central processor can control the artificial intelligence model so that it is not executed on the second neural processor.

[0022] Meanwhile, the central processor can control the first artificial intelligence model for processing first data at a first frame rate in the neural processor to be converted to a second frame rate and executed repeatedly, and during the off period of the first artificial intelligence model, to execute a second artificial intelligence model for processing second data at a second frame rate.

[0023] Meanwhile, the central processor can control the execution of the first artificial intelligence model for processing the first data at the first frame rate in the neural processor, and when the execution of the second artificial intelligence model for processing the second data at the second frame rate is required, to execute the first artificial intelligence model for processing the first data and the second artificial intelligence model for processing the second data at the second frame rate.

[0024] Meanwhile, the central processor can control the frame rate for camera data processing by the neural processor to be reduced when parked.

[0025] Meanwhile, the central processor can control the processing of at least some of the camera data inside the vehicle in the neural processor to be turned off when there is no passenger in the front or rear seat of the vehicle.

[0026] Meanwhile, the central processor can perform multitasking-based signal processing, and the neural processor can perform single-tasking-based signal processing.

[0027] Meanwhile, the signal processing device further includes a graphics processor, and the central processor can control at least one of the graphics processor and the central processor to execute some of the plurality of groups in addition to the neural processor. Accordingly, the signal processing device can be operated efficiently.

[0028] Meanwhile, a signal processing device and a vehicle display device equipped with the same according to another embodiment of the present disclosure comprises at least one neural processor and a central processor that controls the neural processor, wherein the central processor sets a processing order for a plurality of artificial intelligence models executed in the neural processor based on priority or processing frequency, and the neural processor executes a plurality of artificial intelligence models according to the set processing order.

[0029] Meanwhile, a vehicle display device according to an embodiment of the present disclosure includes at least one neural processor, a signal processing device having a central processor that controls the neural processor, and at least one second neural processor, and a second signal processing device having a second central processor that controls the second neural processor.

[0030] A signal processing device and a vehicle display device equipped with the same according to one embodiment of the present disclosure comprise at least one neural processor and a central processor that controls the neural processor. The central processor divides an artificial intelligence model executed in the neural processor into a plurality of groups and controls the execution or suspension of each of the plurality of groups. Accordingly, the neural processor can be operated efficiently.

[0031] Meanwhile, the central processor can determine the division of the artificial intelligence model or the number of divisions of the artificial intelligence model based on the size of the data input to the neural processor or the estimated execution time in the neural processor. Accordingly, the neural processor can be operated efficiently.

[0032] Meanwhile, the central processor can separate the first artificial intelligence model into a first number of groups when the first artificial intelligence model is executed in the neural processor, and separate the second artificial intelligence model into a second number of groups when the second artificial intelligence model is executed in the neural processor. Accordingly, the neural processor can be operated efficiently.

[0033] Meanwhile, the central processor can divide the artificial intelligence model for object detection running on the neural processor into a first artificial intelligence model for lane detection and a second artificial intelligence model for human detection. Accordingly, the neural processor can be operated efficiently.

[0034] Meanwhile, the neural processor can perform human detection by executing a second artificial intelligence model during lane detection based on a first artificial intelligence model. Accordingly, the neural processor can be operated efficiently.

[0035] Meanwhile, the central processor can control some of the multiple groups to be executed on the neural processor, and control others of the multiple groups not to be executed on the neural processor. Accordingly, the neural processor can be operated efficiently.

[0036] Meanwhile, the central processor can divide the first artificial intelligence model by object within the camera data when the artificial intelligence model executed in the neural processor is a first artificial intelligence model for processing based on camera data. Accordingly, the neural processor can be operated efficiently.

[0037] Meanwhile, the central processor can separate a first artificial intelligence model running on a neural processor into multiple binaries and set the processing order of the separated multiple binaries based on priority or processing frequency. Accordingly, the neural processor can be operated efficiently.

[0038] Meanwhile, the central processor sets the processing order of multiple artificial intelligence models executed in the neural processor based on priority or processing frequency, and the neural processor can execute the multiple artificial intelligence models according to the set processing order. Accordingly, the neural processor can be operated efficiently.

[0039] Meanwhile, if the first artificial intelligence model is scheduled to run in the first neural processor for a first period and the second artificial intelligence model is scheduled to run repeatedly in the second neural processor for a period shorter than the first period, the central processor can control the second artificial intelligence model to run repeatedly in the first neural processor while the first artificial intelligence model is running. Accordingly, multiple neural processors can be operated efficiently.

[0040] Meanwhile, the central processor can control the second neural processor so that the artificial intelligence model is not executed. Accordingly, multiple neural processors can be operated efficiently.

[0041] Meanwhile, the central processor can control the neural processor to repeatedly execute a first artificial intelligence model for processing first data at a first frame rate by converting it to a second frame rate, and to execute a second artificial intelligence model for processing second data at a second frame rate during the off period of the first artificial intelligence model. Accordingly, the neural processor can be operated efficiently.

[0042] Meanwhile, the central processor can control the execution of a first artificial intelligence model for processing first data at a first frame rate in the neural processor, and when execution of a second artificial intelligence model for processing second data at a second frame rate is required, to execute the first artificial intelligence model for processing first data and the second artificial intelligence model for processing second data at the second frame rate. Accordingly, the neural processor can be operated efficiently.

[0043] Meanwhile, the central processor can control the frame rate for camera data processing by the neural processor to be reduced when parked. Accordingly, the neural processor can be operated efficiently.

[0044] Meanwhile, the central processor can control the processing of at least a portion of the camera data inside the vehicle in the neural processor to be turned off when there are no passengers in the front or rear seats of the vehicle. Accordingly, the neural processor can be operated efficiently.

[0045] Meanwhile, the central processor performs multitasking-based signal processing, and the neural processor can perform single-tasking-based signal processing. Accordingly, the neural processor can be operated efficiently.

[0046] Meanwhile, the signal processing device further includes a graphics processor, and the central processor can control at least one of the graphics processor and the central processor to execute some of the plurality of groups in addition to the neural processor. Accordingly, the signal processing device can be operated efficiently.

[0047] Meanwhile, a signal processing device and a vehicle display device equipped with the same according to another embodiment of the present disclosure comprises at least one neural processor and a central processor that controls the neural processor, wherein the central processor sets a processing order for a plurality of artificial intelligence models executed in the neural processor based on priority or processing frequency, and the neural processor executes the plurality of artificial intelligence models according to the set processing order. Accordingly, the neural processor can be operated efficiently.

[0048] Meanwhile, a vehicle display device according to an embodiment of the present disclosure includes at least one neural processor, a signal processing device having a central processor that controls the neural processor, and a second signal processing device having at least one second neural processor and a second central processor that controls the second neural processor. Accordingly, the neural processor or the second neural processor can be operated efficiently.

[0049] Figure 1 is a drawing illustrating a vehicle system including a vehicle and a server.

[0050] Figure 2 is a diagram illustrating the architecture of a vehicle signal processing system inside the vehicle of Figure 1.

[0051] FIG. 3a is a drawing illustrating an example of the arrangement of a vehicle display device inside a vehicle.

[0052] FIG. 3b is a drawing illustrating another example of the arrangement of a vehicle display device inside a vehicle.

[0053] Figure 4 is an example of an internal block diagram of the vehicle of Figure 1.

[0054] FIG. 5 is an example of a block diagram of a vehicle display device according to an embodiment of the present disclosure.

[0055] FIGS. 6a to 14b are drawings referenced in the operation description of FIG. 5.

[0056] FIG. 15 is an example of a block diagram of a vehicle display device according to another embodiment of the present disclosure.

[0057] The present disclosure will be described in more detail below with reference to the drawings.

[0058] The suffixes "module" and "part" for components used in the following description are assigned solely for the ease of drafting this specification and do not inherently confer any particularly significant meaning or role. Accordingly, the terms "module" and "part" may be used interchangeably.

[0059] Figure 1 is a drawing illustrating a vehicle system including a vehicle and a server.

[0060] Referring to the drawing, the vehicle system (10) includes a vehicle (200) and a server (900) that exchanges vehicle (200) data.

[0061] The vehicle (200) is operated by a plurality of wheels (103FR, 103FL, 103RL,...) that rotate by a power source, and a steering wheel (150) for controlling the direction of travel of the vehicle (200).

[0062] Meanwhile, the vehicle (200) may further be equipped with a camera (195), etc., for acquiring an image of the front of the vehicle.

[0063] Meanwhile, the vehicle (200) may be equipped with a plurality of displays (180a, 180b) for displaying images, information, etc. inside.

[0064] In FIG. 1, a cluster display (180a) and an AVN (Audio Video Navigation) display (180b) are exemplified as multiple displays (180a, 180b). Other displays such as a HUD (Head Up Display) are also possible.

[0065] Meanwhile, the AVN (Audio Video Navigation) display (180b) may also be named the Center Information Display.

[0066] Meanwhile, the vehicle (200) described in this specification may be a concept that includes all of the following: a vehicle equipped with an engine as a power source, a hybrid vehicle equipped with an engine and an electric motor as a power source, an electric vehicle equipped with an electric motor as a power source, etc.

[0067] Meanwhile, a server (900) according to one embodiment of the present disclosure receives event data and image data related to the event data from a vehicle, and calculates a driving score related to the event data based on the event data or the image data. Accordingly, the neural processor can be operated efficiently.

[0068] Various operations of the server (900) according to one embodiment of the present disclosure will be described later with reference to FIG. 6 and below.

[0069] Figure 2 is a diagram illustrating the architecture of a vehicle signal processing system inside the vehicle of Figure 1.

[0070] Referring to the drawing, the architecture (300a) of the vehicle signal processing system inside the vehicle (200) can correspond to a zone-based architecture.

[0071] Accordingly, sensor devices and processors inside the vehicle may be placed in each of the multiple zones (Z1 to Z4), and a signal processing device (170a) including a vehicle communication gateway (GWDa) may be placed in the central area of ​​the multiple zones (Z1 to Z4).

[0072] Meanwhile, the signal processing device (170a) may additionally include an autonomous driving control module (ACC), a cockpit control module (CPG), etc., in addition to the vehicle communication gateway (GWDa).

[0073] The vehicle communication gateway (GWDa) within the signal processing device (170a) may be a High Performance Computing (HPC) gateway.

[0074] That is, the signal processing device (170a) of FIG. 2 is an integrated HPC and can exchange data with an external communication module (not shown) or a processor (not shown) in a plurality of zones (Z1 to Z4).

[0075] FIG. 3a is a drawing illustrating an example of the arrangement of a vehicle display device inside a vehicle.

[0076] Referring to the drawing, the vehicle interior may be equipped with a cluster display (180a), an AVN (Audio Video Navigation) display (180b), a rear seat entertainment display (180c, 180d), a rearview mirror display (not shown), etc.

[0077] FIG. 3b is a drawing illustrating another example of the arrangement of a vehicle display device inside a vehicle.

[0078] A vehicle display device (100) according to an embodiment of the present disclosure may include a plurality of displays (180a to 180b) and a signal processing device (170) that performs signal processing for displaying images, information, etc. on the plurality of displays (180a to 180b) and outputs an image signal to at least one display (180a to 180b).

[0079] Among the plurality of displays (180a to 180b), the first display (180a) is a cluster display (180a) for displaying driving status, operation information, etc., and the second display (180b) may be an AVN (Audio Video Navigation) display (180b) for displaying vehicle operation information, navigation map, various entertainment information or video.

[0080] The signal processing device (170) has a processor (175) inside and can execute a first virtualization machine to a third virtualization machine (not shown) on a hypervisor (not shown) within the processor (175).

[0081] A second virtualization machine (not shown) operates for the first display (180a), and a third virtualization machine (not shown) can operate for the second display (180b).

[0082] Meanwhile, the first virtualization machine (not shown) within the processor (175) can be controlled to set up a shared memory (508) based on a hypervisor (505) for the same data transmission to the second virtualization machine (not shown) and the third virtualization machine (not shown). Accordingly, the same information or the same image can be synchronized and displayed on the first display (180a) and the second display (180b) within the vehicle.

[0083] Meanwhile, the first virtualization machine (not shown) within the processor (175) shares at least a portion of the data with the second virtualization machine (not shown) and the third virtualization machine (not shown) for data sharing processing. Accordingly, data can be shared and processed by multiple virtualization machines for multiple displays within the vehicle.

[0084] Meanwhile, the first virtualization machine (not shown) within the processor (175) can receive and process wheel speed sensor data of the vehicle and transmit the processed wheel speed sensor data to at least one of the second virtualization machine (not shown) or the third virtualization machine (not shown). Accordingly, the wheel speed sensor data of the vehicle can be shared with at least one virtualization machine, etc.

[0085] Meanwhile, the vehicle display device (100) according to the embodiment of the present disclosure may further include a rear seat entertainment display (180c) for displaying driving status information, simple navigation information, various entertainment information or images.

[0086] The signal processing device (170) can control the RSE display (180c) by running a fourth virtualization machine (not shown) in addition to the first to third virtualization machines (not shown) on a hypervisor (not shown) within the processor (175).

[0087] Accordingly, various displays (180a to 180c) can be controlled using a single signal processing device (170).

[0088] Meanwhile, some of the multiple displays (180a to 180c) operate under a Linux OS, and others can operate under a Web OS.

[0089] A signal processing device (170) according to an embodiment of the present disclosure can control displays (180a to 180c) operating under various operating systems (OS) to synchronize and display the same information or the same image.

[0090] Meanwhile, FIG. 3b illustrates that a vehicle speed indicator (212a) and a vehicle interior temperature indicator (213a) are displayed on a first display (180a), a home screen (222) including a plurality of applications, a vehicle speed indicator (212b), and a vehicle interior temperature indicator (213b) is displayed on a second display (180b), and a second home screen (222b) including a plurality of applications and a vehicle interior temperature indicator (213c) is displayed on a third display (180c).

[0091] Figure 4 is an example of an internal block diagram of the vehicle of Figure 1.

[0092] Referring to the drawings, a vehicle (200) according to an embodiment of the present disclosure may be equipped with a lamp drive unit (751), a steering drive unit (752), a brake drive unit (753), a power source drive unit (754), a suspension drive unit (756), an air conditioning drive unit (757), a window drive unit (758), a seat drive unit (761), and a signal processing device (170).

[0093] Meanwhile, the vehicle (200) may further be equipped with an ECU (770), a plurality of sensor devices (SN), and a plurality of communication modules (EMa~EMd).

[0094] Meanwhile, the vehicle (200) according to the embodiment of the present disclosure may further be equipped with a vehicle display device (100).

[0095] A vehicle display device (100) according to an embodiment of the present disclosure may include an input unit (110), a communication device (120) for communication with an external device, a plurality of communication modules (EMa~EMd) for internal communication, a memory (140), a signal processing device (170), a plurality of displays (180a~180c), an audio output unit (185), and a power supply unit (190).

[0096] Multiple communication modules (EMa~EMd) can be placed in each of the multiple zones (Z1~Z4) of FIG. 2, for example.

[0097] Meanwhile, the signal processing device (170) may have a communication switch (736b) inside for data communication with each communication module (EM1~EM4).

[0098] Each communication module (EM1~EM4) can perform data communication with a plurality of sensor devices (SN), ECU (770), or area signal processing device (170Z).

[0099] Meanwhile, a plurality of sensor devices (SN) may include a camera (195), lidar (196), radar (197), or position sensor (198).

[0100] The input unit (110) may be equipped with physical buttons, pads, etc. for button input, touch input, etc.

[0101] Meanwhile, the input unit (110) may be equipped with a microphone (not shown) for user voice input.

[0102] The communication device (120) can exchange data wirelessly with a mobile terminal (800) or a server (900).

[0103] In particular, the communication device (120) can wirelessly exchange data with the vehicle driver's mobile terminal. Various data communication methods are possible as wireless data communication methods, such as Bluetooth, WiFi, WiFi Direct, and APiX.

[0104] The communication device (120) can receive weather information, road traffic condition information, for example, TPEG (Transport Protocol Expert Group) information from a mobile terminal (800) or a server (900). To this end, the communication device (120) may be equipped with a mobile communication module (not shown).

[0105] Meanwhile, the communication device (120) can exchange data wirelessly with an adjacent vehicle.

[0106] For example, the communication device (120) can exchange vehicle messages wirelessly with an adjacent vehicle through V2X (Vehicle-to-everything) communication.

[0107] A plurality of communication modules (EM1~EM4) can receive sensor data, etc. from an ECU (770), a sensor device (SN), or a region signal processing device (170Z), and transmit the received sensor data to the signal processing device (170).

[0108] Here, the sensor data may include at least one of vehicle direction data, vehicle location data (GPS data), vehicle angle data, vehicle speed data, vehicle acceleration data, vehicle tilt data, vehicle forward / reverse data, battery data, fuel data, tire data, vehicle lamp data, vehicle interior temperature data, and vehicle interior humidity data.

[0109] Such sensor data can be obtained from a heading sensor, a yaw sensor, a gyro sensor, a position module, a vehicle forward / reverse sensor, a wheel sensor, a vehicle speed sensor, a vehicle body inclination sensor, a battery sensor, a fuel sensor, a tire sensor, a steering sensor based on steering wheel rotation, a vehicle interior temperature sensor, a vehicle interior humidity sensor, etc.

[0110] Meanwhile, the position module may include a GPS module or a position sensor (198) for receiving GPS information.

[0111] Meanwhile, at least one of the multiple communication modules (EM1 to EM4) can transmit location information data sensed from a GPS module or a location sensor (198) to a signal processing device (170).

[0112] Meanwhile, at least one of the plurality of communication modules (EM1 to EM4) can receive vehicle front image data, vehicle side image data, vehicle rear image data, and obstacle distance information around the vehicle from a camera (195), lidar (196), radar (197), etc., and transmit the received information to a signal processing device (170).

[0113] The memory (140) can store various data for the overall operation of the vehicle display device (100), such as a program for processing or controlling the signal processing device (170).

[0114] For example, memory (140) can store data regarding a hypervisor, a first virtualization machine to a third virtualization machine, for execution within a processor (175).

[0115] The audio output unit (185) converts an electrical signal from the signal processing device (170) into an audio signal and outputs it. To do this, a speaker or the like may be provided.

[0116] The power supply unit (190) can supply power necessary for the operation of each component under the control of the signal processing unit (170). In particular, the power supply unit (190) can receive power from a battery inside the vehicle, etc.

[0117] The signal processing device (170) controls the overall operation of each unit within the vehicle display device (100) or vehicle (200).

[0118] For example, the signal processing device (170) may include a processor (175) that performs signal processing for a vehicle display (180a, 180b).

[0119] The processor (175) can run a first virtualization machine to a third virtualization machine (not shown) on a hypervisor (not shown) within the processor (175).

[0120] Among the first to third virtual machines (not shown), the first virtual machine (not shown) may be named a Server Virtual Machine, and the second to third virtual machines (not shown) may be named a Guest Virtual Machine.

[0121] For example, a first virtualization machine (not shown) within a processor (175) can receive sensor data from a plurality of sensor devices, such as vehicle sensor data, location information data, camera image data, audio data, or touch input data, and process or modify it to output it.

[0122] In this way, by performing most of the data processing in the first virtualization machine (not shown), 1:N data sharing becomes possible.

[0123] As another example, the first virtualization machine (not shown) can directly receive and process CAN data, Ethernet data, audio data, radio data, USB data, and wireless communication data for the second virtualization machine to the third virtualization machine (not shown).

[0124] And, the first virtualization machine (not shown) can transmit the processed data to the second virtualization machine to the third virtualization machine (not shown).

[0125] Accordingly, among the first to third virtualization machines (not shown), only the first virtualization machine (not shown) receives sensor data, communication data, or external input data from a plurality of sensor devices and performs signal processing, thereby reducing the signal processing burden on other virtualization machines and enabling 1:N data communication, which enables synchronization when sharing data.

[0126] Meanwhile, the first virtualization machine (not shown) can control the sharing of the same data with the second virtualization machine (not shown) and the third virtualization machine (not shown) by writing data to the shared memory (508).

[0127] For example, the first virtualization machine (not shown) can record vehicle sensor data, the location information data, the camera image data, or the touch input data in a shared memory (508) and control the sharing of the same data with the second virtualization machine (not shown) and the third virtualization machine (not shown). Accordingly, data sharing in a 1:N manner becomes possible.

[0128] Ultimately, by performing most of the data processing on the first virtualization machine (not shown), 1:N data sharing becomes possible.

[0129] Meanwhile, the first virtualization machine (not shown) within the processor (175) can control the second virtualization machine (not shown) and the third virtualization machine (not shown) to set up a shared memory (508) based on the hypervisor (505) for the same data transmission.

[0130] Meanwhile, the signal processing device (170) can process various signals such as audio signals, video signals, and data signals. To this end, the signal processing device (170) can be implemented in the form of a System On Chip (SOC).

[0131] FIG. 5 is an example of a block diagram of a vehicle display device according to an embodiment of the present disclosure.

[0132] Referring to the drawings, a vehicle display device (900) according to an embodiment of the present disclosure comprises a signal processing device (170) having at least one neural processor (179) and a central processor (175) that controls the neural processor (179).

[0133] Meanwhile, the vehicle display device (900) according to the embodiment of the present disclosure may further include at least one display.

[0134] Meanwhile, the vehicle display device (900) according to the embodiment of the present disclosure may further include the steering drive unit (752), brake drive unit (753), power source drive unit (754), ECU (770), or a plurality of sensor devices (SN), etc. of FIG. 4.

[0135] Meanwhile, the vehicle display device (900) according to the embodiment of the present disclosure may further include the lamp driving unit (751), suspension driving unit (756), air conditioning driving unit (757), window driving unit (758), seat driving unit (761), or a plurality of communication modules (EMa~EMd), etc. of FIG. 4.

[0136] In the drawing, at least one display is exemplified as a cluster display (180a) and an AVN display (180b).

[0137] Meanwhile, the vehicle display device (900) may further include a plurality of area signal processing devices (170Z1 to 170Z4).

[0138] The signal processing device (170) at this time is a high-performance centralized signal processing and control device having a plurality of CPUs (175), GPUs (178), NPUs (179), etc., and can be named as a High Performance Computing (HPC) signal processing device or a central signal processing device.

[0139] Multiple area signal processing devices (170Z1~170Z4) and signal processing device (170) are connected by wired cables (CB1~CB4).

[0140] Meanwhile, multiple area signal processing devices (170Z1~170Z4) can be connected to each other by wired cables (CBa~CBd).

[0141] The wired cable (CBa~CBd) at this time may include a CAN communication cable, an Ethernet communication cable, or a PCI Express cable.

[0142] Meanwhile, the signal processing device (170) according to the embodiment of the present disclosure may further include a large-capacity storage device (925).

[0143] Meanwhile, the signal processing device (170) according to an embodiment of the present disclosure may further include a graphics processor (178).

[0144] Meanwhile, the signal processing device (170) according to an embodiment of the present disclosure may have at least one central processor (175, 178, 177).

[0145] Meanwhile, sensor data can be transmitted from at least one of the multiple area signal processing devices (170Z1 to 170Z4) to the signal processing device (170). In particular, the sensor data can be stored in a storage device (925) within the signal processing device (170).

[0146] The sensor data at this time may include at least one of camera data, lidar data, radar data, vehicle direction data, vehicle position data (GPS data), vehicle angle data, vehicle speed data, vehicle acceleration data, vehicle tilt data, vehicle forward / reverse data, battery data, fuel data, tire data, vehicle lamp data, vehicle interior temperature data, and vehicle interior humidity data.

[0147] In the drawing, camera data from a camera (195a) and lidar data from a lidar sensor (196) are input to a first area signal processing device (170Z1), and the camera data and lidar data are transmitted to a signal processing device (170) via a second area signal processing device (170Z2) and a third area signal processing device (170Z3), etc.

[0148] Meanwhile, since the data reading or writing speed to the storage device (925) is faster than the network speed when sensor data is transmitted from at least one of the multiple area signal processing devices (170Z1~170Z4) to the signal processing device (170), it is desirable to perform multipath routing so that network bottlenecks do not occur.

[0149] To this end, the signal processing device (170) according to an embodiment of the present disclosure can perform multipath routing based on a Software Defined Network (SDN). Accordingly, a stable network environment can be secured when reading or writing data of the storage device (925). Furthermore, since data can be transmitted to the storage device (925) using multiple paths, data can be transmitted by dynamically changing the network configuration.

[0150] Data communication between a plurality of area signal processing devices (170Z1~170Z4) and a signal processing device (170) within a vehicle display device (900) according to an embodiment of the present disclosure is preferably Peripheral Component Interconnect Express communication for high-bandwidth, low-latency communication.

[0151] Meanwhile, the signal processing device (170) according to an embodiment of the present disclosure can receive an internal image from an internal camera (195i) and perform signal processing on the internal image.

[0152] Meanwhile, the signal processing device (170) according to the embodiment of the present disclosure can receive a front image from a front camera (195a) and perform signal processing on the front image.

[0153] FIGS. 6a to 14b are drawings referenced in the operation description of FIG. 5.

[0154] FIG. 6a is a drawing referenced in the description of the operation of a signal processing device related to the present disclosure.

[0155] Referring to the drawings, the signal processing device (170x) related to the present disclosure may include a central processor (175), a neural processor (179), and a memory (925).

[0156] The neural processor (179) can execute an artificial intelligence model (610) for neural network processing.

[0157] Meanwhile, the neural processor (179) in the signal processing device (170x) related to the present disclosure can execute an artificial intelligence model (610) and output result data.

[0158] At this time, the neural processor (179) in the signal processing device (170x) related to the present disclosure outputs result data only after the execution of the artificial intelligence model (610) is completed, so a considerable amount of time may be required when outputting result data.

[0159] Meanwhile, since the neural processor (179) operates on a single-tasking basis, when the artificial intelligence model (610) is executed, all resources are allocated to the execution of the artificial intelligence model (610), so it may be inefficient.

[0160] Accordingly, the present disclosure proposes a method for efficiently operating a neural processor (179). This is described with reference to FIG. 6b.

[0161] FIG. 6b is a drawing referenced in the description of the operation of a signal processing device according to an embodiment of the present disclosure.

[0162] Referring to the drawings, a signal processing device (170) according to one embodiment of the present disclosure comprises at least one neural processor (179) and a central processor (175) that controls the neural processor (179).

[0163] Meanwhile, a signal processing device (170) according to one embodiment of the present disclosure may further include a memory (925) for storing data for the execution of an artificial intelligence model.

[0164] Meanwhile, a central processor (175) according to one embodiment of the present disclosure divides an artificial intelligence model (620) executed in a neural processor (179) into a plurality of groups (PTa, PTb, PTc) and controls the execution or suspension of each of the plurality of groups (PTa, PTb, PTc). Accordingly, the neural processor (179) can be operated efficiently.

[0165] For example, a central processor (175) according to one embodiment of the present disclosure can divide an artificial intelligence model (620) running on a neural processor (179) into a plurality of groups (PTa, PTb, PTc) with the same latency.

[0166] For example, a central processor (175) according to one embodiment of the present disclosure divides an artificial intelligence model (620) executed in a neural processor (179) into a plurality of binary data (PTa, PTb, PTc) and controls the execution or suspension of each of the plurality of binary data (PTa, PTb, PTc). Accordingly, the neural processor (179) can be operated efficiently.

[0167] For example, the neural processor (179) can execute the second group (PTb) during the execution of the first group (PTa) divided by the central processor (175).

[0168] Accordingly, the output period of the result data of the artificial intelligence model (620) can be shortened by the amount of the overlap between the execution period of the first group (PTa) and the execution period of the second group (PTb).

[0169] As another example, the neural processor (179) can stop the execution of the second group (PTb) and execute the third group (PTc) during the execution of the first group (PTa) divided by the central processor (175). Accordingly, the neural processor (179) can be operated efficiently.

[0170] Meanwhile, the central processor (175) can determine the division of the artificial intelligence model (620) or the number of divisions of the artificial intelligence model (620) based on the size of the data input to the neural processor (179) or the expected execution time in the neural processor (179).

[0171] In the drawing, the number of divisions of the artificial intelligence model (620) is exemplified as 3, but various variations are possible.

[0172] For example, the central processor (175) can control the number of divisions of the artificial intelligence model (620) to increase as the size of the data input to the neural processor (179) increases or as the estimated execution time in the neural processor (179) increases. Accordingly, the neural processor (179) can be operated efficiently.

[0173] Meanwhile, the artificial intelligence model (620) is a learning-based signal processing model and can be various models such as DNN or CNN-based Deep Learning.

[0174] Meanwhile, the artificial intelligence model (620) may be an artificial intelligence model for object detection based on camera data or an artificial intelligence model for person detection based on camera data.

[0175] Meanwhile, the artificial intelligence model (620) may be an artificial intelligence model for natural language recognition based on voice data or a large language model (LLM) based on various sensor data.

[0176] Meanwhile, the central processor (175) can perform multitasking-based signal processing, and the neural processor (179) can perform single-tasking-based signal processing.

[0177] Considering the characteristics of the neural processor (179), as described above, the artificial intelligence model (620) is divided into multiple groups (PTa, PTb, PTc), and by controlling the execution or suspension of each of the multiple groups (PTa, PTb, PTc), the neural processor (179) can be operated efficiently.

[0178] Meanwhile, the central processor (175) can divide the artificial intelligence model (620) for object detection running in the neural processor (179) into a first artificial intelligence model (PTa) for lane detection and a second artificial intelligence model (PTb) for person detection. Accordingly, the neural processor (179) can be operated efficiently.

[0179] Meanwhile, the neural processor (179) can perform human detection by executing the second artificial intelligence model (PTb) during lane detection based on the first artificial intelligence model (PTa). Accordingly, the neural processor (179) can be operated efficiently.

[0180] Meanwhile, the central processor (175) can control some of the multiple groups (PTa, PTb, PTc) to be executed in the neural processor (179) and control other parts of the multiple groups (PTa, PTb, PTc) not to be executed in the neural processor (179). Accordingly, the neural processor (179) can be operated efficiently.

[0181] Meanwhile, the central processor (175) can divide the first artificial intelligence model (PTa) for camera data-based processing into objects within the camera data when the artificial intelligence model (620) executed in the neural processor (179) is the first artificial intelligence model (PTa). Accordingly, the neural processor (179) can be operated efficiently.

[0182] Meanwhile, the central processor (175) can separate the first artificial intelligence model (PTa) running on the neural processor (179) into multiple binaries and set the processing order of the separated multiple binaries based on priority or processing frequency. Accordingly, the neural processor (179) can be operated efficiently.

[0183] Meanwhile, a central processor (175) according to another embodiment of the present disclosure sets a processing order for a plurality of artificial intelligence models (PTa to PTc) executed in a neural processor (179) based on priority or processing frequency, and the neural processor (179) executes the plurality of artificial intelligence models (PTa to PTc) according to the set processing order. Accordingly, the neural processor (179) can be operated efficiently.

[0184] Meanwhile, the signal processing device (170) may further include a graphics processor (178). Meanwhile, the central processor (175) may control at least one of the graphics processor (178) and the central processor (175), in addition to the neural processor (179), to execute some of the plurality of groups (PTa, PTb, PTc). Accordingly, the signal processing device (170) can be operated efficiently.

[0185] For example, the central processor (175) can control at least one of the neural processor (179), the graphics processor (178), and the central processor (175) to execute some of the plurality of groups (PTa, PTb, PTc) based on the utilization rate or resources of the neural processor (179), the graphics processor (178), and the central processor (175). Accordingly, the signal processing device (170) can be operated efficiently.

[0186] FIG. 7 is an example of an internal block diagram of a signal processing device according to an embodiment of the present disclosure.

[0187] Referring to the drawings, a signal processing device (170) according to one embodiment of the present disclosure may have at least one neural processor (179), a central processor (175) that controls the neural processor (179), and a memory (925).

[0188] Meanwhile, the central processor (175) may include a schedule table maker (720) for managing the schedule of the artificial intelligence model (725) and a partitioner (710) for partitioning the artificial intelligence model (725).

[0189] Meanwhile, the central processor (175) may further include an abstraction layer executer (730) that executes an abstraction layer by receiving schedule table information or schedule management information from a schedule table maker (720), and a model executer (735) that executes an artificial intelligence model (725).

[0190] Figure 8 is a drawing referenced in the description of Figure 7.

[0191] Referring to the drawings, a splitter (710) in a central processor (175) in a signal processing device (170) according to one embodiment of the present disclosure can receive an artificial intelligence model from a server (1900) (S810).

[0192] Next, the splitter (710) within the central processor (175) can split the artificial intelligence model into multiple groups.

[0193] And, the divider (710) in the central processor (175) can transmit the first group information among the divided group information to the schedule table maker (720) (S812).

[0194] Next, the schedule table maker (720) in the central processor (175) can generate first schedule table information or first schedule management information based on the first group information received.

[0195] And, the schedule table maker (720) in the central processor (175) can transmit the first schedule table information or the first schedule management information to the abstraction layer executor (730) (S814).

[0196] Next, the abstraction layer executor (730) in the central processor (175) can execute an abstraction layer related to an artificial intelligence model based on the first schedule table information or the first schedule management information (S820).

[0197] And, the model implementer (735) can execute the artificial intelligence model of the first group.

[0198] Next, the abstraction layer executor (730) or model executor (735) can transmit result completion data or feedback data of the first group of artificial intelligence models to the splitter (710) (S822).

[0199] Next, the splitter (710) in the central processor (175) can transmit the second group information among the split group information to the schedule table maker (720) (S824).

[0200] Next, the schedule table maker (720) in the central processor (175) can generate second schedule table information or second schedule management information based on the second group information received.

[0201] And, the schedule table maker (720) in the central processor (175) can transmit the second schedule table information or the second schedule management information to the abstraction layer executor (730) (S826).

[0202] Next, the abstraction layer executor (730) in the central processor (175) can execute an abstraction layer related to an artificial intelligence model based on the second schedule table information or the second schedule management information (S830).

[0203] And, the model executor (735) can execute the second group of artificial intelligence models. Accordingly, the neural processor (179) can be operated efficiently.

[0204] FIG. 9 illustrates an example of the operation of multiple neural processors.

[0205] Referring to the drawing, FIG. 9(a) illustrates that a first artificial intelligence model (TKa) is executed in a first neural processor (NPU0) and a second artificial intelligence model (TKb) is executed in a second neural processor (NPU1).

[0206] Meanwhile, the central processor (175) can separate the first artificial intelligence model (TKa) into a first number of groups (TKa1 to TKa4) as shown in (b) of FIG. 9 when the first artificial intelligence model (TKa) is executed in the first neural processor (NPU0).

[0207] Meanwhile, the central processor (175) can separate the second artificial intelligence model (TKb) into a second number of groups (TKb1 to TKb4) when the second artificial intelligence model (TKb) is executed in the second neural processor (NPU1), as in FIG. 9 (a) or FIG. 9 (b).

[0208] Meanwhile, the central processor (175) can control the second artificial intelligence model (TKb) to be executed repeatedly during the execution of the first artificial intelligence model (TKa) in the first neural processor (NPU0) when the first artificial intelligence model (TKa) is scheduled to be executed during the first period (T0 to Tf) and the second artificial intelligence model (TKb) is scheduled to be executed repeatedly during the period (T0 to T1) shorter than the first period in the second neural processor (NPU1).

[0209] That is, as shown in FIG. 9 (b), the first neural processor (NPU0) can execute the second artificial intelligence model (TKb1) during the period T0 to T1, execute the first artificial intelligence model (TKa1) during the period T1 to T2, execute the second artificial intelligence model (TKb2) during the period T2 to T3, execute the first artificial intelligence model (TKa2) during the period T3 to T4, execute the second artificial intelligence model (TKb3) during the period T4 to T5, execute the first artificial intelligence model (TKa3) during the period T5 to T6, execute the second artificial intelligence model (TKb4) during the period T6 to T7, and execute the first artificial intelligence model (TKa4) during the period T7 to T8. Accordingly, multiple neural processors (NPU0, NPU1) can be operated efficiently.

[0210] Meanwhile, the central processor (175) can control the artificial intelligence model (620) so that it is not executed in the second neural processor (NPU1), as shown in FIG. 9 (b). Accordingly, multiple neural processors (NPU0, NPU1) can be operated efficiently.

[0211] FIG. 10a illustrates another example of the operation of multiple neural processors.

[0212] Referring to the drawing, the central processor (175) can control three artificial intelligence models to be executed repeatedly in the first neural processor (NPU0) and four artificial intelligence models to be executed repeatedly in the second neural processor (NPU1) based on the first schedule information (1010).

[0213] Meanwhile, the first neural processor (NPU0) and the second neural processor (NPU1) can be placed separately within the neural processor (179) of FIG. 6a.

[0214] In the drawing, the neural processor (179) may mean including a first neural processor (NPU0) and a second neural processor (NPU1).

[0215] Meanwhile, the central processor (175) according to an embodiment of the present disclosure can set the processing order of a plurality of artificial intelligence models based on priority or processing frequency.

[0216] For example, a central processor (175) according to an embodiment of the present disclosure can control the artificial intelligence model (TKd) with a lower processing frequency among the artificial intelligence model (TKc) with a higher processing frequency and the artificial intelligence model (TKd) with a lower processing frequency, based on the second schedule information (1020), to be executed later in the first neural processor (NPU0) by transferring it from the second neural processor (NPU1) to the first neural processor (NPU0). Accordingly, a plurality of neural processors (NPU0, NPU1) can be operated efficiently.

[0217] FIG. 10b illustrates another example of the operation of multiple neural processors.

[0218] Referring to the drawing, the central processor (175) can control the execution of a plurality of artificial intelligence models in the first neural processor (NPU0) and the execution of a plurality of artificial intelligence models in the second neural processor (NPU1) based on the first schedule information (1010).

[0219] Meanwhile, the central processor (175) can set the processing order of the multiple artificial intelligence models based on priority or processing frequency when the execution of a new artificial intelligence model (TKe) is required while the multiple artificial intelligence models are being executed repeatedly in the first neural processor (NPU0) and the multiple artificial intelligence models are being executed repeatedly in the second neural processor (NPU1).

[0220] In response to this, the schedule table maker (720) in the central processor (175) can update the schedule table information or the schedule management information.

[0221] Meanwhile, the second neural processor (NPU1) can first execute a part of the new artificial intelligence model (TKe) based on the updated second schedule information (1030), and the first neural processor (NPU0) can execute another part of the new artificial intelligence model (TKe). Accordingly, multiple neural processors (NPU0, NPU1) can be operated efficiently. Furthermore, the new artificial intelligence model can be executed efficiently.

[0222] FIG. 11 is an example of the operation of a signal processing device according to an embodiment of the present disclosure.

[0223] Referring to the drawing, the central processor (175) within the signal processing device (170) may be equipped with a dispatcher (740) that distributes a plurality of artificial intelligence models to a plurality of neural processors (NPU0, NPU1) based on schedule information.

[0224] Meanwhile, the central processor (175) in the signal processing device (170) can control the execution of multiple artificial intelligence models in the first neural processor (NPU0) and the execution of multiple artificial intelligence models in the second neural processor (NPU1) based on the first schedule information (1110).

[0225] Accordingly, the first neural processor (NPU0) and the second neural processor (NPU1) in the signal processing device (170) can execute each artificial intelligence model and output execution result data.

[0226] Meanwhile, a plurality of artificial intelligence models executed in the first neural processor (NPU0) and the second neural processor (NPU1), respectively, within the signal processing device (170) may include an artificial intelligence model based on camera data inside the vehicle, an artificial intelligence model based on camera data outside the vehicle, etc.

[0227] Meanwhile, the central processor (175) within the signal processing device (170) receives execution result data from each first neural processor (NPU0) and second neural processor (NPU1), and can control the display (180) to display an image based on the execution result data.

[0228] For example, the central processor (175) within the signal processing unit (170) can control the display (180) to display an image (1120) including an image of the vehicle interior (1122), an image of the vehicle interior (1124), an image of the vehicle front (1126), and schedule information for the neural processor (179).

[0229] Figure 12a is a diagram illustrating the additional implementation of a new artificial intelligence model.

[0230] Referring to the drawing, the central processor (175) can control the first artificial intelligence model (1212) to be executed repeatedly in the first neural processor (NPU0) and the second artificial intelligence model (1214) to be executed repeatedly in the second neural processor (NPU1), based on the first frame rate as in (a) of FIG. 12a.

[0231] Meanwhile, the central processor (175) can set the processing order of multiple artificial intelligence models based on priority or processing frequency when the execution of a new third artificial intelligence model is required while the first artificial intelligence model (1212) is being executed repeatedly in the first neural processor (NPU0) and the second artificial intelligence model (1214) is being executed repeatedly in the second neural processor (NPU1).

[0232] That is, the central processor (175) can control the execution of a plurality of artificial intelligence models (1221, 1223, 1224) including a novel third artificial intelligence model in the first neural processor (NPU0) and a plurality of artificial intelligence models (1226, 1228, 1229) including a novel third artificial intelligence model in the second neural processor (NPU1) based on the first frame rate as in (b) of FIG. 12a.

[0233] Accordingly, it becomes possible to efficiently execute multiple artificial intelligence models, including new artificial intelligence models.

[0234] Meanwhile, a plurality of artificial intelligence models (1221, 1223, 1224) or a plurality of artificial intelligence models (1226, 1228, 1229) may be at least part of the first to third artificial intelligence models.

[0235] FIG. 12b illustrates an image corresponding to the execution of FIG. 12a.

[0236] Referring to the drawing, the central processor (175) within the signal processing device (170) receives execution result data from each first neural processor (NPU0) and second neural processor (NPU1), and can control the display (180) to display an image based on the execution result data.

[0237] For example, the central processor (175) within the signal processing unit (170) can control the display (180) to display an image (12323) of the vehicle interior, an image (1234) of the vehicle interior, and an image (1238) containing schedule information for the neural processor (179).

[0238] At this time, the image (1238) containing schedule information for the neural processor (179) can correspond to the schedule information containing the novel third artificial intelligence model of (b) of FIG. 12a.

[0239] FIG. 12c is a diagram illustrating the execution of an artificial intelligence model operating at different frame rates.

[0240] Referring to the drawings, the central processor (175) can control the execution of a plurality of artificial intelligence models (1221, 1223, 1224) in the first neural processor (NPU0) and the execution of a plurality of artificial intelligence models (1226, 1228, 1229) in the second neural processor (NPU1) based on a first frame rate (e.g., 30 FPS), as in (a) of FIG. 12c.

[0241] Meanwhile, the central processor (175) determines whether execution of the artificial intelligence model (1262) is required based on a second frame rate (e.g., 25 FPS), as in (b) of FIG. 12c.

[0242] Meanwhile, the multiple artificial intelligence models of (a) in FIG. 12c are artificial intelligence models based on an internal vehicle camera, and the artificial intelligence model (1262) of (b) in FIG. 12c may be an artificial intelligence model based on an external vehicle camera.

[0243] Meanwhile, the central processor (175) can control the execution of the first artificial intelligence model for processing the first data at the first frame rate (e.g., 30 FPS) in the second neural processor (NPU1), and when the execution of the second artificial intelligence model for processing the second data at the second frame rate (e.g., 25 FPS) is required, the first artificial intelligence model for processing the first data and the second artificial intelligence model for processing the second data can be executed at the second frame rate.

[0244] In particular, the central processor (175) can control the second neural processor (NPU1) to repeatedly execute a first artificial intelligence model for processing first data at a first frame rate (e.g., 30 FPS) at a second frame rate (e.g., 25 FPS), as shown in (c) of FIG. 12c, and to execute a second artificial intelligence model (1262) for processing second data at a second frame rate during the off period of the first artificial intelligence model. Accordingly, the neural processor can be operated efficiently.

[0245] Meanwhile, while operating at the second frame rate of the second neural processor (NPU1), the first neural processor (NPU0) can repeatedly execute a plurality of artificial intelligence models (1221, 1223, 1224) based on the first frame rate (e.g., 30 FPS).

[0246] In other words, multiple neural processors can operate at different frame rates. Accordingly, multiple neural processors can be operated efficiently.

[0247] FIG. 12d illustrates an image corresponding to the execution of FIG. 12c.

[0248] Referring to the drawing, the central processor (175) within the signal processing device (170) receives execution result data from each first neural processor (NPU0) and second neural processor (NPU1), and can control the display (180) to display an image based on the execution result data.

[0249] For example, the central processor (175) within the signal processing unit (170) can control the display (180) to display an image (1288) including an image (1282) inside the vehicle, an image (1264) inside the vehicle, an image (1286) outside the vehicle, and schedule information for the neural processor (179).

[0250] At this time, the image (1288) containing schedule information for the neural processor (179) may correspond to schedule information including a plurality of artificial intelligence models operating at different frame rates of (c) of FIG. 12c.

[0251] FIG. 13a is a diagram illustrating frame rate variation according to the division of an artificial intelligence model.

[0252] Referring to the drawing, the central processor (175) can control the execution of a plurality of artificial intelligence models in a first neural processor (NPU0) and a plurality of artificial intelligence models in a second neural processor (NPU1) based on a first frame rate (e.g., 25 FPS) according to the first schedule information (1270), as in (a) of FIG. 13a.

[0253] Meanwhile, the central processor (175) can control the frame rate to increase when splitting of any one of the multiple artificial intelligence models running in the first neural processor (NPU0) is required.

[0254] That is, the central processor (175) can control the execution of a plurality of artificial intelligence models, each including a divided artificial intelligence model (1321), in the first neural processor (NPU0) and in the second neural processor (NPU1), based on a second frame rate (e.g., 30 FPS) greater than the first frame rate (e.g., 25 FPS), according to the second schedule information (1320) as in (a) of FIG. 13a. Accordingly, the divided artificial intelligence model can be executed efficiently.

[0255] FIG. 13b is a diagram illustrating frame rate variation according to the division and movement of an artificial intelligence model.

[0256] Referring to the drawings, the central processor (175) can control the execution of a plurality of artificial intelligence models in a first neural processor (NPU0) and a plurality of artificial intelligence models in a second neural processor (NPU1) based on a first frame rate (e.g., 25 FPS) according to the first schedule information (1270), as in (a) of FIG. 13b.

[0257] Meanwhile, the central processor (175) can control the split model to be moved and executed without changing the frame rate when splitting and moving any one of the multiple artificial intelligence models running in the first neural processor (NPU0) is required.

[0258] That is, the central processor (175) can control that, according to the third schedule information (1330) as in (a) of FIG. 13b, a plurality of artificial intelligence models are each executed repeatedly in the first neural processor (NPU0) based on the first frame rate (e.g., 25 FPS), and a plurality of artificial intelligence models including a split and repositioned artificial intelligence model (1331) are each executed repeatedly in the second neural processor (NPU1). Accordingly, the split artificial intelligence model can be executed efficiently.

[0259] FIG. 13c illustrates an image corresponding to the execution of FIG. 13a.

[0260] Referring to the drawing, the central processor (175) within the signal processing device (170) receives execution result data from each first neural processor (NPU0) and second neural processor (NPU1), and can control the display (180) to display an image based on the execution result data.

[0261] For example, the central processor (175) within the signal processing unit (170) can control the display (180) to display an image (1358) containing schedule information for an image inside the vehicle (1352), an image inside the vehicle (1354), an image outside the vehicle (1356), and a neural processor (179).

[0262] At this time, the image (1358) containing schedule information for the neural processor (179) can correspond to schedule information containing a plurality of artificial intelligence models with varying frame rates of (b) of FIG. 13a.

[0263] FIG. 14a illustrates an example of the operation of a neural processor when parked.

[0264] Referring to the drawing, the central processor (175) can control the frame rate for camera data processing of the neural processor (179) to be reduced when parked. Accordingly, the neural processor (179) can be operated efficiently.

[0265] For example, the central processor (175) can control the execution of multiple artificial intelligence models in the first neural processor (NPU0) and the second neural processor (NPU1) based on a first frame rate (e.g., 25 FPS) while driving, as in (a) of FIG. 13b, and when parked, control the execution of one artificial intelligence model (1412) in the first neural processor (NPU0) and multiple artificial intelligence models (1413, 1416) in the second neural processor (NPU1) based on a third frame rate (e.g., 5 FPS), as in (a) of FIG. 14a.

[0266] Accordingly, multiple neural processors can be efficiently operated in response to vehicle movements.

[0267] Meanwhile, the central processor (175) within the signal processing device (170) receives execution result data from each first neural processor (NPU0) and second neural processor (NPU1), and can control the display (180) to display an image based on the execution result data.

[0268] For example, the central processor (175) within the signal processing unit (170) can control the display (180) to display an image (1422) of the vehicle interior when the vehicle is parked, an image (1424) of the vehicle interior, an image (1426) of the vehicle exterior, and an image (1428) containing schedule information for the neural processor (179), as in (a) of FIG. 14a.

[0269] At this time, the image (1428) containing schedule information for the neural processor (179) can correspond to the schedule information (1410) containing a plurality of artificial intelligence models of (a) of FIG. 14a.

[0270] FIG. 14b illustrates an example of the operation of a neural processor when there is no passenger in the passenger seat or rear seat of the vehicle.

[0271] Referring to the drawing, the central processor (175) can control the processing of at least some of the camera data inside the vehicle in the neural processor (179) to be turned off when there is no passenger in the passenger seat or rear seat inside the vehicle. Accordingly, the neural processor (179) can be operated efficiently.

[0272] For example, the central processor (175) can control multiple artificial intelligence models to be executed repeatedly in the first neural processor (NPU0) and the second neural processor (NPU1), respectively, as in (a) of FIG. 13b when there are passengers other than the driver inside the vehicle.

[0273] As another example, the central processor (175) can control the execution of multiple artificial intelligence models (1441–1443) in the first neural processor (NPU0) and one artificial intelligence model (1446) in the second neural processor (NPU1) when there is no passenger in the passenger seat or rear seat in the vehicle, as in (a) of FIG. 14b.

[0274] That is, the central processor (175) can control the processing of at least some of the camera data inside the vehicle in the second neural processor (NPU1) to be turned off. Accordingly, multiple neural processors can be operated efficiently in response to vehicle operation.

[0275] Meanwhile, the central processor (175) within the signal processing device (170) receives execution result data from each first neural processor (NPU0) and second neural processor (NPU1), and can control the display (180) to display an image based on the execution result data.

[0276] For example, the central processor (175) within the signal processing unit (170) can control the display (180) to display an image (1452) of the vehicle interior when there is no passenger in the passenger seat or rear seat of the vehicle as in (a) of FIG. 14b, an image (1456) of the vehicle exterior, and an image (1458) containing schedule information for the neural processor (179).

[0277] At this time, the image (1458) containing schedule information for the neural processor (179) may correspond to the schedule information (1440) containing a plurality of artificial intelligence models of (a) of FIG. 14b.

[0278] FIG. 15 is an example of a block diagram of a vehicle display device according to another embodiment of the present disclosure.

[0279] Referring to the drawings, a vehicle display device (1500) according to another embodiment of the present disclosure may have a first signal processing device (170a) and a second signal processing device (170b).

[0280] The first signal processing device (170a) may include at least one first neural processor (179a), a first central processor (175a) that controls the first neural processor (179a), and a memory (925a).

[0281] The second signal processing device (170b) may include at least one second neural processor (179b), a second central processor (175b) that controls the second neural processor (179b), and a second memory (925b).

[0282] Meanwhile, the first central processor (175a) and the second central processor (175b) may each be equipped with a dispatcher (740a, 740b) for artificial intelligence division, etc.

[0283] Meanwhile, the first central processor (175a) or the second central processor (175b) can operate like the central processor (175) described in FIGS. 6b to FIGS. 14b.

[0284] For example, the first central processor (175a) or the second central processor (175b) can control the execution of multiple artificial intelligence models in the first neural processor (179a) and the second neural processor (179b), respectively, based on schedule information (1110). Accordingly, the artificial intelligence models can be executed efficiently in multiple signal processing devices. In particular, each neural processor within each signal processing device can be operated efficiently.

[0285] Meanwhile, the first central processor (175a) and the second central processor (175b) can each output an image based on result data from the first neural processor (179a) and the second neural processor (179b) to the display (180).

[0286] Accordingly, the display (180) can display an image (1122) of the vehicle interior, an image (1124) of the vehicle interior, an image (1126) of the vehicle exterior, and an image (1128) containing schedule information for the neural processor (179).

[0287] At this time, the image (1158) containing schedule information for the neural processor (179) can correspond to schedule information (1110) containing multiple artificial intelligence models.

[0288] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. Various modifications are possible by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure.

Claims

1. At least one neural processor; A central processor that controls the above neural processor; is provided, The above central processor is, A signal processing device that divides an artificial intelligence model running on the above neural processor into multiple groups and controls the execution or suspension of each of the multiple groups.

2. In Paragraph 1, The above central processor is, A signal processing device that determines the division of an artificial intelligence model or determines the number of divisions of an artificial intelligence model based on the size of data input to the neural processor or the expected execution time in the neural processor.

3. In Paragraph 1, The above central processor is, When a first artificial intelligence model is executed in the above neural processor, the first artificial intelligence model is divided into a first number of groups, and A signal processing device that separates the second artificial intelligence model into two groups when the second artificial intelligence model is executed in the neural processor.

4. In Paragraph 1, The above central processor is, Controls some of the above plurality of groups to be executed in the neural processor, and A signal processing device that controls other parts of the above plurality of groups so that they are not executed in the neural processor.

5. In Paragraph 1, The above central processor is, A signal processing device that separates a first artificial intelligence model executed on the above neural processor into a plurality of binaries and sets the processing order of the separated plurality of binaries based on priority or processing frequency.

6. In Paragraph 1, The above central processor is, A processing order is set for multiple artificial intelligence models running on the above neural processor based on priority or processing frequency, and The above neural processor is, A signal processing device that executes the plurality of artificial intelligence models according to the processing order set above.

7. In Paragraph 1, The above central processor is, If a first artificial intelligence model is scheduled to run on a first neural processor for a first period, and a second artificial intelligence model is scheduled to run repeatedly on a second neural processor for a period shorter than the first period, A signal processing device that controls the second artificial intelligence model to be executed repeatedly during the execution of the first artificial intelligence model in the first neural processor.

8. In Paragraph 1, The above central processor is, A signal processing device that converts a first artificial intelligence model for processing first data at a first frame rate to a second frame rate and executes it repeatedly in the neural processor, and controls the execution of a second artificial intelligence model for processing second data at a second frame rate during the off period of the first artificial intelligence model.

9. In Paragraph 1, The above central processor is, While executing a first artificial intelligence model for processing first data at a first frame rate in the above neural processor, if it is necessary to execute a second artificial intelligence model for processing second data at a second frame rate, A signal processing device that controls the execution of the first artificial intelligence model for processing the first data and the second artificial intelligence model for processing the second data at the second frame rate.

10. In Paragraph 1, The above central processor is, A signal processing device that controls the frame rate for camera data processing of the neural processor to be reduced when parked.

11. In Paragraph 1, The above central processor is, A signal processing device that controls the processing of at least a portion of camera data inside the vehicle in the neural processor to turn off when there is no passenger in the passenger seat or rear seat inside the vehicle.

12. In Paragraph 1, It further includes a graphics processor, The above central processor is, A signal processing device in which, in addition to the neural processor, at least one of the graphics processor and the central processor controls some of the plurality of groups to execute.

13. At least one neural processor; A central processor that controls the above neural processor; is provided, The above central processor is, A processing order is set for multiple artificial intelligence models running on the above neural processor based on priority or processing frequency, and The above neural processor is, A signal processing device that executes the plurality of artificial intelligence models according to the processing order set above.

14. In Paragraph 13, The above central processor is, If a first artificial intelligence model is scheduled to run on a first neural processor for a first period, and a second artificial intelligence model is scheduled to run repeatedly on a second neural processor for a period shorter than the first period, A signal processing device that controls the second artificial intelligence model to be executed repeatedly during the execution of the first artificial intelligence model in the first neural processor.

15. In Paragraph 13, The above central processor is, A signal processing device that converts a first artificial intelligence model for processing first data at a first frame rate to a second frame rate and executes it repeatedly in the neural processor, and controls the execution of a second artificial intelligence model for processing second data at a second frame rate during the off period of the first artificial intelligence model.

16. In Paragraph 13, The above central processor is, While executing a first artificial intelligence model for processing first data at a first frame rate in the above neural processor, if it is necessary to execute a second artificial intelligence model for processing second data at a second frame rate, A signal processing device that controls the execution of the first artificial intelligence model for processing the first data and the second artificial intelligence model for processing the second data at the second frame rate.

17. A signal processing device according to any one of claims 1 to 16; comprising a vehicle display device.

18. In Paragraph 17, A second signal processing device comprising at least one second neural processor and a second central processor controlling the second neural processor; further comprising The second signal processing device above is, A vehicle display device comprising a signal processing device according to any one of claims 1 to 16.

Images