Signal processing device and vehicle display device having the signal processing device
By employing a virtual machine monitoring program and a shared memory management mechanism in the vehicle signal processing device, data transmission and communication efficiency are optimized, solving the problem of poor efficiency in the prior art and achieving efficient data transmission and display.
Patent Information
- Application Number
- CN202480090775.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-14
- Publication Date
- 2026-11-17
AI Technical Summary
In the prior art, vehicle signal processing devices are inefficient in data transmission and communication between virtual machines and shared memory, especially in the transmission of image information, which consumes a lot of time.
A processor running a hypervisor is used to run multiple virtual machines on the hypervisor. The first communication library is used to store topic data in shared memory, and the second communication library is used to receive topic data from shared memory based on interrupts. This realizes a publish/subscribe mechanism and parallel data reception. Combined with the management of information area, heartbeat area and storage area, the data transmission process is optimized.
It achieves efficient data transmission and communication between the virtual machine and shared memory, improving data transmission efficiency, especially in the transmission of camera data and remotely updated data, ensuring stable data display and synchronization.
Smart Images

Figure CN123125577A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to servers, and more specifically, to a signal processing apparatus capable of efficiently transmitting data and a display device for a vehicle having the signal processing apparatus. Background Technology
[0002] A vehicle is a device that allows its passengers to move in a desired direction. A car is a representative example.
[0003] On the other hand, for the convenience of users of the vehicle, a vehicle signal processing device is installed inside the vehicle.
[0004] The vehicle's internal signal processing unit receives and processes sensor data from various internal sensor devices.
[0005] On the other hand, due to the increasing types and number of sensors installed in vehicles, such as Advanced Driver Assistance Systems (ADAS) or autonomous driving, the amount of data that needs to be processed is also increasing.
[0006] On the other hand, European Patent Publication No. EP3092566, which is existing literature, relates to a vehicle with a multi-user interface operating domain, which discloses virtualized shared memory in a multi-core, multi-process environment.
[0007] According to existing literature, since image information is transmitted to domains allocated to each process, one-way data storage is performed, which results in a lot of time being spent on retrieving data in shared memory, thus resulting in poor efficiency. Summary of the Invention
[0008] The problem that the invention aims to solve
[0009] The problem to be solved by the present invention is to provide a signal processing device capable of efficiently transmitting data and a display device for a vehicle having the signal processing device.
[0010] Another problem to be solved by the present invention is to provide a signal processing apparatus capable of efficiently performing communication between a virtual machine and a shared memory based on a service-oriented architecture, and a display device for a vehicle having the signal processing apparatus.
[0011] Technical solutions to the problem
[0012] A signal processing apparatus and a vehicle display device having the signal processing apparatus according to an embodiment of the present invention include a processor running a virtual machine monitor; the processor runs a plurality of virtual machines on the virtual machine monitor; among the plurality of virtual machines, a first virtual machine runs a first application, and a second virtual machine runs a second application; a first communication library in the first application is controlled to store subject data in a shared memory; and a second communication library in the second application receives subject data from the shared memory based on an interrupt.
[0013] On the other hand, the first communication library can be controlled to store topic data in shared storage based on publishing; the second communication library can receive topic data from shared storage based on subscription.
[0014] On the other hand, when the second virtual machine runs the second application and the third application, and the second communication library within the second application and the third communication library within the third application access the shared memory, the second communication library and the third communication library can receive topic data from the shared memory in parallel.
[0015] On the other hand, the first or second communication library can send or receive heartbeat signals from the moment it is connected to the shared memory.
[0016] On the other hand, shared memory can include information areas, heartbeat areas, and storage areas related to subject data.
[0017] On the other hand, the first communication library can be controlled such that if no first topic data is registered in the storage area of the shared memory, the first topic data is stored in the free area of the storage area, information related to the first topic data is recorded in the information area, and timestamp information is recorded in the heartbeat area.
[0018] On the other hand, the first communication library can be controlled to periodically update the timestamp information in the heartbeat region.
[0019] On the other hand, the second communication library can be controlled to receive information in the information area within the shared memory, and if it is confirmed based on the received information that the first topic data has been registered in the storage area, receive the first topic data stored in the storage area, update the information related to the first topic data in the information area, and update the timestamp information in the heartbeat area.
[0020] On the other hand, the second communication library can be controlled to periodically update the timestamp information in the heartbeat region.
[0021] On the other hand, the first communication library can be controlled to delete the information area, heartbeat area, and data in the storage area of the topic data if the timestamp of the publisher of the topic data in the heartbeat area of the shared memory is not updated.
[0022] On the other hand, the first communication library can be controlled to delete at least a portion of the data in the information area of the corresponding participant and at least a portion of the data in the heartbeat area when the timestamp of the specified participant in the heartbeat area of the shared memory is not updated.
[0023] On the other hand, the first communication library can output a first interrupt when storing or updating subject data in shared memory; the second communication library can receive subject data from shared memory based on the first interrupt.
[0024] On the other hand, the second communication library can output a second interrupt upon receiving topic data from the shared memory; the first communication library can store or update subsequent topic data to the shared memory after receiving the second interrupt.
[0025] On the other hand, when the topic data is of the first type, the first communication library can store or update subsequent data to the shared memory after the second application or all subscribers have completed receiving the first type of topic data; when the topic data is of the second type, the first communication library can store or update subsequent data to the shared memory before the second application or all subscribers have completed receiving the first type of topic data.
[0026] On the other hand, the first type of subject data may include remotely updated data.
[0027] On the other hand, the second type of subject data may include camera data.
[0028] On the other hand, when the topic data is first topic data with first priority, the first communication library can perform the transmission of the first topic data and the output of the first interrupt during the first time slot. When the topic data is second topic data with lower priority than the first, the first communication library can perform the transmission of the second topic data or the output of the second interrupt after the first time slot.
[0029] On the other hand, if the topic data is the first topic data with the first priority, and the interruption of the first topic data occurs before the first time slot period which is the first priority, the first communication library can perform the transmission of the first topic data and the output of the first interruption during the first time slot period.
[0030] On the other hand, applications running in multiple virtual machines can each run their own communication libraries for publish-or-subscribe-based communication with shared memory.
[0031] On the other hand, the communication library can perform at least one of the following: creation, access, deletion, reading, writing, quality of service management, interrupt management, and heartbeat management of regions within shared memory.
[0032] On the other hand, the communication library can delay the occurrence of an interrupt until the end of a higher-priority time slot.
[0033] On the other hand, the second application can be controlled to display camera images based on subject data on a monitor.
[0034] On the other hand, a third virtual machine among the plurality of virtual machines can run a third application; a third communication library within the third application can receive topic data from shared memory based on interrupts, and can be controlled to display objects based on the topic data on the display.
[0035] Invention Effects
[0036] A signal processing apparatus and a vehicle display device having the signal processing apparatus according to an embodiment of the present invention include a processor running a virtual machine monitor; the processor runs a plurality of virtual machines on the virtual machine monitor; among the plurality of virtual machines, a first virtual machine runs a first application, a second virtual machine runs a second application, a first communication library within the first application is configured to store subject data in shared memory; and a second communication library within the second application receives subject data from the shared memory based on an interrupt. This enables efficient data transmission. In particular, it enables efficient communication between virtual machines and shared memory based on a service-oriented architecture.
[0037] On the other hand, the first communication library can be controlled to store topic data in shared memory based on publishing; the second communication library can receive topic data from shared memory based on subscribing. This enables efficient data transmission.
[0038] On the other hand, when a second application and a third application are running in a second virtual machine, and the second communication library within the second application and the third communication library within the third application access shared memory, the second and third communication libraries can receive topic data from the shared memory in parallel. This enables efficient data transmission.
[0039] On the other hand, the first or second communication library can send or receive heartbeat signals from the moment it is connected to the shared memory. This enables efficient data transmission.
[0040] On the other hand, shared memory can include information areas related to the subject data, heartbeat areas, and storage areas. This enables efficient data transfer.
[0041] On the other hand, the first communication library can be controlled such that if no first topic data is registered in the storage area of the shared memory, the first topic data is stored in the free area of the storage area, information related to the first topic data is recorded in the information area, and timestamp information is recorded in the heartbeat area. This enables efficient data transmission.
[0042] On the other hand, the first communication library can be controlled to periodically update the timestamp information in the heartbeat region. This enables efficient data transmission.
[0043] On the other hand, the second communication library can be controlled to receive information within the information area of the shared memory, and, if it is confirmed based on the received information that the first topic data has been registered in the storage area, receive the first topic data stored in the storage area, update the information related to the first topic data in the information area, and update the timestamp information in the heartbeat area. This enables efficient data transmission.
[0044] On the other hand, the second communication library can be controlled to periodically update the timestamp information in the heartbeat region. This enables efficient data transmission.
[0045] On the other hand, the first communication library can be controlled to delete data in the information area, heartbeat area, and storage area related to the topic data if the timestamp of the publisher of the topic data in the heartbeat area of the shared memory has not been updated. This enables efficient data transmission.
[0046] On the other hand, the first communication library can be controlled to delete at least a portion of the data in the information area of the corresponding participant and at least a portion of the data in the heartbeat area when the timestamp of a specified participant in the heartbeat area of the shared memory has not been updated. This enables efficient data transmission.
[0047] On the other hand, the first communication library can output a first interrupt when storing or updating subject data in shared memory; the second communication library can receive subject data from shared memory based on the first interrupt. This enables efficient data transmission.
[0048] On the other hand, the second communication library can output a second interrupt when it receives topic data from the shared memory. After receiving the second interrupt, the first communication library can store or update subsequent topic data in the shared memory. This enables efficient data transmission.
[0049] On the other hand, when the topic data is of the first type, the first communication library can store or update subsequent data in the shared memory after the second application or all subscribers have completed receiving the first type of topic data; when the topic data is of the second type, the first communication library can store or update subsequent data in the shared memory before the second application or all subscribers have completed receiving the first type of topic data. This enables efficient data transmission.
[0050] On the other hand, the first type of subject data can include remotely updated data. This enables the efficient transmission of remotely updated data.
[0051] On the other hand, the second type of subject data can include camera data. This allows for efficient transmission of camera data.
[0052] On the other hand, when the topic data is first topic data with the highest priority, the first communication library can perform the transmission of the first topic data and the output of the first interrupt during the first time slot; when the topic data is second topic data with a lower priority than the first topic data, the first communication library can perform the transmission of the second topic data or the output of the second interrupt after the first time slot. This enables efficient data transmission.
[0053] On the other hand, if the topic data is the first topic data with the highest priority, and the interruption of the first topic data occurs before the first time slot period, which is also the first priority, the first communication library can perform the transmission of the first topic data and the output of the first interruption during the first time slot period. This enables efficient data transmission.
[0054] On the other hand, applications running in multiple virtual machines can each run their own communication libraries for publish-or-subscribe-based communication with shared memory. This enables efficient data transfer.
[0055] On the other hand, the communication library can perform at least one of the following: creation, access, deletion, reading, writing, quality of service management, interrupt management, and heartbeat management of regions within shared memory. This enables efficient data transmission.
[0056] On the other hand, the communication library can delay the occurrence of interrupts until the end of a higher-priority time slot. This enables efficient data transmission.
[0057] On the other hand, the second application can control the display of camera images based on subject data. This allows for stable display of camera images based on efficient transmission of camera data.
[0058] On the other hand, a third virtual machine among the plurality of virtual machines can run a third application; a third communication library within the third application can receive topic data from shared memory based on interrupts, and can be controlled to display objects based on the topic data on the display. Thus, objects can be displayed stably based on efficient data transmission. Attached Figure Description
[0059] Figure 1 This is a diagram showing a vehicle system that includes vehicles and servers.
[0060] Figure 2 It is shown Figure 1 A diagram showing the architecture of the vehicle's internal signal processing system.
[0061] Figure 3a This is a diagram showing an example of the configuration of a vehicle display device inside a vehicle.
[0062] Figure 3b This is another example of the configuration of a vehicle display device inside a vehicle.
[0063] Figure 4 yes Figure 1 An example of an internal block diagram of a vehicle.
[0064] Figure 5 This is an example of a block diagram of a vehicle display device according to an embodiment of the present invention.
[0065] Figure 6 This is a diagram used to illustrate the operation of a display device related to the present invention.
[0066] Figure 7 This is an example of an internal block diagram of a signal processing device according to an embodiment of the present invention.
[0067] Figures 8 to 16b This is an explanation Figure 7 The action is referenced in the diagram. Detailed Implementation
[0068] The present invention will now be described in more detail with reference to the accompanying drawings.
[0069] The suffixes “module” and “section” used in the following description of the constituent elements are assigned only for the convenience of drafting this specification and do not in themselves have any particularly important meaning or function. Therefore, the terms “module” and “section” may be used interchangeably.
[0070] Figure 1 This is a diagram showing a vehicle system that includes vehicles and servers.
[0071] Referring to the accompanying drawings, the vehicle system 10 includes a vehicle 200 and a server 900 that exchanges data with the vehicle 200.
[0072] The vehicle 200 is operated by a plurality of wheels 103FR, 103FL, 103RL, ... powered by a power source, and a square disc 150 used to adjust the direction of travel of the vehicle 200.
[0073] On the other hand, the vehicle 200 may also include a camera 195 for acquiring an image of the front of the vehicle.
[0074] On the other hand, multiple displays 180a and 180b for displaying images, information, etc. can be installed inside the vehicle 200.
[0075] exist Figure 1 In the example, 180a and 180b are plural displays, including instrument cluster display 180a and AVN (Audio Video Navigation) display 180b. Other examples include HUD (Head-Up Display).
[0076] On the other hand, the AVN (Audio Video Navigation) display 180b can also be called a Central Information Display.
[0077] On the other hand, the concept of vehicle 200 described in this specification may encompass vehicles with an engine as a power source, hybrid vehicles with an engine and an electric motor as power sources, electric vehicles with an electric motor as a power source, etc.
[0078] Figure 2 It is shown Figure 1 A diagram showing the architecture of the vehicle's internal signal processing system.
[0079] Referring to the attached diagram, the architecture 300a of the vehicle signal processing system inside the vehicle 200 can correspond to a zone-based architecture.
[0080] Therefore, sensor devices and processors inside the vehicle can be arranged in the plurality of regions Z1 to Z4 respectively, and a signal processing device 170a including a vehicle communication gateway GWDa can be arranged in the central region of the plurality of regions Z1 to Z4.
[0081] On the other hand, in addition to the vehicle communication gateway GWDa, the signal processing device 170a may also include an autonomous driving control module ACC and a cockpit control module CPG, etc.
[0082] The vehicle communication gateway GWDa within this signal processing device 170a can be an HPC (High Performance Computing) gateway.
[0083] Right now, Figure 2 The signal processing device 170a can exchange data with an external communication module (not shown) or processors (not shown) in multiple areas Z1 to Z4 as an integrated HPC.
[0084] Figure 3a This is a diagram showing an example of the configuration of a vehicle display device inside a vehicle.
[0085] Referring to the attached diagram, the vehicle interior may be equipped with an instrument cluster display 180a, an AVN (Audio Video Navigation) display 180b, a rear seat entertainment display 180c, 180d, and a rearview mirror display (not shown).
[0086] Figure 3b This is another example of the configuration of a vehicle display device inside a vehicle.
[0087] The vehicle display device 100 of this embodiment may include a plurality of displays 180a-180b and a signal processing device 170 that performs signal processing for displaying images, information, etc. on the plurality of displays 180a-180b and outputs image signals on at least one display 180a-180b.
[0088] Among the plurality of displays 180a to 180b, the first display 180a may be an instrument cluster display 180a for displaying driving status, action information, etc., and the second display 180b may be an AVN (Audio Video Navigation) display 180b for displaying vehicle driving information, navigation maps, various entertainment information or images.
[0089] The signal processing device 170 may have a processor 175 internally configured, and may run a first virtual machine to a third virtual machine (not shown) on a hypervisor (not shown) within the processor 175.
[0090] The second virtual machine (not shown) can operate for the first monitor 180a, and the third virtual machine (not shown) can operate for the second monitor 180b.
[0091] On the other hand, the first virtual machine (not shown) within the processor 175 can be controlled to transmit the same data to the second virtual machine (not shown) and the third virtual machine (not shown) based on the shared memory 508 of the virtual machine monitor 505. Thus, the same information or the same image can be displayed synchronously on the first display 180a and the second display 180b within the vehicle.
[0092] On the other hand, for data sharing, a first virtual machine (not shown) within the processor 175 shares at least a portion of the data with a second virtual machine (not shown) and a third virtual machine (not shown). Thus, data processing can be shared among multiple virtual machines used for multiple displays within the vehicle.
[0093] On the other hand, a first virtual machine (not shown) within the processor 175 can receive and process the vehicle's wheel speed sensor data, and transmit the processed wheel speed sensor data to at least one of a second virtual machine (not shown) and a third virtual machine (not shown). Thus, the vehicle's wheel speed sensor data can be shared with at least one virtual machine, etc.
[0094] On the other hand, the vehicle display device 100 of the present invention may also include a rear seat entertainment display 180c for displaying driving status information, brief navigation information, various entertainment information or images.
[0095] In addition to the first to third virtual machines (not shown), the signal processing device 170 can also control the RSE display 180c by adding a fourth virtual machine (not shown) to the virtual machine monitor (not shown) within the processor 175.
[0096] Therefore, a signal processing device 170 can be used to control various displays 180a~180c.
[0097] On the other hand, some of the multiple displays 180a~180c can operate based on Linux OS, while others can operate based on Internet OS.
[0098] The signal processing device 170 of this embodiment can be controlled to synchronously display the same information or the same image even on displays 180a to 180c that operate under various operating systems (OS).
[0099] On the other hand, Figure 3bThe example shows a first display 180a displaying a vehicle speed indicator 212a and an interior temperature indicator 213a, a second display 180b displaying a main screen 222 including a plurality of applications, a vehicle speed indicator 212b and an interior temperature indicator 213b, and a third display 180c displaying a second main screen 222b including a plurality of applications and an interior temperature indicator 213c.
[0100] Figure 4 yes Figure 1 An example of an internal block diagram of a vehicle.
[0101] Referring to the accompanying drawings, the vehicle 200 of the present invention may include a lamp drive unit 751, a steering drive unit 752, a braking drive unit 753, a power source drive unit 754, a suspension drive unit 756, an air conditioning drive unit 757, a window drive unit 758, a seat drive unit 761, and a signal processing device 170.
[0102] On the other hand, vehicle 200 may also include ECU 770, a plurality of sensor devices SN, and a plurality of communication modules EMa~EMd.
[0103] On the other hand, the vehicle 200 of the present invention may also include a vehicle display device 100.
[0104] The vehicle display device 100 of this embodiment may include an input unit 110, a communication device 120 for communicating with external devices, a plurality of communication modules EMa~Emd for internal communication, a memory 140, a signal processing device 170, a plurality of displays 180a~180c, an audio output unit 185, and a power supply unit 190.
[0105] Multiple communication modules EMa~EMd can be arranged separately, for example, in... Figure 2 The multiple regions Z1~Z4.
[0106] On the other hand, a communication switch 736b for data communication with each communication module EM1 to EM4 can be provided inside the signal processing device 170.
[0107] Each communication module EM1~EM4 can communicate with multiple sensor devices SN or ECU770 or area signal processing device 170Z.
[0108] On the other hand, the plurality of sensor devices SN may include a camera 195, a lidar 196, a radar 197, and a position sensor 198.
[0109] The input unit 110 may have physical buttons, touchpads, etc. for key input, touch input, etc.
[0110] On the other hand, the input unit 110 may have a microphone (not shown) for user voice input.
[0111] The communication device 120 can exchange data wirelessly with the mobile terminal 800 or the server 900.
[0112] In particular, the communication device 120 can wirelessly exchange data with the vehicle driver's mobile terminal. Various wireless data communication methods can be used, such as Bluetooth, WiFi, WiFi Direct, and Apix.
[0113] The communication device 120 can receive weather information and road traffic condition information, such as TPEG (Transport Protocol Expert Group) information, from the mobile terminal 800 or the server 900. For this purpose, the communication device 120 may have a mobile communication module (not shown).
[0114] On the other hand, the communication device 120 can exchange data wirelessly with adjacent vehicles.
[0115] For example, the communication device 120 can communicate via V2X (Vehicle-to-everything) to wirelessly exchange vehicle information with neighboring vehicles.
[0116] Multiple communication modules EM1~EM4 can receive sensor data from ECU770, sensor device SN, or area signal processing device 170Z, and transmit the received sensor data to signal processing device 170.
[0117] Here, sensor data may include at least one of the following: vehicle direction data, vehicle position data (GPS data), vehicle angle data, vehicle speed data, vehicle acceleration data, vehicle tilt data, vehicle forward / reverse data, battery data, fuel data, tire data, headlight data, vehicle interior temperature data, and vehicle interior humidity data.
[0118] Such sensor data can be obtained from heading sensors, yaw sensors, gyroscope sensors, position modules, vehicle forward / reverse sensors, wheel sensors, vehicle speed sensors, vehicle tilt detection sensors, battery sensors, fuel sensors, tire sensors, steering sensors based on steering wheel rotation, in-vehicle temperature sensors, and in-vehicle humidity sensors.
[0119] On the other hand, the positioning module may include a GPS module or a position sensor 198 for receiving GPS information.
[0120] On the other hand, at least one of the plurality of communication modules EM1 to EM4 can transmit the location information data sensed by the GPS module or the position sensor 198 to the signal processing device 170.
[0121] On the other hand, at least one of the plurality of communication modules EM1 to EM4 can receive frontal image data, side image data, rear image data, and distance information of obstacles around the vehicle from the camera 195, lidar 196, or radar 197, and transmit the received information to the signal processing device 170.
[0122] The memory 140 can store various data for the overall operation of the vehicle display device 100, such as programs for processing or controlling the signal processing device 170.
[0123] For example, memory 140 may store data about a hypervisor, a first virtual machine, to a third virtual machine, used to run within processor 175.
[0124] The audio output unit 185 converts the electrical signal from the signal processing device 170 into an audio signal and outputs it. For this purpose, a speaker or the like may be included.
[0125] The power supply unit 190 can supply the power required for the operation of each component according to the control of the signal processing device 170. In particular, the power supply unit 190 can receive power from the battery or the like inside the vehicle.
[0126] The signal processing device 170 controls the overall operation of the vehicle display device 100 or the various units within the vehicle 200.
[0127] For example, signal processing device 170 may include processor 175 that performs signal processing for vehicle displays 180a, 180b.
[0128] Processor 175 can run a first virtual machine to a third virtual machine (not shown) on a hypervisor (not shown) within processor 175.
[0129] Among the first to third virtual machines (not shown), the first virtual machine (not shown) can be named the Server Virtual Machine, and the second to third virtual machines (not shown) can be named the Guest Virtual Machine.
[0130] For example, a first virtual machine (not shown) within processor 175 can receive sensor data from a plurality of sensor devices, such as vehicle sensor data, location information data, camera image data, audio data, touch input data, etc., and output the data after processing or manipulation.
[0131] As described above, data sharing in a 1:N manner can be achieved by performing most of the data processing in the first virtual machine (not shown).
[0132] As another example, the first virtual machine (not shown) can directly receive and process CAN data, Ethernet data, audio data, radio data, USB data, and wireless communication data for the second to third virtual machines (not shown).
[0133] In addition, the first virtual machine (not shown) can transfer the processed data to the second to third virtual machines (not shown).
[0134] Therefore, by having only the first virtual machine (not shown) among the first to third virtual machines (not shown) receive sensor data, or communication data, or external input data from a plurality of sensor devices and perform signal processing, the signal processing burden in other virtual machines can be reduced, and 1:N data communication can be achieved, thereby enabling synchronization during data sharing.
[0135] On the other hand, the first virtual machine (not shown) can be controlled to record data in the shared memory 508 and share the same data with the second virtual machine (not shown) and the third virtual machine (not shown).
[0136] For example, the first virtual machine (not shown) can be controlled to record vehicle sensor data, or the location information data, or the camera image data, or the touch input data in the shared memory 508, and share the same data with the second virtual machine (not shown) and the third virtual machine (not shown). This enables 1:N data sharing.
[0137] Ultimately, by performing most of the data processing in the first virtual machine (not shown), 1:N data sharing can be achieved.
[0138] On the other hand, the first virtual machine (not shown) within the processor 175 can be controlled to set up a shared memory 508 based on the hypervisor 505 to transmit the same data to the second virtual machine (not shown) and the third virtual machine (not shown).
[0139] On the other hand, the signal processing device 170 can process various signals such as audio signals, video signals, and data signals. Therefore, the signal processing device 170 can be implemented as a system on chip (SOC).
[0140] Figure 5 This is an example of a block diagram of a vehicle display device according to an embodiment of the present invention.
[0141] Referring to the accompanying drawings, the vehicle display device 900 of this embodiment includes a signal processing device 170, which includes at least one neural processor 179 and a central processing unit 175 for controlling the neural processor 179.
[0142] On the other hand, the vehicle display device 900 of the present invention may also include at least one display.
[0143] On the other hand, the vehicle display device 900 of this embodiment may also include Figure 4 The steering drive unit 752, braking drive unit 753, power source drive unit 754, ECU 770, and multiple sensor devices SN, etc.
[0144] On the other hand, the vehicle display device 900 of this embodiment may also include Figure 4 The light drive unit 751, suspension drive unit 756, air conditioning drive unit 757, window drive unit 758, seat drive unit 761, and multiple communication modules EMa~EMd, etc.
[0145] In the accompanying drawings, instrument cluster display 180a and AVN display 180b are illustrated as at least one display.
[0146] On the other hand, the vehicle display device 900 may also include a plurality of area signal processing devices 170Z1 to 170Z4.
[0147] At this time, the signal processing device 170, as a high-performance central signal processing and control device with multiple CPUs 175, GPUs 178, NPUs 179, etc., can be named HPC (High Performance Computing) signal processing device or central signal processing device.
[0148] The multiple area signal processing devices 170Z1~170Z4 and the signal processing device 170 are connected by wired cables CB1~CB4.
[0149] On the other hand, the multiple regional signal processing devices 170Z1 to 170Z4 can be connected to each other using wired cables CBa to CBd.
[0150] At this time, the wired cables CBa~CBd can include CAN communication cables, Ethernet communication cables, or PCI Express (peripheral component interconnect express, high-speed serial computer expansion bus) cables.
[0151] On the other hand, the signal processing apparatus 170 of the present invention may also include a large-capacity storage device 925.
[0152] On the other hand, the signal processing apparatus 170 of the present invention may also include a graphics processor 178.
[0153] On the other hand, the signal processing device 170 of the present invention may include at least one central processing unit 175, 178, 177.
[0154] On the other hand, sensor data can be transmitted to signal processing device 170 from at least one of the plurality of regional signal processing devices 170Z1 to 170Z4. In particular, the sensor data can be stored in storage device 925 within signal processing device 170.
[0155] The sensor data at this time may include at least one of the following: camera data, lidar data, radar data, vehicle direction data, vehicle position data (GPS data), vehicle angle data, vehicle speed data, vehicle acceleration data, vehicle tilt data, vehicle forward / reverse data, battery data, fuel data, tire data, headlight data, vehicle interior temperature data, and vehicle interior humidity data.
[0156] The accompanying drawings illustrate camera data from camera 195a and lidar data from lidar sensor 196 being input to the first area signal processing device 170Z1. The camera data and lidar data are then transmitted to the signal processing device 170 via the second area signal processing device 170Z2 and the third area signal processing device 170Z3, etc.
[0157] On the other hand, since the data read speed or write speed of the storage device 925 is faster than the network speed when transmitting sensor data from at least one of the multiple area signal processing devices 170Z1 to 170Z4 to the signal processing device 170, it is preferable to perform multi-path routing to prevent network bottlenecks.
[0158] Therefore, the signal processing apparatus 170 of this embodiment of the invention can perform multi-path routing based on Software-Defined Network (SDN). This ensures a stable network environment for data reading or writing to the storage device 925. Furthermore, since multiple paths can be used to transmit data to the storage device 925, data can be transmitted by dynamically changing the network configuration.
[0159] For high-frequency, low-latency communication, the data communication between the plurality of area signal processing devices 170Z1 to 170Z4 within the vehicle display device 900 of this embodiment and the signal processing device 170 is preferably PCIe (Peripheral Component Interconnect Express) communication.
[0160] On the other hand, the signal processing apparatus 170 of this embodiment of the invention can receive internal images from the internal camera 195i and perform signal processing on the internal images.
[0161] On the other hand, the signal processing device 170 of this embodiment of the invention can receive a front image from the front camera 195a and perform signal processing on the front image.
[0162] Figure 6 This is a diagram used to illustrate the operation of a display device related to the present invention.
[0163] Referring to the accompanying drawings, the display device 900a related to the present invention may include a signal processing device 170 and a switch TSW.
[0164] The signal processing apparatus 170 associated with the present invention may include a plurality of processor cores CR1~CRn, MR and interface INT.
[0165] On the other hand, Ethernet communication via Ethernet interface ESW or PCIe communication via PCIe interface PSW can be performed between signal processing device 170 and external switch TSW.
[0166] The first processor 175 within the signal processing device 170 may include processor cores CR1~CRn of a first type among a plurality of processor cores, and the second processor 177 may include processor cores MR of a second type among a plurality of processor cores.
[0167] For example, one of communication using shared memory 508 or switch 509 within hypervisor 505 and Ethernet communication can be performed among a plurality of virtual machines 820-850.
[0168] On the other hand, the processor 175 within the signal processing device 170 can run the first virtual machine 850, the second virtual machine 830, and the third virtual machine 820 on the hypervisor 505.
[0169] On the other hand, the second processor 177 within the signal processing device 170 can run the fourth virtual machine 840 without running the virtual machine monitor 505.
[0170] On the other hand, each virtual machine 820~850 may include network interfaces or drivers 821, 831, 841, 851, which can perform Ethernet communication with an external switch TSW through the Ethernet interface ESW in the interface INT, or perform PCIe communication through the PCIe interface PSW in the interface INT.
[0171] On the other hand, processor 175 can transmit third data to second processor 177 based on inter-platform communication (IPC).
[0172] On the other hand, according to Figure 6 When communicating with the shared memory 508, the signal processing device 170 uses the shared memory interface in the network drivers 831, 841, and 851 to access the address of the shared memory 508 and store or receive data.
[0173] In this way, in order to communicate with the shared memory 508, the address of the shared memory 508 is required, and the specified data must be transmitted only through the prescribed path.
[0174] In particular, when using shared memory 508 for large-scale data transfer between virtual machines, the address of shared memory 508 is required, and the specified data must be transferred only through the prescribed path.
[0175] Additionally, if communication using shared memory 508 terminates abnormally, a situation arises where shared memory 508 becomes unusable.
[0176] Ultimately, the transmission of large volumes of data, such as camera data and LiDAR data, or time-critical data, becomes difficult.
[0177] Therefore, this invention proposes a solution for efficient data transmission. Referring to this, [further details are needed]. Figure 7 The following diagrams will illustrate this.
[0178] Figure 7 This is an example of an internal block diagram of a signal processing device according to an embodiment of the present invention.
[0179] Referring to the accompanying drawings, the signal processing apparatus 170 of this embodiment of the invention includes a processor 175 running a virtual machine monitor 505.
[0180] In one embodiment of the present invention, the processor 175 runs a plurality of virtual machines 520-550 on the hypervisor 505.
[0181] On the other hand, among the plurality of virtual machines 520 to 550, the first virtual machine 520 runs the first application 528, and the second virtual machine 530 runs the second application 538.
[0182] On the other hand, the first application 528 in the first virtual machine 520 runs the first communication library 529, and the second application 538 in the second virtual machine 530 runs the second communication library 539.
[0183] At this point, the first communication library 529 and the second communication library 539 can be communication libraries based on inter-VM communication.
[0184] On the other hand, the first communication library 529 within the first application 528 controls the storage of topic data in the shared memory 508, and the second communication library 539 within the second application 538 receives topic data from the shared memory 508 based on interrupts. This enables efficient data transmission. In particular, it enables efficient communication between a service-oriented architecture-based virtual machine and the shared memory 508.
[0185] On the other hand, the first communication library 529 can be controlled to store topic data in the shared memory 508 based on publishing, and the second communication library 539 can receive topic data from the shared memory 508 based on subscribing. Thus, data can be transmitted efficiently.
[0186] That is, the first application 528 can be the publisher of the topic data, and the second application 538 can be the subscriber of the topic data. This enables efficient data transmission.
[0187] On the other hand, the first application 528 can run main code 527 in addition to the first communication library 529, and the second application 538 can run main code 537 in addition to the second communication library 539.
[0188] On the other hand, among the plurality of virtual machines 520 to 550, the third virtual machine 540 can run the third application 548, and the fourth virtual machine 550 can run the fourth application 558.
[0189] On the other hand, the third application 548 in the third virtual machine 540 runs the third communication library 549, and the fourth application 558 in the fourth virtual machine 550 runs the fourth communication library 559.
[0190] On the other hand, the third application 548 can run the main code 547 in addition to the third communication library 549, and the fourth application 558 can run the main code 557 in addition to the fourth communication library 559.
[0191] On the other hand, unlike the attached diagram, the second virtual machine 530 can also run the second application 538 and the third application 548.
[0192] On the other hand, when the second virtual machine 530 runs the second application 538 and the third application 548, and the second communication library 539 in the second application 538 and the third communication library 549 in the third application 548 access the shared memory 508, the second communication library 539 and the third communication library 549 can receive topic data in parallel from the shared memory 508.
[0193] exist Figure 6 In the case where data is simultaneously connected to the shared memory 508, it is received sequentially. However, according to an embodiment of the present invention, even when simultaneously connected to the shared memory 508, topic data can be received from the shared memory 508 in parallel. Thus, data can be transmitted efficiently.
[0194] On the other hand, the first communication library 529 or the second communication library 539 can send or receive heartbeat signals from the moment they are connected to the shared memory 508. This enables efficient data transmission.
[0195] On the other hand, the shared memory 508 may include an information area SMa for topic data, a heartbeat area SMb, and storage areas SMC~SMf. This enables efficient data transmission.
[0196] On the other hand, according to Figure 7 Applications 528-558 running in multiple virtual machines 520-550 can each run communication libraries 529-559 for publish-or-subscribe-based communication with shared memory 508. This enables efficient data transfer.
[0197] On the other hand, applications 528-558 that communicate with the shared memory 508 can use communication libraries 529-559 to register for publishing or subscribing to topic data.
[0198] On the other hand, communication libraries 529-559 can perform at least one of the following operations within the shared memory 508: creation, access, deletion, reading, writing, quality of service management, interrupt management, and heartbeat management. This enables efficient data transmission.
[0199] For example, communication libraries 529-559 can allocate free storage areas SMC-SMf within shared memory 508 to enable the transmission of subject data.
[0200] On the other hand, the communication libraries 529-559 can be controlled to store the publisher information, subscriber information, and region information of the topic data in the information region SMa of the shared memory 508 so as to be able to share the corresponding information.
[0201] On the other hand, communication libraries 529-559 can store or periodically update relevant information in the heartbeat area SMb of shared memory 508 so that publishers and subscribers of topic data can confirm each other's status.
[0202] On the other hand, communication libraries 529-559 can delay the occurrence of an interrupt until the end of a higher-priority time slot. This enables time-deterministic transmission, ensuring that data arrives before the specified time.
[0203] Figures 8 to 16b This is an explanation Figure 7 The action is referenced in the diagram.
[0204] Figure 8 This is a diagram illustrating the initialization of shared memory.
[0205] Referring to the accompanying drawings, the main code 527 within the first application 528 can send an access request (SA1) to the first communication library 529 within the first application 528 to the shared memory 508.
[0206] Next, the first communication library 529 within the first application 528 can access the shared memory 508 (SA2).
[0207] Next, the first communication library 529 within the first application 528 can access the information area SMa (SA3) of the shared memory 508.
[0208] Next, the first communication library 529 within the first application 528 reads information (SA4) from the information area SMa of the shared memory 508.
[0209] Next, the first communication library 529 within the first application 528 can store the information read from the shared memory 508 internally (SA5).
[0210] Next, the first communication library 529 within the first application 528 can access the heartbeat region SMb (SA6) of the shared memory 508.
[0211] Next, the main code 537 within the second application 538 can send an access request (SA7) to the second communication library 539 within the second application 538 to the shared memory 508.
[0212] Next, the second communication library 539 within the second application 538 can access the shared memory 508 (SA8).
[0213] Next, the second communication library 539 within the second application 538 can access the information area SMa (SA9) of the shared memory 508.
[0214] Next, the second communication library 539 within the second application 538 reads information (SA10) from the information area SMa of the shared memory 508.
[0215] Next, the second communication library 539 within the second application 538 can store the information read from the shared memory 508 internally (SA11).
[0216] The second communication library 539 within the second application 538 can access the heartbeat region SMb (SA12) of the shared memory 508.
[0217] then, Figure 9 This is a diagram illustrating the registration or subscription of topic data.
[0218] Referring to the accompanying drawings, the main code 527 within the first application 528 can request the publication registration (SB1) of the first topic data from the first communication library 529 within the first application 528.
[0219] Next, the first communication library 529 within the first application 528 can read the information (SB2) in the information area SMa of the shared memory 508.
[0220] Next, the first communication library 529 within the first application 528 can confirm whether the first topic data has been registered (SB3) based on the information read from the shared memory 508.
[0221] On the other hand, if no first topic data is registered in the shared memory 508, the first communication library 529 in the first application 528 can search for a free area (SB4) in the shared memory 508.
[0222] Next, the first communication library 529 within the first application 528 can access the free area (SB5) of the shared memory 508.
[0223] At this time, the free area within the shared memory 508 can become the data storage area for the primary subject data. That is, it can be... Figure 7 Any one of the plurality of storage regions SMC~SMf.
[0224] Next, the first communication library 529 within the first application 528 can record the first topic data information (SB6) in the information area SMa of the shared memory 508.
[0225] For example, the first communication library 529 within the first application 528 can record the name information, data area information, publisher client ID information, etc. of the first topic data in the information area SMa of the shared memory 508.
[0226] Next, the first communication library 529 within the first application 528 can record information (SB7) related to the first topic data in the heartbeat area SMb of the shared memory 508.
[0227] For example, the first communication library 529 within the first application 528 can record the name information of the first topic data, its own client ID information, current timestamp information, etc. in the heartbeat area SMb of the shared memory 508.
[0228] Next, the first communication library 529 within the first application 528 can periodically update the timestamp (SB8) of the information related to the first topic data in the heartbeat region SMb of the shared memory 508.
[0229] For example, the first communication library 529 within the first application 528 can update the timestamp of information related to the first topic data in the heartbeat region SMb of the shared memory 508 approximately every 1 second.
[0230] Next, the main code 537 within the second application 538 can request the subscription registration (SB9) of the first topic data from the second communication library 539 within the second application 538.
[0231] Next, the second communication library 539 within the second application 538 can read the information (SB10) in the information area SMa of the shared memory 508.
[0232] Next, the second communication library 539 within the second application 538 can confirm whether the first topic data has been registered (SB11) based on the information read from the shared memory 508.
[0233] Next, if the first topic data has been registered or the subscriber has been registered, the second communication library 539 in the second application 538 can access the data storage area SMC~SMf (SB12) pointed to by the first topic data based on the information in the shared memory 508.
[0234] Next, the second communication library 539 within the second application 538 can update the first topic data information (SB13) in the information area SMa of the shared memory 508.
[0235] For example, the second communication library 539 within the second application 538 can update the name information, data area information, publisher client ID information, etc. of the first topic data in the information area SMa of the shared memory 508.
[0236] Next, the second communication library 539 within the second application 538 can update the first topic data information (SB14) in the heartbeat region SMb of the shared memory 508.
[0237] For example, the second communication library 539 within the second application 538 can update the name information, its own client ID information, current timestamp information, etc. of the first topic data in the heartbeat area SMb of the first shared memory 508.
[0238] Next, the second communication library 539 within the second application 538 can periodically update the timestamp (SB15) of the information related to the first topic data in the heartbeat region SMb of the shared memory 508.
[0239] For example, the second communication library 539 within the second application 538 can update the timestamps of information related to the first topic data in the heartbeat region SMb of the shared memory 508 approximately every 1 second.
[0240] Ultimately, according to Figure 9The first communication library 529 can be controlled such that if no first topic data is registered in the storage areas SMC~SMf within the shared memory 508, the first topic data is stored in the free area of the storage areas SMC~SMf, information related to the first topic data is recorded in the information area SMa, and timestamp information is recorded in the heartbeat area SMb. This enables efficient data transmission.
[0241] On the other hand, the first communication library 529 can be controlled to periodically update the timestamp information in the heartbeat region SMb. Thus, the first communication library 529 can notify other communication libraries that it is operating normally.
[0242] On the other hand, the second communication library 539 can be controlled to receive information in the information area SMa within the shared memory 508, and, if it is confirmed based on the received information that the first topic data has been registered in the storage areas SMC~SMf, receive the first topic data stored in the storage areas SMC~SMf, update the information related to the first topic data in the information area SMa, and update the timestamp information in the heartbeat area SMb. This enables efficient data transmission.
[0243] On the other hand, the second communication library 539 can be controlled to periodically update the timestamp information in the heartbeat region SMb. Thus, the second communication library 539 can notify other communication libraries that it is operating normally.
[0244] then, Figure 10 This is a diagram illustrating the settings for service quality or key data.
[0245] Referring to the accompanying drawings, the main code 527 within the first application 528 can request the first topic data publication registration and the service quality information or time-critical information registration (SC1) from the first communication library 529 within the first application 528.
[0246] Next, the first communication library 529 within the first application 528 can read the information (SC2) in the information area SMa of the shared memory 508.
[0247] Next, the first communication library 529 within the first application 528 can confirm whether the first topic data has been registered based on the information read from the shared memory 508 (SC3).
[0248] On the other hand, if no first topic data is registered in the shared memory 508, the first communication library 529 in the first application 528 can search for a free area (SC4) in the shared memory 508.
[0249] Next, the first communication library 529 within the first application 528 can access the free area (SC5) of the shared memory 508.
[0250] Next, the first communication library 529 within the first application 528 can record the first topic data information (SC6) in the information area SMa of the shared memory 508.
[0251] For example, the first communication library 529 within the first application 528 can record the name information, data area information, publisher client ID information, service quality information, time-critical information, etc. of the first topic data in the information area SMa of the shared memory 508.
[0252] At this time, the Quality of Service (QoS) information can include information indicating that messages must be transmitted unconditionally (Reliable) or information indicating that it is okay not to transmit messages (Best effort).
[0253] On the other hand, Quality of Service (QoS) information can include either a first type of information that all subscribers must read or receive, or a second type of information that all subscribers do not necessarily need to read or receive.
[0254] On the other hand, the first type of information can correspond to information that indicates a message must be transmitted unconditionally (Reliable), and the second type of information can correspond to information that indicates a message that does not need to be transmitted (Best effort).
[0255] On the other hand, time-critical information can include information with the highest priority or information with the lowest priority.
[0256] Next, the first communication library 529 within the first application 528 can record information related to the first topic data (SC7) in the heartbeat area SMb of the shared memory 508.
[0257] For example, the first communication library 529 within the first application 528 can record the name information of the first topic data, its own client ID information, current timestamp information, etc. in the heartbeat area SMb of the shared memory 508.
[0258] Next, the first communication library 529 within the first application 528 can periodically update the timestamps (SC8) of information related to the first topic data in the heartbeat region SMb of the shared memory 508.
[0259] For example, the first communication library 529 within the first application 528 can update the timestamp of information related to the first topic data in the heartbeat region SMb of the shared memory 508 approximately every 1 second.
[0260] Next, the main code 537 within the second application 538 can request the subscription registration of the first topic data and the registration of service quality information or time-critical information (SC9) from the second communication library 539 within the second application 538.
[0261] Next, steps SC10 to SC15 in the diagram can be combined with... Figure 9 Steps SB10 to SB15 are the same.
[0262] On the other hand, according to Figure 10 The communication libraries 529-559 within the application programs 528-558 that communicate with the shared memory 508 can set quality of service information or time-critical information for the subject data.
[0263] For example, each communication library 529-559 can distinguish subject data into messages that must be transmitted unconditionally or messages that do not need to be transmitted.
[0264] On the other hand, each communication library 529~559 can confirm whether all subscribers of the topic data have read the topic data before the publisher of the topic data transmits additional data.
[0265] On the other hand, each communication library 529~559 can set an upper limit on the number of unread messages that can be stored.
[0266] On the other hand, when setting the storage area for topic data in the shared memory 508, each communication library 529 to 559 can set the size of the storage area based on the maximum size of the topic data or the historical record.
[0267] On the other hand, each communication library 529-559 can output an interrupt to the publisher when the subscriber reads topic data to notify it of where the data has been read. Thus, the communication library within the publisher can know where the data in the shared memory 508 can be deleted to.
[0268] On the other hand, each communication library 529-559 can be controlled to allocate a specified time slot for the transmission of time-critical data when the transmission of time-critical data has been registered, so that only time-critical data is transmitted within that time slot. This enables time-deterministic transmission, ensuring that data arrives before the specified time.
[0269] then, Figure 11 This is a diagram illustrating the transmission or reception of subject data.
[0270] Referring to the accompanying drawings, the main code 527 within the first application 528 can request the data transmission of the first topic data (SD1) from the first communication library 529 within the first application 528.
[0271] Next, the first communication library 529 within the first application 528 can retrieve the storage areas SMC~SMf (SD2) of the shared memory 508.
[0272] Next, the first communication library 529 within the first application 528 can record the first topic data (SD3) in the storage areas SMC~SMf of the retrieved shared memory 508.
[0273] Next, the first communication library 529 within the first application 528 can output a first interrupt (SD4) to notify that the first topic data has been updated after recording the first topic data.
[0274] Next, the second communication library 539 within the second application 538 can receive the first interrupt (SD5) generated by the first communication library 529.
[0275] Next, the second communication library 539 within the second application 538 can read the stored first topic data (SD6) from the storage areas SMC~SMf of the shared memory 508 based on an interrupt.
[0276] Next, the second communication library 539 within the second application 538 can notify the main code 537 within the second application 538 of the reading of the first topic data (SD7).
[0277] Next, if the first topic data read by the second communication library 539 in the second application 538 is topic data of the first type that must be transmitted unconditionally, the second communication library 539 can output a second interrupt (SD8) indicating that the reading is complete to the first communication library 529.
[0278] On the other hand, the first communication library 529 within the first application 528 can, based on the second interrupt, confirm that the reading of the first topic data has been completed and prepare to transmit subsequent topic data (SD9).
[0279] according to Figure 11 The first communication library 529 can output a first interrupt when storing or updating topic data in the shared memory 508, and the second communication library 539 can receive topic data from the shared memory 508 based on the first interrupt. This enables efficient data transmission.
[0280] On the other hand, the second communication library 539 can output a second interrupt when it receives topic data from the shared memory 508, and the first communication library 529 can store or update subsequent topic data to the shared memory 508 after receiving the second interrupt. This enables efficient data transmission.
[0281] On the other hand, when the topic data is of the first type of topic data corresponding to a message that must be transmitted unconditionally, the first communication library 529 may store or update subsequent topic data to the shared memory 508 after the second application 538 or all subscribers have completed receiving the first type of topic data.
[0282] On the other hand, the first type of subject data can include remotely updated data. This enables the efficient transmission of remotely updated data.
[0283] On the other hand, when the topic data is of the second type, corresponding to a message that does not require transmission, the first communication library 529 can store or update subsequent data in the shared memory 508 before the second application 538 or all subscribers have completed receiving the first type of topic data. This enables efficient data transmission.
[0284] On the other hand, the second type of subject data can include camera data. This allows for efficient transmission of camera data.
[0285] Figure 12 It is a sequence diagram representing the transmission of time-critical data.
[0286] Referring to the accompanying drawings, the first communication library 529 within the first application 528 can transmit topic data to the shared memory 508 (S1210).
[0287] At this time, the first communication library 529 in the first application 528 determines whether the topic data is time-critical data (S1215). If so, it determines whether it is within the first time slot period corresponding to the first priority (S1220). If so, an interrupt can be output within the first time slot period (S1230).
[0288] Therefore, when the subject data is time-critical, data transmission can be performed quickly and efficiently.
[0289] On the other hand, when the topic data is not time-critical data, the first communication library 529 in the first application 528 determines whether it is within the first time slot period corresponding to the first priority (S1225). If not, it can output an interrupt during the second time slot period (S1230).
[0290] Therefore, data transmission can be performed stably even when the subject data is not time-critical data.
[0291] Figure 13 This is an explanation Figure 12 The diagram used for reference at the time.
[0292] Referring to the attached diagram, the GRa area represents multiple types of data transmission requests, and the GRb area illustrates the occurrence of various data transmissions and interruptions.
[0293] For example, when a transmission request for time-critical data Dtc is received at time T1 in the first time slot period STa corresponding to the first priority, the first communication library 529 in the first application 528 can be controlled to transmit data to the shared memory 508 at time T1 and output a first interrupt.
[0294] Then, at time T2 after the first time slot STa, when a transmission request for ordinary data Dtd (non-time-critical data) is received, the first communication library 529 within the first application 528 can be controlled to transmit data to the shared memory 508 at time T2 and output a second interrupt.
[0295] On the other hand, when a transmission request for time-critical data Dtc is received at time point T3 in the second time slot STb corresponding to the first priority, the first communication library 529 in the first application 528 can be controlled to transmit data to the shared memory 508 at time point T3 and output a first interrupt.
[0296] On the other hand, when a request to transmit ordinary data Dtg (non-time-critical data) is received at time T5 in the second time slot STb corresponding to the first priority, the first communication library 529 within the first application 528 can be controlled to transmit data to the shared memory 508 and output a second interrupt at time T5 after the end of the second time slot STb. This enables efficient data transmission.
[0297] Then, at time T6 after the second time slot STb, when a transmission request for ordinary data Dtg (non-time-critical data) is received, the first communication library 529 within the first application 528 can be controlled to transmit data to the shared memory 508 at time T6 and output a second interrupt.
[0298] On the other hand, when a transmission request for time-critical data Dtc is received at time T7 in the third time slot corresponding to the first priority, the first communication library 529 in the first application 528 can be controlled to transmit data to the shared memory 508 at time T7 and output a first interrupt.
[0299] Then, at time T8 after the third time slot STc, when a transmission request for ordinary data Dtg (non-time-critical data) is received, the first communication library 529 within the first application 528 can be controlled to transmit data to the shared memory 508 at time T8 and output a second interrupt.
[0300] Next, at time T9 after the third time slot STc, when a transmission request for time-critical data Dtc is received, the first communication library 529 within the first application 528 can be controlled to transmit data to the shared memory 508 and output a first interrupt at time T10 corresponding to the subsequent fourth time slot STd. This enables efficient data transmission.
[0301] On the other hand, when a request to transmit ordinary data Dtg (non-time-critical data) is received at time T10 during the fourth time slot STd corresponding to the first priority, the first communication library 529 within the first application 528 can be controlled to transmit data to the shared memory 508 and output a second interrupt at time T11 after the end of the fourth time slot STd. This enables efficient data transmission.
[0302] Then, at time point T12 after the fourth time slot STd, when a transmission request for ordinary data Dtg (non-time-critical data) is received, the first communication library 529 within the first application 528 can be controlled to transmit data to the shared memory 508 at time point T12 and output a second interrupt.
[0303] Reference Figure 13 When the topic data is the first topic data with the first priority, the first communication library 529 within the first application 528 can perform the transmission of the first topic data and the output of the first interrupt within the first time slot STa.
[0304] In particular, when the topic data is first topic data of first priority and a transmission request is received within the first time slot STa, the first communication library 529 within the first application 528 can perform the transmission of the first topic data and the output of the first interrupt within the first time slot STa.
[0305] At this point, the top priority data can be time-critical data. This allows for the rapid transmission of time-critical data.
[0306] On the other hand, if the topic data is second topic data with a lower priority than the first topic data, the first communication library 529 within the first application 528 can perform the transmission of the second topic data or the output of the second interrupt after the first time slot STa.
[0307] For example, if the topic data is second-priority second-topic data and a transmission request is received within the first time slot STa, the first communication library 529 within the first application 528 can perform the transmission of the second topic data and the output of the second interrupt between time points T2 and T3 after the end of the first time slot STa. This enables efficient data transmission.
[0308] On the other hand, when the topic data is the first topic data with the first priority, and the interruption of the first topic data occurs before the first time slot period with the first priority, the first communication library 529 can perform the transmission of the first topic data and the output of the first interruption during the first time slot period.
[0309] For example, when a transmission request for time-critical data Dtc is received at time T9 after the third time slot STc, the first communication library 529 within the first application 528 can be controlled to transmit data to the shared memory 508 and output a first interrupt at time T10, which corresponds to the subsequent fourth time slot STd after time T9. This enables efficient data transmission.
[0310] On the other hand, the first communication library 529 within the first application 528 can set the period of the first to fourth time slots based on the occurrence cycle of the subject data to be transmitted.
[0311] For example, when the occurrence period of the subject data to be transmitted is Ta, the first communication library 529 within the first application 528 can set the period between the first and fourth time slots to Ta, or it can be set to Ta / 2, Ta / 4, etc. This enables efficient data transmission.
[0312] On the other hand, the first communication library 529 within the first application 528 can set the length of the first to fourth time slots based on the time required for the transmission of the subject data to be transmitted.
[0313] For example, the first communication library 529 within the first application 528 can be controlled such that the longer the transmission time of the subject data to be transmitted, the longer the length of the first to fourth time slots. This enables efficient data transmission.
[0314] Figure 14 This is a diagram used to illustrate the use of heartbeat malfunctions or safety features.
[0315] Referring to the accompanying drawings, the first communication library 529 in the first application 528 or the second communication library 539 in the second application 538 can read information (SE1) in the heartbeat region SMb of the shared memory 508 based on a timer.
[0316] Next, the first communication library 529 in the first application 528 or the second communication library 539 in the second application 538 can confirm whether the timestamps of all participants in the heartbeat region SMb of the shared memory 508 have exceeded the set time (SE2).
[0317] Next, when there is a participant that has exceeded the set time, the first communication library 529 in the first application 528 can determine that the connection between the corresponding participant and the shared memory 508 has been disconnected (SE3).
[0318] For example, if the participant that has exceeded the set time is the second application 538, the first communication library 529 in the first application 528 can determine that the connection between the second application 538 and the shared memory 508 has been disconnected.
[0319] Furthermore, the first communication library 529 within the first application 528 can be controlled to delete information related to the second application 538 in the information area SMa and heartbeat area SMb within the shared memory 508 based on the disconnection of the second application 538 from the shared memory 508.
[0320] Therefore, even after the second application 538 is disconnected, the transmission of topic data can still be performed stably.
[0321] Next, the first communication library 529 within the first application 528 can record information (SE4) about the first subject data in the information area SMa of the shared memory 508.
[0322] For example, the first communication library 529 within the first application 528 can delete the IDs of participants that have exceeded a set time.
[0323] Next, the first communication library 529 within the first application 528 can record information (SE5) about the first subject data in the heartbeat area SMb of the shared memory 508.
[0324] For example, the first communication library 529 within the first application 528 can delete the IDs and timestamps of participants that have exceeded a set time.
[0325] On the other hand, when the participant exceeding the set time is the first application 528, which is the publisher of the first topic data, the first communication library 529 within the first application 528 can be controlled to delete the first topic data in the storage areas SMC~SMf within the shared memory 508, and delete information about the first topic data in the information area SMa and the heartbeat area SMb (SE6). Thus, unnecessary data exceeding the set time can be disconnected.
[0326] That is, according to Figure 14 If the timestamp of the publisher of the topic data in the heartbeat area SMb in the shared memory 508 is not updated, the first communication library 529 in the first application 528 can control the deletion of data in the information area SMa, the heartbeat area SMb, and the storage areas SMC~SMf regarding the topic data.
[0327] On the other hand, if the timestamp of the specified participant in the heartbeat region SMb in the shared memory 508 is not updated, the first communication library 529 in the first application 528 can be controlled to delete at least a portion of the data in the information region SMa of the corresponding participant and at least a portion of the data in the heartbeat region SMb.
[0328] Figure 15a and Figure 15b These are diagrams illustrating various examples of images displayed on a monitor.
[0329] Figure 15a An example is shown where the panoramic image is displayed on the second display 180b.
[0330] Referring to the attached diagram, the central signal processing unit 170 can run the server virtual machine 520, the first client virtual machine 540, and the second client virtual machine 550 on the hypervisor 505.
[0331] For example, the first application 528 within the server virtual machine 520 can use the internal first communication library 529 to store camera data in the storage area SMC within the shared memory 508.
[0332] On the other hand, application 548 in the first guest virtual machine 540 or application 558 in the second guest virtual machine 550 can use the internal communication libraries 549 and 559 to receive camera data in the storage area SMC in the shared memory 508.
[0333] For example, application 548 in the first guest virtual machine 540 or application 558 in the second guest virtual machine 550 can receive camera data in the storage area SMC in the shared memory 508 based on an interrupt that occurs after the camera data has been stored in the storage area SMC in the shared memory 508.
[0334] On the other hand, the application 548 within the first guest virtual machine 540 can be controlled to display camera images based on received subject data on a monitor. Thus, camera images can be displayed stably based on efficient transmission of camera data.
[0335] For example, the application 548 within the first client virtual machine 540 can be controlled to perform signal processing on camera data as subject data and display the image IGa, including the rear image 1050 and the panoramic image 1507, on the second display 180b.
[0336] Figure 15b An example is shown where the vehicle in front is displayed on the first display 180a.
[0337] Referring to the attached diagram, the central signal processing unit 170 can run the server virtual machine 520, the first client virtual machine 540, and the second client virtual machine 550 on the hypervisor 505.
[0338] For example, the first application 528 within the server virtual machine 520 can use the internal first communication library 529 to store camera data in the storage area SMC within the shared memory 508.
[0339] On the other hand, application 548 in the first guest virtual machine 540 or application 558 in the second guest virtual machine 550 can use the internal communication libraries 549 and 559 to receive camera data in the storage area SMC in the shared memory 508.
[0340] On the other hand, the application 558 within the second client virtual machine 550 can be controlled to extract a vehicle object in front from the camera image based on the received subject data and display the vehicle object IGb on the first display 180a. Thus, the object can be displayed stably based on efficient data transmission.
[0341] Figure 16a and Figure 16b This is a diagram used as a reference when explaining the content related to service quality.
[0342] Figure 16a The transmission of the second type of subject data is illustrated.
[0343] Referring to the attached diagram, the central signal processing unit 170 can run the server virtual machine 520 and the client virtual machine 550 on the hypervisor 505.
[0344] For example, a first application 528 within a server virtual machine 520 can use an internal first communication library 529 to store second-type subject data in a storage area SMC within shared memory 508.
[0345] On the other hand, the application 558 within the guest virtual machine 550 can utilize the internal communication library 559 to receive second-type topic data within the storage area SMC in the shared memory 508.
[0346] For example, application 558 within guest virtual machine 550 can receive the second type of topic data in storage area SMC within shared memory 508 based on an interrupt that occurs after the storage area SMC within shared memory 508 has finished storing the second type of topic data.
[0347] The second type of subject data at this point can include camera data. This allows for efficient transmission of camera data.
[0348] Figure 16b The transmission of the first type of subject data is illustrated.
[0349] Referring to the attached diagram, the central signal processing unit 170 can run the server virtual machine 520 and the client virtual machine 530 on the hypervisor 505.
[0350] For example, a first application 528 within a server virtual machine 520 can utilize an internal first communication library 529 to store first type subject data in a storage area SMC within a shared memory 508.
[0351] On the other hand, the application 538 within the guest virtual machine 530 can utilize the internal communication library 539 to receive first-type topic data within the storage area SMC in the shared memory 508.
[0352] For example, application 538 within guest virtual machine 530 can receive the first type of topic data in storage area SMC within shared memory 508 based on an interrupt that occurs after the first type of topic data has been stored in storage area SMC within shared memory 508.
[0353] The first type of subject data at this point can include remotely updated data. This enables efficient transmission of remotely updated data.
[0354] On the other hand, the application 538 within the guest virtual machine 530 can perform an update after receiving remote update data.
[0355] The preferred embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. Without departing from the technical concept of the present invention as claimed in the claims, those skilled in the art can make various modifications. Such modifications should not be understood separately from the technical concept or prospect of the present invention.
Claims
1. A signal processing apparatus, wherein, This includes the processor that runs the hypervisor; The processor runs a plurality of virtual machines on the hypervisor; In the plurality of virtual machines, the first virtual machine runs the first application, and the second virtual machine runs the second application; The first communication library within the first application is configured to store topic data in shared memory. The second communication library within the second application receives the topic data from the shared memory based on an interrupt.
2. The signal processing apparatus according to claim 1, wherein, The first communication library control is to store the topic data in the shared memory based on the publication; The second communication library receives the topic data from the shared memory based on a subscription.
3. The signal processing apparatus according to claim 1, wherein, When the second application and the third application are running in the second virtual machine, and the second communication library within the second application and the third communication library within the third application access the shared memory, the second communication library and the third communication library receive the topic data from the shared memory in parallel.
4. The signal processing apparatus according to claim 1, wherein, The first or second communication library sends or receives heartbeat signals from the moment it is connected to the shared memory.
5. The signal processing apparatus according to claim 1, wherein, The shared memory includes an information area, a heartbeat area, and a storage area for the topic data.
6. The signal processing apparatus according to claim 5, wherein, The first communication library control is as follows: if no first topic data is registered in the storage area within the shared memory, the first topic data is stored in the free area of the storage area, information related to the first topic data is recorded in the information area, and timestamp information is recorded in the heartbeat area.
7. The signal processing apparatus according to claim 6, wherein, The first communication library control is to periodically update the timestamp information in the heartbeat region.
8. The signal processing apparatus according to claim 5, wherein, The second communication library control is to receive information in the information area within the shared memory, and if it is confirmed based on the received information that the first topic data has been registered in the storage area, receive the first topic data stored in the storage area, update the information related to the first topic data in the information area, and update the timestamp information in the heartbeat area.
9. The signal processing apparatus according to claim 8, wherein, The second communication library control is to periodically update the timestamp information in the heartbeat region.
10. The signal processing apparatus according to claim 5, wherein, The first communication library control is to delete the information area, the heartbeat area, and the data in the storage area related to the topic data if the timestamp of the publisher of the topic data in the heartbeat area of the shared memory is not updated.
11. The signal processing apparatus according to claim 5, wherein, The first communication library control is to delete at least a portion of the data in the information area of the corresponding participant and at least a portion of the data in the heartbeat area if the timestamp of the specified participant in the heartbeat area of the shared memory is not updated.
12. The signal processing apparatus according to claim 1, wherein, The first communication library outputs a first interrupt when storing or updating the topic data in the shared memory; The second communication library receives the topic data from the shared memory based on the first interrupt.
13. The signal processing apparatus according to claim 1, wherein, The second communication library outputs a second interrupt upon receiving the topic data from the shared memory; After receiving the second interrupt, the first communication library stores or updates subsequent topic data to the shared memory.
14. The signal processing apparatus according to claim 1, wherein, In the case that the topic data is of the first type, the first communication library will store or update subsequent data to the shared memory after the second application or all subscribers have completed receiving the topic data of the first type. In the case that the topic data is of the second type, the first communication library stores or updates subsequent data to the shared memory before the second application or all subscribers have finished receiving the first type of topic data.
15. The signal processing apparatus according to claim 1, wherein, In the case where the topic data is first topic data with first priority, the first communication library performs the transmission of the first topic data and the output of the first interrupt during the first time slot; If the topic data is a second topic data with a lower priority than the first topic data, the first communication library performs the transmission of the second topic data or the output of the second interrupt after the first time slot.
16. The signal processing apparatus according to claim 1, wherein, If the topic data is a first topic data of first priority, and the interruption of the first topic data occurs before the first time slot period which is the first priority, the first communication library performs the transmission of the first topic data and the output of the first interruption during the first time slot period.
17. The signal processing apparatus according to claim 1, wherein, Each application running in the plurality of virtual machines runs a communication library for publishing or subscribing to the shared memory.
18. The signal processing apparatus according to claim 1, wherein, The second application control is to display camera images based on the topic data on a monitor.
19. The signal processing apparatus according to claim 1, wherein, Among the plurality of virtual machines, the third virtual machine runs the third application; The third communication library within the third application receives the topic data from the shared memory based on an interrupt and controls the display of objects based on the topic data.
20. A display device for a vehicle, wherein, The signal processing apparatus includes any one of claims 1 to 19.
Citation Information
Patent Citations
Vehicle with multiple user interface operating domains
EP3092566A1
