Information processing device, information processing method, and information processing system

Abstract

A processor (100): acquires a display image from a first semiconductor device (6) that executes image processing of a display image to be displayed on a display device (4); acquires safety display information for making a display related to safety from a second semiconductor device (5) that is a semiconductor device different from the first semiconductor device (6); outputs the display image acquired from the first semiconductor device (6) to the display device (4); generates, on the basis of the safety display information acquired from the second semiconductor device (5), a safety information image including information on safety; and outputs the safety information image to the display device (4).

Description

Information processing device, information processing method, and information processing system

[0001] The present invention relates to an information processing device, an information processing method, and an information processing system.

[0002] A technology is known in which, during normal operation, a first microcomputer generates a display image from a vehicle information image and an image of information other than vehicle information obtained from a second microcomputer, outputs the image to a display device, and displays the vehicle information image on a display device if an abnormality is detected in the operation of the second microcomputer (Patent Document 1).

[0003] JP 2014-88128 A

[0004] However, the technology described in Patent Document 1 is a technology that prevents inappropriate images associated with an abnormality from being displayed when an abnormality is detected in the second microcomputer, and there is a problem in that if an abnormality occurs in the first microcomputer, safety-related images cannot be displayed while the first microcomputer is being restored from the abnormal state.

[0005] The problem that the present invention aims to solve is to provide an information processing device, an information processing method, and an information processing system that, when an abnormality occurs in a semiconductor device that performs image processing of a display image to be displayed on a display device, can display a safety-related image on the display device while the semiconductor device is being restored from the abnormal state.

[0006] The present invention solves the above problem by obtaining a display image from a first semiconductor device that performs image processing of the display image to be displayed on a display device, obtaining safety display information for displaying safety information from a second semiconductor device that is a semiconductor device different from the first semiconductor device, outputting the display image to the display device, and generating a safety information image containing information related to safety based on the safety display information and outputting it to the display device.

[0007] According to the present invention, when an abnormality occurs in a semiconductor device that performs image processing of a display image to be displayed on a display device, a safety-related image can be displayed on the display device while the semiconductor device is being restored from the abnormal state.

[0008] FIG. 1 is a block diagram showing the configuration of an information processing system including an information processing device according to a first embodiment of the present invention. FIG. 2 is a diagram illustrating switching control of an image displayed on a display device according to the first embodiment. FIG. 3 is a diagram illustrating processing in each functional block of a processor according to the first embodiment. FIG. 4 is a diagram showing an example of a flowchart of control processing of an information processing method executed by the information processing device according to the first embodiment. FIG. 5 is a diagram showing an example of a flowchart of control processing of an abnormality detection processing executed by a microcontroller according to the first embodiment. FIG. 6 is a diagram showing an example of a flowchart of control processing of a return display processing executed by a microcontroller according to the first embodiment. FIG. 7 is a diagram showing an example of a flowchart of control processing of a display image output processing executed by a system-on-chip according to the first embodiment. FIG. 8 is a diagram showing an example of a flowchart of control processing of a return processing executed by a system-on-chip according to the first embodiment. FIG. 9 is a diagram showing an example of a screen of a display device according to a second embodiment of the present invention. FIG. 10 is a diagram showing an example of a flowchart of control processing of an information processing method executed by an information processing device according to the second embodiment of the present invention.

[0009] Hereinafter, an information processing device, an information processing method, and an information processing system according to an embodiment of the present invention will be described with reference to the drawings. In the present embodiment, an example in which the information processing device is provided in a vehicle will be described.

[0010] First Embodiment An information processing system including an information processing device according to a first embodiment of the present invention will be described with reference to FIG. 1 . The information processing system is a system that executes information processing for displaying images to a user. FIG. 1 is a block diagram showing the configuration of the information processing system including the information processing device according to the first embodiment. The information processing system 1 includes an ECU 2, a CAN bus 3, a display device 4, and an information processing device 10. These devices are mounted on a vehicle, connected via communication means, and can transmit and receive information to and from each other. The information processing device 10 acquires information from the ECU 2 and outputs an image to the display device 4. The information acquired by the information processing device 10 from the ECU 2 may be an image or information other than an image. The information other than an image is information with a smaller data volume than an image, such as identification information for identifying an image. A user is a vehicle occupant, primarily the driver of the vehicle, but the user may also be a passenger other than the driver. In this embodiment, the information processing system 1 is described as being applied to an example of displaying an image to a user in a vehicle, but the information processing system 1 of this embodiment can be applied to anything that displays an image to a user, and is not limited to applications in vehicles.

[0011] The ECU 2 is an electronic control unit (ECU). The ECU 2 is one of the on-board ECUs and controls the image display on the display device 4. The image displayed on the display device 4 includes, for example, vehicle information and information other than the vehicle information to be provided to the user. The vehicle information is information related to the vehicle's status, such as vehicle meter information. The information other than the vehicle information to be provided to the user is, for example, infotainment information including navigation information and entertainment information. The ECU 2 is connected to an on-board network, such as a CAN (Controller Area Network), and can exchange information with other on-board ECUs (not shown) using a CAN bus 3 or the like. The other on-board ECUs include, for example, a body ECU, a driving ECU, a power supply ECU, and a multimedia ECU. The ECU 2 acquires information from the other on-board ECUs, performs image processing to generate a display image, and outputs the generated display image to the display device 4 via the information processing device 10.

[0012] The ECU 2 includes two semiconductor devices: a system-on-chip 5 and a microcontroller 6. The system-on-chip 5 and the microcontroller 6 are different semiconductor devices having different functions. HDMI (registered trademark) is used as an image input / output interface between the system-on-chip 5 and the information processing device 10 and between the information processing device 10 and the display device 4. SPI communication is used for information communication between the microcontroller 6 and the information processing device 10.

[0013] Furthermore, the system-on-chip 5 and the microcontroller 6 are semiconductor devices different from the information processing device 10, and each has a different function. By dividing the functions into devices in this way, as will be described below, even if an abnormality occurs in one of the devices, the display device 4 can continue to display information regarding safety. The system-on-chip 5 and the microcontroller 6 are examples of a "first semiconductor device" and a "second semiconductor device" respectively as set forth in the claims.

[0014] In this embodiment, the first semiconductor device and the second semiconductor device are described as a system-on-chip and a microcontroller, but these semiconductor devices are not limited to this as long as they have the functions to realize each process. For example, the semiconductor device may be a device that includes multiple semiconductor chips for realizing each function, and the semiconductor chips are connected to each other.

[0015] The system-on-chip 5 is a semiconductor device (system on a chip) that includes a CPU and peripheral circuits such as memory and is designed to function as a system that executes multiple processes on a single chip. The memory stores programs for implementing the software of the system-on-chip 5. The software of the system-on-chip 5 is implemented by the CPU executing the programs stored in the memory to perform various processes. The system-on-chip 5 is a semiconductor device with higher processing capabilities than the microcontroller 6. In other words, the system-on-chip 5 processes information on a larger amount of data than the information processed by the microcontroller 6.

[0016] The system-on-chip 5 is a device that has an image processing function and controls image display on the display device 4 under normal circumstances. For example, the system-on-chip 5 is configured with a GPU (Graphics Processing Unit). The system-on-chip 5 executes each of the above functions or processes through cooperation between the software and hardware to implement each function. In this embodiment, the system-on-chip 5 executes image processing for generating a display image to be displayed on the display device 4 using the image processing function. The system-on-chip 5 outputs the generated display image to the display device 4 via the information processing device 10.

[0017] In this embodiment, the display image generated by the system-on-chip 5 is an image including vehicle information and information other than the vehicle information to be provided to the user. The vehicle information includes, for example, vehicle speed information and vehicle energy information. The display image may also include safety information. The information other than the vehicle information to be provided to the user is, for example, so-called infotainment information including navigation information and entertainment information. The navigation information is information showing route guidance for the vehicle. The navigation information includes, for example, map information, the current position of the vehicle, route information from the current position to the destination, and guidance information for the vehicle to travel the route. The entertainment information is information including entertainment content such as music and videos.

[0018] The system-on-chip 5 acquires information output from other vehicle ECUs via the microcontroller 6. The information output from the other vehicle ECUs includes, for example, vehicle information including the vehicle status and infotainment information. Infotainment information is information output from a multimedia ECU that controls the vehicle's navigation system and / or content playback system. The system-on-chip 5 generates a display image including vehicle information and infotainment information based on various information acquired from the other vehicle ECUs. The display image is an image with a larger amount of data than the safety display information described below. For example, the display image is an image with a large amount of data, such as a moving image or a color image. More specifically, the display image is a 3D image or an animation. That is, the system-on-chip 5 performs information processing for video with a large amount of data, such as video containing entertainment content such as music and videos. The display image is not limited to including vehicle information, and may also be an image that does not include vehicle information and includes information provided to the user other than vehicle information.

[0019] Furthermore, in this embodiment, the system-on-chip 5 executes the restoration process when restoration instruction information for executing the restoration process is input from the microcontroller 6. Specifically, the system-on-chip 5 restarts the system. When the restoration process is completed, the system-on-chip 5 outputs restoration completion information indicating the completion of the restoration process to the microcontroller 6. When the restoration process is completed, the system-on-chip 5 also resumes image processing of the display image. The system-on-chip 5 generates a display image by image processing, and when preparation of the display image is complete, outputs the display image to the display device 4 via the information processing device 10.

[0020] The microcontroller 6 is a so-called MCU (microcontroller unit), which is a semiconductor device in which a CPU and peripheral circuits such as memory are integrated on a single chip. The memory stores programs for implementing the software of the microcontroller 6. The software of the microcontroller 6 is implemented by the CPU executing the programs stored in the memory to perform various processes. The microcontroller 6 has a power management function for managing the power supply of the ECU 2 and a CAN communication function for communicating with other in-vehicle ECUs via the CAN bus 3. The microcontroller 6 executes each function through cooperation between the software for implementing each of the above functions or executing each process and the hardware.

[0021] In this embodiment, the microcontroller 6 acquires various types of information from other in-vehicle ECUs via the CAN communication function. The acquired information includes vehicle information including the vehicle status and / or infotainment-related information. For example, the vehicle information includes meter information such as vehicle speed information and vehicle energy information. The vehicle information also includes safety-related vehicle status. The safety-related vehicle status includes information regarding the status of safety devices such as brakes, ABS, airbags, and seat belts. When the system-on-chip 5 is normal, the microcontroller 6 outputs the information acquired from the other in-vehicle ECUs to the system-on-chip 5.

[0022] In the present embodiment, the microcontroller 6 also has a safety display function. When the system-on-chip 5 is abnormal, the microcontroller 6 generates safety display information for displaying a safety-related display using the safety display function and outputs the safety display information to the information processing device 10. For example, when abnormality detection information indicating an abnormality in the display image is input from the information processing device 10, the microcontroller 6 generates the safety display information based on the vehicle information acquired from the CAN bus 3. The safety display information includes, for example, instruction information for instructing the safety information image to be displayed on the display device 4. In the present embodiment, a corresponding safety information image is associated with each vehicle state. The safety information image is an image including safety information. The safety information is information necessary for the user to make safety decisions, and is information regarding the vehicle state, particularly information warning that there is an abnormality in the vehicle state. For example, a vehicle state in which there is an abnormality in the vehicle brakes is associated with a safety information image including a warning display such as a telltale warning that there is an abnormality in the brakes. Furthermore, safety information images including a warning display that warns that the vehicle door is open and a warning display that warns that the vehicle energy is insufficient are linked to vehicle states such as a vehicle door being open and a vehicle having insufficient remaining energy, respectively. Furthermore, the vehicle state may be the vehicle speed. A safety information image indicating the vehicle speed is linked to each vehicle speed.

[0023] In this embodiment, the microcontroller 6 may include a memory that stores identification information for identifying a safety information image corresponding to each vehicle state. The identification information may be, for example, an ID number assigned to each safety information image. The safety display information is not the safety information image itself, but identification information for identifying the safety information image, and has a smaller data volume than the display image. Therefore, the processing load on the microcontroller 6 related to the generation and output processes of the safety display information can be reduced, so that even if an abnormality occurs in the system-on-chip 5, the safety display information can be output to the information processing device 10 at high speed. Note that in this embodiment, the microcontroller 6 may store the safety information image in its memory and output the safety information image to the information processing device 10 as the safety display information instead of the instruction information or identification information. Even in this case, the safety information image has a smaller data volume than the display image. For example, the safety information image may be an image of a number such as the vehicle speed, a symbol indicating meter information, or an icon indicating a warning display.

[0024] As described above, the microcontroller 6 identifies a corresponding safety information image according to the vehicle state, and generates instruction information including identification information for identifying the identified safety information image as safety display information. For example, if there is a safety device with an abnormality, the microcontroller 6 generates instruction information that instructs the display of a safety information image including a warning display that warns of the abnormality in the safety device, based on vehicle information related to the state of the vehicle's safety devices. The generated safety display information is output to the information processing device 10.

[0025] In this embodiment, the microcontroller 6 also has a recovery processing function. The microcontroller 6 executes recovery processing of the system-on-chip 5 by using the recovery processing function. In the recovery processing, the microcontroller 6 restarts the system-on-chip 5, which is in an abnormal state where an abnormality has occurred. Specifically, when the microcontroller 6 receives abnormality detection information from the information processing device 10, it outputs recovery instruction information to the system-on-chip 5 to cause the system-on-chip 5 to execute the recovery processing. When the microcontroller 6 receives recovery completion information from the system-on-chip 5 indicating that the recovery processing has been completed, it outputs safety display stop information to the information processing device 10 to instruct the information processing device 10 to stop displaying safety information.

[0026] The display device 4 is a display that displays images to the user. The display device 4 is configured, for example, with a liquid crystal display or an organic EL display. The display device 4 may also include a speaker that outputs audio information. In this embodiment, the display device 4 is a display that is disposed in front of the driver's seat of the vehicle, spanning from the driver's seat to the passenger seat. The display device 4 is a display that includes a so-called instrument panel.

[0027] The display device 4 displays a display image including, for example, vehicle information and information other than the vehicle information to be provided to the user. The vehicle information is, for example, information about the vehicle's status, including vehicle speed information and vehicle energy information. The display image may also include safety information. The information other than the vehicle information to be provided to the user is, for example, so-called infotainment information, including navigation information and entertainment information. The navigation information is information that provides route guidance for the vehicle. The navigation information includes, for example, map information, the current position of the vehicle, route information from the current position to the destination, and guidance information for the vehicle to travel along the route. The entertainment information is information that includes entertainment content such as music and videos. The display image displayed on the display device 4 is, for example, a moving image or a color image.

[0028] The display device 4 has a normal display function for displaying a display image generated by the system-on-chip 5 and an on-screen display (OSD) display function for displaying an OSD image generated by the information processing device 10. In normal operation (when the system-on-chip 5 is in a normal state), the display device 4 displays a display image by the normal display function when a display image is output as an image signal from the system-on-chip 5 via the information processing device 10. In abnormal operation (when the system-on-chip 5 is in an abnormal state), the display device 4 displays a safety information image by the OSD display function when a safety information image generated by the information processing device 10 is output as an image signal.

[0029] In conventional technology, a semiconductor device normally processes display images including vehicle information and infotainment information, and a display device can present vehicle information and infotainment information to a user by displaying the display image output from the semiconductor device. However, if an abnormality occurs in the semiconductor device and the display image output from the semiconductor device is abnormal, the display device may display a black screen or freeze. The semiconductor device in which the abnormality occurred performs a system restart and performs a recovery process. While the recovery process of the semiconductor device is being performed, the display device continues to display a black screen or freeze. Therefore, there is a problem that safety displays such as warnings, vehicle speed, etc. are not displayed on the display device 4 during the recovery process of the semiconductor device. In particular, because the processing load required for restarting is large and the recovery process takes time, the display device continues to display a black screen or freeze for a long time.

[0030] In this embodiment, a semiconductor device (microcontroller 6) different from the semiconductor device (system-on-chip 5) that performs image processing of the display image outputs safety display information for displaying safety information to the information processing device 10, so that even during recovery processing of the semiconductor device that performs image processing of the display image, the information processing device 10 generates an image including safety information and outputs it to the display device 4. As a result, if an abnormality occurs in the semiconductor device that performs image processing of the display image, a safety-related image can be displayed on the display device while the semiconductor device is being recovered from the abnormal state.

[0031] The information processing device 10 according to this embodiment is a device that processes information to display an image on the display device 4. In this embodiment, the information processing device 10 is connected between the system-on-chip 5 and the display device 4, and detects an abnormality in the display image output from the system-on-chip 5 to the display device 4. Furthermore, if there is an abnormality in the display image, the information processing device 10 generates a safety information image as an OSD image and outputs it to the display device 4.

[0032] The information processing device 10 includes a memory 7 and a processor 100. The memory 7 is a storage medium that stores information for generating an OSD image. For example, the memory 7 is a flash memory that stores a safety information image associated with identification information. The processor 100 is configured as a computer having hardware and software, including a ROM that stores programs, a CPU that executes the programs stored in the ROM, and a RAM that functions as an accessible storage device. The processor 100 includes at least an abnormality detection unit 101, an OSD image generation unit 102, and an output unit 103 as functional blocks. In this embodiment, the functions of the processor 100 are divided into three blocks and the functions of each functional block are described. However, the functions of the processor 100 do not necessarily have to be divided into three blocks; they may be divided into two or fewer functional blocks, or four or more functional blocks. Furthermore, the information processing device 10 may be two different devices. For example, the information processing device 10 includes a device including an abnormality detection unit 101 and a device including an OSD image generation unit 102 .

[0033] The anomaly detection unit 101 executes an anomaly detection process to detect an anomaly in the display image output from the system-on-chip 5 to the display device 4. In the anomaly detection process, the anomaly detection unit 101 acquires the display image from the system-on-chip 5 and detects whether or not there is an anomaly in the acquired display image. If the anomaly detection unit 101 detects that there is an anomaly in the display image, it outputs anomaly detection information indicating that there is an anomaly in the display image to the microcontroller 6. The anomaly detection unit 101 detects whether or not there is an anomaly in the display image, for example, by checking for a communication error in the display image, freezing of the display image, etc.

[0034] The OSD image generation unit 102 executes an OSD image generation process for generating a safety information image as an OSD image. In the OSD image generation process, the OSD image generation unit 102 first acquires safety display information from the microcontroller 6. The safety display information is information for displaying safety information. The safety display information is output from the microcontroller 6 when the microcontroller 6 acquires abnormality detection information. When the safety display information is input from the microcontroller 6, the OSD image generation unit 102 generates a safety information image including safety-related information as an OSD image based on the safety display information. In this embodiment, the information processing device 10 includes a flash memory. The flash memory stores safety information images in association with identification information for identifying each image. The OSD image generation unit 102 generates a safety information image by reading the safety information image corresponding to the identification information from the flash memory in accordance with the identification information included in the acquired safety display information. The safety information image has a smaller data volume than a display image. For example, the safety information image is an image of a number such as the vehicle speed, a symbol indicating meter information, or an icon indicating a warning display.

[0035] The output unit 103 outputs an image to the display device 4. When a display image is acquired from the system-on-chip 5, the output unit 103 outputs the acquired display image to the display device 4. When a safety information image is generated by the OSD image generation unit 102, the output unit 103 outputs the generated safety information image to the display device 4. That is, in this embodiment, when there is an abnormality in the display image, the output unit 103 does not output the display image to the display device 4, but outputs the safety information image to the display device 4. Furthermore, when there is no abnormality in the display image, the output unit 103 outputs the display image to the display device 4, but does not output the safety information image to the display device 4.

[0036] Here, switching control of images displayed on the display device according to this embodiment will be described with reference to FIG. 2. FIG. 2 is a diagram for explaining switching control of images displayed on the display device according to this embodiment. In the example of FIG. 2, the upper row shows an example of processing of an image displayed on the display device 4 (foreground processing), and the lower row shows an example of processing of an image not displayed on the display device 4 (background processing). In FIG. 2, the processing contents of the foreground processing and the background processing are shown in chronological order (from the first scene to the fourth scene) from left to right of the figure. In the first scene under normal conditions, in the foreground processing, the system-on-chip 5 outputs a display image D1 to the display device 4. The display image D1 is displayed on the display device 4. The display of the display image D1 is a normal display. The display image D1 includes vehicle information and infotainment-related information. At this time, in the background processing, the microcontroller 6 receives CAN signals including warning displays such as telltales and vehicle information such as speed displays, and prepares to immediately display safety information images as necessary.

[0037] When an abnormality is detected in the display image displayed on the display device 4 (second scene), in the foreground processing, a safety information image D2 generated based on safety display information from the microcontroller 6 is displayed on the display device 4. The display of the safety information image is a display for the abnormality. The safety information image D2 includes safety information such as the vehicle speed. The safety information image D2 is an image with a smaller amount of data than the display image D1. At this time, in the background processing, the microcontroller 6, having acquired the abnormality detection information from the information processing device 10, outputs restoration instruction information to the system-on-chip 5, thereby executing restoration processing of the system-on-chip 5. During the restoration processing, in the foreground, the display device 4 continues to display the safety information image D2. Thereafter, as shown in the third scene, in the background processing, the system-on-chip 5 completes the restoration processing and prepares the display image D1'. Until this point, in the foreground, the display device 4 continues to display the safety information image D2. Thereafter, as shown in the fourth scene, in the foreground processing, the display image D1' prepared in the background processing is displayed on the display device 4. As described above, in this embodiment, safety display information is output from the microcontroller 6 to the information processing device 10 so that the screen does not freeze or a black screen is not displayed, and the image displayed on the display device 4 is switched.

[0038] Next, an example of an information processing method according to this embodiment will be described with reference to FIG. 3 . FIG. 3 is a diagram for explaining the processing in each functional block of the processor according to this embodiment. In this embodiment, a display image is output from the system-on-chip 5 to the anomaly detection unit 101. The anomaly detection unit 101 detects an anomaly in the display image. The detected anomaly is, for example, an image anomaly such as a communication error or freezing. If no anomaly is detected in the display image, i.e., if the display image is normal, the anomaly detection unit 101 outputs the display image to the output unit 103. If a display image is input, the output unit 103 outputs the display image to the display device 4. As such, under normal circumstances, the display image is output from the system-on-chip 5 to the display device via the information processing device 10. If an anomaly is detected in the display image, the anomaly detection unit 101 outputs anomaly detection information to the microcontroller 6. At this time, the anomaly detection unit 101 stops outputting the display image to the output unit 103.

[0039] When the microcontroller 6 receives abnormality detection information from the abnormality detection unit 101, it outputs safety display information to the OSD image generation unit 102. When the OSD image generation unit 102 receives the safety display information, it generates a safety information image based on the safety display information. For example, the OSD image generation unit 102 reads a safety information image specified by the instruction information from the memory 7 based on instruction information included in the safety display information, and generates an OSD image. The instruction information includes identification information for identifying the safety information image to be displayed on the display device 4. The memory 7 stores safety information images in association with the identification information. The OSD image generation unit 102 reads the safety information image corresponding to the identification information from the memory 7 and stores it in the SRAM. The OSD image generation unit 102 reads and writes information to and from the SRAM, thereby enabling high-speed information processing with low load. In other words, the information processing device 10 does not need to start an OS, load a library for image display into RAM memory, and prepare a display image with a large amount of data, as in the system-on-chip 5.

[0040] The OSD image generation unit 102 outputs the generated safety information image to the abnormality detection unit 101. The abnormality detection unit 101 detects whether or not there is an abnormality in the safety information image, just like the display image, and outputs the safety information image to the output unit 103 if no abnormality is detected in the safety information image. The output unit 103 outputs the safety information image to the display device 4 if the safety information image is input. Furthermore, when the microcontroller 6 acquires abnormality detection information from the abnormality detection unit 101, it outputs recovery instruction information to the system-on-chip 5. In response to this, the system-on-chip 5 executes recovery processing to recover from the abnormal state. When the recovery processing is completed, the system-on-chip 5 outputs recovery processing completion information to the microcontroller 6 and re-executes image processing of the display image and outputs it to the abnormality detection unit 101. When the microcontroller 6 acquires the recovery completion information from the system-on-chip 5, it outputs safety display stop information to the OSD image generation unit. When the safety display stop information is input, the OSD image generation unit 102 ends the generation of the OSD image.

[0041] Next, an example of a control procedure for executing the information processing method according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of a flowchart of the control process of the information processing method executed by the information processing device according to this embodiment. When the system-on-chip 5 is started, the processor 100 starts the flow from step S101.

[0042] In step S101, the processor 100 acquires a display image to be displayed on the display device 4 from the system-on-chip 5. The display image is generated by image processing of the system-on-chip 5 and output to the processor 100. The display image is an image that includes vehicle information and information other than the vehicle information to be provided to the user. In step S102, the processor 100 detects whether or not there is an abnormality in the acquired display image. If the processor 100 detects that there is an abnormality in the display image in step S102, the processor 100 proceeds to step S103. If the processor 100 detects that there is no abnormality in the display image in step S102, the processor 100 proceeds to step S107.

[0043] In step S103, the processor 100 outputs abnormality detection information indicating that there is an abnormality in the display image to the microcontroller 6. In step S104, the processor 100 acquires safety display information from the microcontroller 6 for displaying a safety-related display. The safety display information is information that is output from the microcontroller 6 to the processor 100 when the microcontroller 6 acquires abnormality detection information. In step S105, the processor 100 generates a safety information image based on the safety display information. In step S106, the processor 100 outputs the safety information image to the display device 4. In step S107, the processor 100 outputs the display image acquired in step S101 to the display device 4. When an image is input, the display device 4 displays the input image.

[0044] Next, an example of a control procedure for executing the abnormality detection process by the microcontroller 6 according to this embodiment will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of a flowchart of the control procedure for the abnormality detection process executed by the microcontroller 6. The microcontroller 6 starts the flow from step S201 when detection information indicating that the state of the display image has been detected is input from the information processing device 10.

[0045] In step S201, the microcontroller 6 acquires detection information indicating the state of the display image from the information processing device 10. The detection information is abnormality detection information indicating that the display image has an abnormality, or normality detection information indicating that the display image is normal (normal). In step S202, the microcontroller 6 determines whether or not the display image has an abnormality. If the microcontroller 6 acquires abnormality detection information, it determines that the display image has an abnormality. If the microcontroller 6 acquires normality detection information, it determines that the display image has no abnormality. If it determines that the display image has an abnormality, the microcontroller 6 proceeds to step S203. If it determines that the display image has no abnormality (normal), the microcontroller 6 terminates the abnormality detection process. In step S203, the microcontroller 6 outputs recovery instruction information to the system-on-chip 5. In step S204, the microcontroller 6 sets the abnormality display flag to ON. In step S205, the microcontroller 6 sets the recovery flag to OFF. After the process of step S205 is completed, the microcontroller 6 terminates the abnormality detection process.

[0046] Next, an example of a control procedure for executing the display restoration process by the microcontroller according to this embodiment will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of a flowchart of the control procedure for the display restoration process executed by the microcontroller 6 according to this embodiment. The microcontroller 6 checks the abnormality display flag at regular intervals and starts the flow from step S301.

[0047] In step S301, the microcontroller 6 determines whether the abnormality display flag is on. If it is determined that the abnormality display flag is on, the microcontroller 6 proceeds to step S302. If it is determined that the abnormality display flag is not on (off), the microcontroller 6 proceeds to step S309. In step S302, the microcontroller 6 changes the output settings. Specifically, the microcontroller 6 sets the OSD output to on and sets the image output of the display image to off. In step S303, the microcontroller 6 filters the CAN signal acquired from the CAN bus 3. In step S304, the microcontroller 6 performs abnormality display processing based on the filtered CAN signal. In the abnormality display processing, the microcontroller 6 outputs safety display information to the information processing device 10. In step S305, the microcontroller 6 sets a recovery flag. Specifically, when the microcontroller 6 receives setting information regarding the recovery flag setting, for example, recovery completion information, from the system-on-chip 5, the microcontroller 6 sets the recovery flag to on.

[0048] In step S306, the microcontroller 6 determines whether the return flag is on. If it is determined that the return flag is on, the microcontroller 6 proceeds to step S307. If it is determined that the return flag is not on (off), the microcontroller 6 proceeds to step S309. In step S307, the microcontroller 6 changes the output settings. Specifically, the microcontroller 6 sets the OSD output to off and sets the image output of the display image to on. In step S308, the microcontroller 6 sets the abnormality display flag to off. In step S309, the microcontroller 6 clears the watchdog. In this embodiment, the return display process by the microcontroller 6 is a simple process specialized for abnormality display, which is independent from other processes. This ensures that the abnormality display is executed more reliably when an abnormality is detected in the display image. Furthermore, by including a process to clear the watchdog in the return display process, even if an abnormality occurs in the return display process itself, the watchdog can detect the abnormality and restart the microcontroller 6 to restore the display.

[0049] Next, an example of a control procedure for executing the display image output process by the system-on-chip according to this embodiment will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of a flowchart of the control procedure for the display image output process executed by the system-on-chip according to this embodiment. When the ECU 2 is started up, the system-on-chip 5 starts the flow from step S401.

[0050] In step S401, the system-on-chip 5 receives a CAN signal from the microcontroller 6. The CAN signal includes vehicle information and infotainment-related information. In step S402, the system-on-chip 5 performs image processing of a display image based on the CAN signal. In step S403, the system-on-chip 5 outputs the generated display image to the display device 4 via the information processing device 10.

[0051] Next, an example of a control procedure for executing the restoration process by the system-on-chip according to this embodiment will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of a flowchart of the control procedure for the restoration process executed by the system-on-chip according to this embodiment. When the system-on-chip 5 acquires restoration instruction information from the microcontroller 6, the system-on-chip 5 starts the flow from step S501.

[0052] In step S501, the system-on-chip 5 executes reboot processing. In step S502, the system-on-chip 5 loads a graphics driver. In step S503, the system-on-chip 5 loads a graphics library. In step S504, the system-on-chip 5 prepares graphics data. In step S505, the system-on-chip 5 outputs setting information related to the setting of the return flag, for example, return completion information, to the microcontroller 6. In step S506, the system-on-chip 5 executes display image output processing. The display image output processing is executed according to the flow of FIG. 7. When the display image output processing starts, the system-on-chip 5 ends the return processing.

[0053] Next, as another example of the present embodiment, an example of switching control for switching the image displayed on the display device 4 from a safety information image to a display image after execution of the restoration process will be described. In this embodiment, the switching control is control for stopping the display of the safety information image on the display device 4 after acquiring the display image output from the system-on-chip 5, and outputting the acquired display image to the display device 4. That is, when the information processing device 10 acquires restoration completion information indicating that the restoration process is completed from the microcontroller 6, it outputs safety display stop information to the information processing device 10 on the condition that the display image has been acquired. Then, the information processing device 10 outputs the display image to the display device 4 after outputting the safety display stop information. This shortens the time from outputting the safety display stop information to outputting the display image to the display device 4 compared to acquiring the display image from the system-on-chip 5 and outputting the display image to the display device 4 after outputting the safety display stop information, thereby shortening the time for which a black screen or the like is displayed.

[0054] Furthermore, in the present embodiment, an example of switching control may be control in which the display device 4 performs a fade-out process to gradually erase the safety information image, and then performs a fade-in process to gradually display the display image. This prevents the display device 4 from suddenly switching from the safety information image to the display image, which would cause discomfort to the user. Specifically, when the return flag is on, the information processing device 10 performs the fade-out process by gradually decreasing the alpha value of the safety information image. Then, the information processing device 10 sets the output of the display image to on, and performs the fade-in process by gradually increasing the alpha value of the display image.

[0055] As described above, in the information processing device, information processing method, and information processing system according to the present embodiment, the processor acquires a display image from a first semiconductor device that executes image processing of the display image to be displayed on the display device, acquires safety display information for displaying safety information from a second semiconductor device that is a semiconductor device different from the first semiconductor device, outputs the display image acquired from the first semiconductor device to the display device, and generates a safety information image including safety information based on the safety display information acquired from the second semiconductor device and outputs it to the display device. As a result, if an abnormality occurs in the semiconductor device that executes image processing of the display image to be displayed on the display device, the safety-related image can be displayed on the display device while the semiconductor device is being restored from the abnormal state.

[0056] Furthermore, in the information processing device, information processing method, and information processing system according to the present embodiment, the processor determines whether or not there is an abnormality in the display image acquired from the first semiconductor device, and if it determines that there is an abnormality in the display image, outputs abnormality detection information to the second semiconductor device, and the safety display information is information that is output from the second semiconductor device to the processor when the second semiconductor device acquires the abnormality detection information. As a result, if an abnormality occurs in the semiconductor device that performs image processing of the display image to be displayed on the display device, a safety-related image can be displayed on the display device while the semiconductor device is being restored from the abnormal state.

[0057] In the information processing device, information processing method, and information processing system according to the present embodiment, the safety display information has a smaller data volume than the display image, so that if an abnormality occurs in the semiconductor device that processes the display image, the load on the information processing can be reduced and the safety-related image can be displayed on the display device at high speed.

[0058] In the information processing device, the information processing method, and the information processing system according to the present embodiment, the display image is a moving image or a color image, so that the display device can display the moving image or the color image when no abnormality occurs in the semiconductor device that executes the generation process and output process of the display image to be displayed on the display device.

[0059] In the information processing device, the information processing method, and the information processing system according to the present embodiment, the safety display information includes instruction information for instructing the display device to display a safety information image, thereby allowing the display device to display the instructed image.

[0060] Furthermore, in the information processing device, information processing method, and information processing system according to the present embodiment, the safety information image has a smaller data volume than the display image, which reduces the load on information processing when an abnormality occurs in the semiconductor device that processes the display image, allowing the safety-related image to be displayed on the display device at high speed.

[0061] Furthermore, in the information processing device, information processing method, and information processing system according to the present embodiment, when the restoration process of the first semiconductor device based on the restoration instruction information output from the second semiconductor device is executed, the processor executes switching control to switch the image to be displayed on the display device from the safety information image to the display image, and the switching control is control to stop displaying the safety information image on the display device after acquiring the display image output from the first semiconductor device, and output the acquired display image to the display device. This makes it possible to prevent a black screen from being displayed on the display device due to a timing discrepancy during switching.

[0062] Furthermore, in the information processing device, information processing method, and information processing system according to the present embodiment, when the restoration process of the first semiconductor device is executed based on the restoration instruction information output from the second semiconductor device, the processor executes switching control to switch the image displayed on the display device from the safety information image to the display image, and the switching control is control to execute a fade-out process on the display device to gradually erase the safety information image, and then a fade-in process to gradually display the display image, thereby preventing the user from feeling uncomfortable due to a sudden switch from the safety information image to the display image.

[0063] <<Second Embodiment>> In the first embodiment, the display device 4 switches between the display image and the safety information image on a single screen, but this is not limited thereto. The display device 4 may have a plurality of display areas, and the display may be controlled independently in each display area. In the second embodiment, the configuration other than the following configuration is the same as that of the first embodiment described above. In the following description, the same configuration and control process as in the first embodiment will not be described, but the description of the first embodiment will be used as appropriate for the omitted description.

[0064] In the second embodiment, the display device 4 has a first display area and a second display area different from the first display area. A display image output from the system-on-chip 5 is displayed in the first display area of ​​the display device 4, and a safety information image based on safety display information output from the microcontroller 6 is displayed in the second display area of ​​the display device 4. In this embodiment, the display image and the safety information image are independently output to the display device 4. The display image is output from the system-on-chip 5 to the display device 4 via the information processing device 10. Specifically, the system-on-chip 5 outputs the display image to the display device 4 to be displayed in the first display area. When the safety display information is output from the microcontroller 6 to the information processing device 10, the safety information image is generated based on the safety display information and output from the information processing device 10 to the display device 4. Specifically, the microcontroller 6 outputs the safety display information to the information processing device 10. The information processing device 10 acquires the safety display information from the microcontroller 6 and generates the safety information image based on the safety display information. The information processing device 10 outputs the safety information image to the display device 4 to be displayed in the second display area. The positions of the first display area and the second display area on the display device 4 are preset design values.

[0065] As described above, in this embodiment, when the ECU 2 is started, the safety information image and the display image are displayed in each display area of ​​the display device 4, regardless of whether an abnormality is detected in the display image. If an abnormality is detected in the display image while the safety information image and the display image are displayed in each display area, the information processing device 10 stops displaying the display image on the display device 4 and continues displaying the safety information image. As a result, even if an abnormality occurs in the system-on-chip 5, the display device 4 continues to display the safety information image in the second display area. Therefore, if an abnormality occurs in the system-on-chip 5, a safety-related image can be displayed on the display device 4 while the system-on-chip 5 is being restored from the abnormal state. Furthermore, the microcontroller 6 may execute a watchdog clear process during the safety display information output process. As a result, even if an abnormality occurs in the microcontroller 6, the microcontroller 6 can be restored by restarting it using the watchdog.

[0066] A screen of a display device in the second embodiment will be described with reference to FIG. 9 . FIG. 9 is a diagram showing an example of a screen of a display device according to the second embodiment of the present invention. As shown in FIG. 9 , in the second embodiment, a first display area D21 of the display device 4 is an infotainment display area, in which a display image including information related to infotainment is displayed. A second display area D22 (area surrounded by a dotted line) of the display device 4 is an area including a speed display area D22a and a display area D22b for telltales, warning lights, etc., in which a safety information image is displayed. For example, a safety information image including speed information is displayed in the speed display area D22a.

[0067] In the first embodiment, the display device 4 switches between the display image and the safety information image, so there may be a discrepancy between the timing at which the image display is stopped and the timing at which the image is displayed. For example, if the timing at which the image display is stopped is too early and the timing at which the image is displayed is too late, a black screen will be displayed at the timing of switching. In contrast, in the second embodiment, the safety information image is independently displayed in a predetermined display area of ​​the display device 4 even before an abnormality is detected in the display image, so that the safety information image continues to be displayed in that display area even if an abnormality is detected in the display image.

[0068] Furthermore, for example, if the timing to stop displaying an image is delayed and the timing to display an image is accelerated, the display image and the safety information image may be superimposed at the timing of switching. In contrast to this, in the second embodiment, the safety information image is displayed in a predetermined display area of ​​the display device 4 before an abnormality is detected in the display image, and therefore even if an abnormality is detected in the display image, the safety information image continues to be displayed in that display area, thereby preventing overlapping of image displays at the timing of switching.

[0069] Next, the control process of the information processing method executed by the information processing device in the second embodiment will be described with reference to Fig. 10. Fig. 10 is a diagram showing an example of a flowchart of the control process of the information processing method executed by the information processing device in the second embodiment. In this embodiment, when the ECU 2 is started, the control flow starts from step S601.

[0070] In step S601, the information processing device 10 acquires a display image to be displayed on the display device 4 from the system-on-chip 5. The display image includes vehicle information and information other than the vehicle information to be provided to the user. In step S602, the processor 100 acquires safety display information for displaying safety information from the microcontroller 6. In step S603, the processor 100 generates a safety information image based on the safety display information. In step S604, the processor 100 outputs the display image and the safety information image to the display device 4. Specifically, the processor outputs the display image to be displayed in the first display area and outputs the safety information image to be displayed in the second display area. In step S605, the processor 100 detects whether or not there is an abnormality in the acquired display image. If there is an abnormality in the display image in step S605, the processor 100 proceeds to step S606. If there is no abnormality in the display image in step S605, the processor 100 returns to step S601 and repeats the subsequent steps. In step S606, the processor 100 outputs abnormality detection information indicating that there is an abnormality in the display image to the microcontroller 6. In step S607, the processor 100 stops displaying the display image and continues displaying the safety information image.

[0071] In the flowchart of FIG. 10, in steps S601 to S602, the display image is acquired and then the safety display information is acquired. However, this is not limited to this, and the display image and the safety display information may be acquired simultaneously, or the display image may be acquired after the safety display information is acquired.

[0072] As described above, in the information processing device, information processing method, and information processing system according to the present embodiment, the processor outputs a display image acquired from a first semiconductor device to the display device so that the display image is displayed in a first display area of ​​the display device, and outputs a safety information image to the display device based on safety display information acquired from a second semiconductor device so that the safety information image is displayed in a second display area different from the first display area of ​​the display device, and if an abnormality is detected in the display image, the display device stops displaying the display image and continues displaying the safety information image. Thus, if an abnormality occurs in the semiconductor device that executes the generation process and output process of the display image to be displayed on the display device, a safety-related image can be displayed on the display device while the semiconductor device is being restored from the abnormal state.

[0073] It should be noted that the above-described embodiments have been described to facilitate understanding of the present invention, and are not intended to limit the present invention. Therefore, each element disclosed in the above-described embodiments is intended to include all design modifications and equivalents that fall within the technical scope of the present invention.

[0074] REFERENCE SIGNS LIST 1... Information processing system 10... Information processing device 100... Processor 101... Abnormality detection unit 102... OSD image generation unit 103... Output unit 2... ECU 3... CAN bus 4... Display device 5... System on chip 6... Microcontroller

Claims

1. An information processing device equipped with a processor, The aforementioned processor, The display image is acquired from a first semiconductor device that performs image processing on the display image to be displayed on the display device. Safety display information for displaying safety information is obtained from a second semiconductor device, which is a different semiconductor device from the first semiconductor device. The display image acquired from the first semiconductor device is output to the display device. Based on the safety display information acquired from the second semiconductor device, a safety information image containing safety information is generated and output to the display device. Determine whether or not there is an abnormality in the display image acquired from the first semiconductor device. If it is determined that there is an abnormality in the displayed image, abnormality detection information is output to the second semiconductor device. When the recovery process of the first semiconductor device is executed based on the recovery instruction information output from the second semiconductor device, a switching control is executed to switch the image to be displayed on the display device from the safety information image to the display image. The switching control, after acquiring the display image output from the first semiconductor device, outputs safety display stop information to the display device to stop the display of the safety information image, and after outputting the safety display stop information, outputs the acquired display image to the display device. The safety display information is an information processing device which is output from the second semiconductor device to the processor when the second semiconductor device acquires the abnormality detection information.

2. An information processing apparatus according to claim 1, The aforementioned safety display information is an information processing device with a smaller data volume than the aforementioned display image.

3. An information processing apparatus according to claim 1 or 2, The aforementioned display image is an information processing device that is a moving image or a color image.

4. An information processing apparatus according to any one of claims 1 to 3, The safety information includes an information processing device that provides instruction information for instructing the display device to display the safety information image.

5. An information processing apparatus according to any one of claims 1 to 4, The aforementioned safety information image is an information processing device with a smaller data volume than the aforementioned display image.

6. An information processing apparatus according to claim 1, The aforementioned processor, When the recovery process of the first semiconductor device is executed based on the recovery instruction information output from the second semiconductor device, a switching control is executed to switch the image to be displayed on the display device from the safety information image to the display image. The switching control is an information processing device in which, after performing a fade-out process to gradually erase the safety information image in the display device, a fade-in process is performed to gradually display the display image.

7. An information processing device equipped with a processor, The aforementioned processor, The display image is acquired from a first semiconductor device that performs image processing on the display image to be displayed on the display device. Safety display information for displaying safety information is obtained from a second semiconductor device, which is a different semiconductor device from the first semiconductor device. The display image acquired from the first semiconductor device is output to the display device. Based on the safety display information acquired from the second semiconductor device, a safety information image containing safety information is generated and output to the display device. Determine whether or not there is an abnormality in the display image acquired from the first semiconductor device. If it is determined that there is an abnormality in the displayed image, abnormality detection information is output to the second semiconductor device. The safety display information is information output from the second semiconductor device to the processor when the second semiconductor device acquires the abnormality detection information, The second semiconductor device is an information processing device that, when it acquires the abnormality detection information, outputs recovery instruction information to the first semiconductor device for executing recovery processing.

8. An information processing method performed by a processor, The aforementioned processor, The display image is acquired from a first semiconductor device that performs image processing on the display image to be displayed on the display device. Safety display information for displaying safety information is obtained from a second semiconductor device, which is a different semiconductor device from the first semiconductor device. The display image acquired from the first semiconductor device is output to the display device. Based on the safety display information acquired from the second semiconductor device, a safety information image containing safety information is generated and output to the display device. Determine whether or not there is an abnormality in the display image acquired from the first semiconductor device. If it is determined that there is an abnormality in the displayed image, abnormality detection information is output to the second semiconductor device. When the recovery process of the first semiconductor device is executed based on the recovery instruction information output from the second semiconductor device, a switching control is executed to switch the image to be displayed on the display device from the safety information image to the display image. The switching control, after acquiring the display image output from the first semiconductor device, outputs safety display stop information to the display device to stop the display of the safety information image, and after outputting the safety display stop information, outputs the acquired display image to the display device. The safety display information is information output from the second semiconductor device to the processor when the second semiconductor device acquires the abnormality detection information.

9. An information processing device equipped with a processor, A first semiconductor device that performs image processing on a display image to be displayed on a display device, An information processing system comprising a second semiconductor device which is a different semiconductor device from the first semiconductor device, The first semiconductor device is The display image is output to the processor, The second semiconductor device is Safety display information for displaying safety information is output to the processor. The aforementioned processor, The display image is acquired from the first semiconductor device. The safety display information is obtained from the second semiconductor device. The acquired display image is output to the display device, Based on the acquired safety display information, a safety information image containing safety-related information is generated and output to the display device. Determine whether or not there is an abnormality in the display image acquired from the first semiconductor device. If it is determined that there is an abnormality in the displayed image, abnormality detection information is output to the second semiconductor device. When the recovery process of the first semiconductor device is executed based on the recovery instruction information output from the second semiconductor device, a switching control is executed to switch the image to be displayed on the display device from the safety information image to the display image. The switching control, after acquiring the display image output from the first semiconductor device, outputs safety display stop information to the display device to stop the display of the safety information image, and after outputting the safety display stop information, outputs the acquired display image to the display device. The safety display information is an information processing system in which the second semiconductor device outputs information to the processor when the second semiconductor device acquires the abnormality detection information.