Information processor
The device addresses the challenge of monitoring multi-threaded video data generation by generating a specific image in a non-visible area and restarting software if the image deviates, ensuring correct operation and display integrity.
Patent Information
- Application Number
- JP2024065776
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-04-15
- Publication Date
- 2025-10-27
AI Technical Summary
Conventional systems face difficulties in monitoring whether software generating video data for display is functioning correctly, especially in multi-threaded environments where watchdog signals fail to accurately detect normal operation.
The information processing device generates a specific image in a non-visible area of the video data using predetermined rules, and monitors this image to ensure the software is operating correctly, restarting it if the image deviates from the expected pattern.
Enables accurate detection of video data generation issues by ensuring the software continues to function correctly, thereby maintaining display integrity.
Smart Images

Figure 2025162459000001_ABST
Abstract
Description
[Technical Field]
[0001] The present disclosure relates to an information processing device. [Background technology]
[0002] Patent Document 1 discloses a system that allows multiple instances of the same application program to listen simultaneously on the same IP port. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-154518 Summary of the Invention [Problem to be solved by the invention]
[0004] In conventional systems such as that described in Patent Document 1, each piece of software uses a watchdog function to send a watchdog signal at regular intervals to notify that it is operating normally. However, it is generally known that it is difficult to use a watchdog signal to monitor whether specific software that generates video data showing images to be displayed on a display is generating video data.
[0005] The present disclosure aims to detect whether specific software that generates video data showing an image to be displayed on a display is correctly generating the video data. [Means for solving the problem]
[0006] The information processing device of claim 1 comprises a generation unit that uses specific software to generate video data showing an image to be displayed on a display, and generates a specific image in accordance with predetermined rules in a specific area of the video data that is not visible to the user when the image is displayed on the display, and a control unit that continues to run the specific software if the image shown in the specific area is the specific image, and restarts the specific software if the image shown in the specific area is not the specific image.
[0007] In the information processing device according to claim 1, when an image is displayed on the display, the generation unit uses specific software that generates video data to generate a specific image in accordance with a predetermined rule in a specific area of the video data that is not visible to the user. The control unit then continues to run the specific software if the image displayed in the specific area is the specific image. On the other hand, the control unit restarts the specific software if the image is not the specific image. This allows the information processing device to detect that the specific software is properly generating video data if the image displayed in the specific area is properly in accordance with the predetermined rule. Furthermore, the information processing device can detect that the specific software is not properly generating video data if the image displayed in the specific area is not properly in accordance with the predetermined rule. In this way, the information processing device can detect whether the specific software is properly generating video data. [Effects of the Invention]
[0008] The information processing device according to the present disclosure can detect whether specific software that generates video data showing an image to be displayed on a display is correctly generating the video data. [Brief explanation of the drawings]
[0009] [Figure 1] FIG. 2 is a block diagram showing the hardware configuration of a vehicle. [Figure 2] 10 is a flowchart showing the flow of a specification process. [Figure 3]FIG. 10 is a schematic diagram showing the flow of a specification process. DETAILED DESCRIPTION OF THE INVENTION
[0010] A vehicle 10 according to this embodiment will now be described. As shown in Fig. 1, the vehicle 10 includes a meter ECU (Electronic Control Unit) 15 and a display 50. The meter ECU 15 is an example of an "information processing device."
[0011] The meter ECU 15 includes a microcomputer 20, a software storage unit 30, and a GPU (Graphics Processing Unit) 40.
[0012] The microcomputer 20 includes a CPU (Central Processing Unit) 21, a ROM (Read Only Memory) 22, a RAM (Random Access Memory) 23, and a storage 24. Each component is connected to each other via an internal bus 25 so that they can communicate with each other.
[0013] The CPU 21 is a central processing unit that executes various programs and controls each part. That is, the CPU 21 reads programs from the ROM 22 or the storage 24 and executes the programs using the RAM 23 as a work area. The CPU 21 controls each of the above components and performs various arithmetic processing in accordance with the programs recorded in the ROM 22 or the storage 24.
[0014] The ROM 22 stores various programs and various data. The RAM 23 serves as a working area for temporarily storing programs or data.
[0015] The storage 24 is a storage device that stores various programs and various data, and stores an information processing program 24A that causes the CPU 21 to execute specific processing, which will be described later.
[0016] The software storage unit 30 stores control software 32, drawing software 34, and survival management software 36 (see FIG. 3) as software used in specific processes.
[0017] The control software 32 interprets various information input from other ECUs of the vehicle 10, such as CAN signals, determines the display content of the display 50 based on the various information, and sends display instructions (input signals) to the drawing software 34.
[0018] The drawing software 34 manages the display state of the display 50 based on display instructions received from the control software 32, generates video data in accordance with the display state, and displays the video on the display 50 using the GPU 40. The drawing software 34 is an example of "specific software."
[0019] The survival management software 36 monitors whether the drawing software 34 is operating normally, and restarts the drawing software 34 if it is determined that the drawing software 34 is not operating normally.
[0020] The GPU 40 includes a frame buffer 42 into which video data generated by the drawing software 34 is written. The video data is data representing an image to be displayed on the display 50, and the GPU 40 transmits the video data written in the frame buffer 42 to the display 50.
[0021] The display 50 is provided on the meter panel and is a meter display that displays operation suggestions related to functions of the vehicle 10, images related to explanations of the functions, and the like.
[0022] The CPU 21 of the meter ECU 15 has, as functional components, an acquisition unit 21A, a generation unit 21B, and a control unit 21C. Each functional component is realized by the CPU 21 reading and executing an information processing program 24A stored in the storage 24.
[0023] The acquisition unit 21A acquires various information input from other ECUs of the vehicle 10, such as CAN signals. The generating unit 21B uses drawing software 34 to generate video data.
[0024] The control unit 21C uses the survival management software 36 to monitor whether the drawing software 34 is operating normally.
[0025] Next, the flow of the identification process executed by the meter ECU 15 will be described with reference to the flowchart shown in Fig. 2 and the schematic diagram shown in Fig. 3. The CPU 21 reads the information processing program 24A from the storage 24, deploys it in the RAM 23, and executes it, thereby performing the identification process. As an example, the identification process is performed when the meter ECU 15 acquires a predetermined CAN signal input from another ECU.
[0026] 2, the CPU 21 transmits to the drawing software 34 an instruction to display an image on the display 50. In this case, as indicated by the arrow (a) in FIG. 3, the control software 32 interprets the acquired CAN signal and transmits a display instruction to the drawing software 34 based on the interpreted content. Then, the CPU 21 proceeds to step S11.
[0027] 2, in step S11, the CPU 21 generates video data, and then the CPU 21 proceeds to step S12.
[0028] 3, the drawing software 34 is implemented in a multi-threaded manner, having three threads: a receiving thread 34A, a state management thread 34B, and a drawing thread 34C. Because the drawing software 34 is implemented in a multi-threaded manner, simply sending a watchdog signal merely notifies the software that sent the watchdog signal that the thread is operating normally, and does not eliminate the possibility that other threads are not operating normally. Therefore, for software implemented in a multi-threaded manner, such as the drawing software 34, it is difficult to use a watchdog function to monitor whether the software is operating normally.
[0029] In step S11 above, within the drawing software 34, as indicated by arrow (b), the reception thread 34A receives the display instruction sent from the control software 32 and transmits the received display instruction to the state management thread 34B. The state management thread 34B updates the display state of the display 50 in accordance with the acquired display instruction and determines the drawing content. The state management thread 34B then transmits the determined drawing content to the drawing thread 34C as indicated by arrow (c). The drawing thread 34C then interprets the acquired drawing content, generates (rasterizes) one frame of video data using the GPU 40, and writes the generated video data to the frame buffer 42 as indicated by arrow (d).
[0030] 3, the video data written to the frame buffer 42 in step S11 above is represented by rectangular video data 60. The video data 60 has a visible area 62 that is visible to the user when the video represented by the video data 60 is displayed on the display 50, and an invisible area 64 that is not visible to the user. The visible area 62 is an area that is located within the display area of the display 50 when the video is displayed on the display 50, and an image is generated (drawn) in accordance with the contents of the display instruction transmitted from the control software 32. The invisible area 64 is an area that is located outside the display area of the display 50 when the video is displayed on the display 50, and a specific video is generated (drawn) in accordance with a predetermined rule in a specific area 64A that is a part of the invisible area 64. For example, if the predetermined rule is to generate a predetermined shape (e.g., a star), the specific video will be an image of the predetermined shape (e.g., a star).
[0031] 2, in step S12, CPU 21 analyzes specific area 64A. In this case, as indicated by arrow (e) in Fig. 3, survival management software 36 performs known image processing on specific area 64A of video data 60 written to frame buffer 42, and analyzes the video shown in specific area 64A. CPU 21 then proceeds to step S13.
[0032] 2, in step S13, if the CPU 21 determines that the image displayed in the specific area 64A is a specific image (step S13: YES), the process proceeds to step S14. On the other hand, if the CPU 21 determines that the image displayed in the specific area 64A is not a specific image (step S13: NO), the process proceeds to step S15. For example, if the specific image is an image of stars, and the image of the stars is displayed in the specific area 64A, the CPU 21 proceeds to step S14. If the image displayed in the specific area 64A is an image other than the image of the stars, the CPU 21 proceeds to step S15.
[0033] In step S14, the CPU 21 continues to execute the drawing software 34. Then, the CPU 21 ends the specific processing.
[0034] In step S15, the CPU 21 determines that the drawing software 34 is not operating normally, and then the CPU 21 proceeds to step S16.
[0035] In step S16, the CPU 21 restarts the drawing software 34 using the survival management software 36. Then, the CPU 21 ends the identification process.
[0036] As described above, in the meter ECU 15, the CPU 21 uses the drawing software 34 that generates the video data 60 to generate a specific video in accordance with a predetermined rule in the specific area 64A of the video data 60 that is not visible to the user when the video is displayed on the display 50. The CPU 21 then continues to execute the drawing software 34 if the video displayed in the specific area 64A is the specific video. On the other hand, the CPU 21 restarts the drawing software 34 if the video is not the specific video. As a result, the meter ECU 15 can detect that the drawing software 34 is correctly generating the video data 60 if the specific video displayed in the specific area 64A is correctly generated according to the predetermined rule. Furthermore, the meter ECU 15 can detect that the drawing software 34 is not correctly generating the video data 60 if the specific video displayed in the specific area 64A is not correctly generated according to the predetermined rule. In this way, the meter ECU 15 can detect whether the drawing software 34 is correctly generating the video data 60.
[0037] (others) In the above embodiment, specific area 64A that is not visible to the user is an area that is located outside the display area of display 50 when an image is displayed on display 50. However, specific area 64A is not limited to this, and may be an area that is located within the display area of display 50 when an image is displayed on display 50 and that is hidden by physical decoration.
[0038] 2, if the image displayed in the specific area 64A is an image that does not conform to a predetermined rule, the CPU 21 determines that the image displayed in the specific area 64A is not a specific image. However, without being limited to this, the CPU 21 may determine that the image displayed in the specific area 64A is not a specific image if no image is present in the specific area 64A.
[0039] In the above embodiment, the display 50 is a meter display, but is not limited to this, and the display 50 may be a multimedia display, a head-up display, etc. When the display 50 is a display other than the meter display, an example of the information processing device is an ECU that controls the other display.
[0040] In the above embodiment, the specific process executed by the CPU 21 after reading the software (program) may be executed by various processors other than a CPU. Examples of such processors include programmable logic devices (PLDs) (such as field-programmable gate arrays (FPGAs)) whose circuit configuration can be changed after fabrication, and dedicated electrical circuits such as application-specific integrated circuits (ASICs) that are processors with circuit configurations specifically designed to execute specific processes. The specific process may be executed by one of these processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). The hardware structure of these processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor devices.
[0041] In the above embodiment, the information processing program 24A is pre-stored (installed) in the storage 24, but the present invention is not limited to this. The information processing program 24A may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The information processing program 24A may also be downloaded from an external device via a network. [Explanation of symbols]
[0042] 15 Meter ECU (information processing unit) 21B Generation part 21C Control Unit 34 Drawing software (specific software) 50 displays 60 Video Data 64A Specific area
Claims
[Claim 1] a generating unit that uses specific software to generate video data showing a video to be displayed on a display, and generates a specific video in accordance with a predetermined rule in a specific area of the video data that is invisible to a user when the video is displayed on the display; a control unit that continues to execute the specific software when the image displayed in the specific area is the specific image, and restarts the specific software when the image displayed in the specific area is not the specific image; An information processing device comprising:
Citation Information
Patent Citations
Method and system for dispatching received sessions between multiple instances of application using the same IP port
JP2010154518A