Information display device, information display method, and program

Abstract

To facilitate identifying an occurrence location and cause of abnormal sound in a mobile body.SOLUTION: An information display device simultaneously displays mobile body data representing movement state of a mobile body and sound data generated from a sound produced during movement of the mobile body, aligned vertically and in synchronism on a vertically oriented display part,.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information display device, an information display method, and a program. [Background technology]

[0002] Patent Document 1 discloses a driving condition reproduction device that is useful for identifying the source and cause of abnormal vehicle noise. The driving condition reproduction device includes a vehicle data acquisition unit that acquires vehicle data that indicates the driving condition of the vehicle, a sound collection unit that uses a microphone to collect sounds generated while the vehicle is driving and generate sound data, a recording unit that records recorded data including the sound data generated by the sound collection unit and the vehicle data acquired by the vehicle data acquisition unit, and a playback unit that plays back the sound data included in the recorded data, and the playback unit is capable of playing back the sound data in synchronization with the vehicle data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-85059 Summary of the Invention [Problem to be solved by the invention]

[0004] The above techniques are useful for identifying the source and cause of abnormal noise. However, there is room for further improvement in terms of ease of identifying the source and cause of abnormal noise.

[0005] The present invention has been made in view of these technical problems, and aims to make it easier to identify the source and cause of abnormal noise in a moving body. [Means for solving the problem]

[0006] According to one aspect of the present invention, there is provided an information display device that synchronizes moving body data representing the moving state of a moving body with sound data generated from sounds made while the moving body is moving, and displays them simultaneously, one above the other, on a vertically long display unit. [Effects of the Invention]

[0007] According to the above aspect, not only are the sound and video synchronized, but the moving object data and sound data are simultaneously displayed one above the other on the vertically long display unit, making it easier to identify the source and cause of abnormal noise from the moving object. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a schematic diagram showing the configuration of an information display device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram for explaining how a moving image is captured. [Figure 3] FIG. 3 is a diagram for explaining a moving image. [Figure 4] FIG. 4 is a diagram for explaining how the information display process is executed, showing a start screen, a moving image selection screen, and an FFT analysis in progress screen. [Figure 5] FIG. 5 is a diagram for explaining how the information display process is executed, showing an analysis result display screen and a video playback screen. [Figure 6] FIG. 6 is a flowchart of the information display process. DETAILED DESCRIPTION OF THE INVENTION

[0009] An information display device 100 according to an embodiment of the present invention will now be described with reference to the accompanying drawings.

[0010] The information display device 100 is a device that synchronizes moving body data that indicates the moving state of a moving body with sound data generated from sounds made while the moving body is moving and displays them simultaneously, one above the other, on a vertically long display unit, and is useful for identifying the source and cause of abnormal sounds made by the moving body.

[0011] 1 is a schematic diagram of an information display device 100 according to an embodiment of the present invention. The information display device 100 according to this embodiment is realized using a smartphone 101.

[0012] As shown in FIG. 1, the smartphone 101 (information display device 100) has a CPU (Central Processing Unit) 10, a ROM (Read Only Memory) 11, a RAM (Random Access Memory) 12, a camera 13, a microphone 14, a touch panel display 15 as a display unit, a speaker 16, physical buttons 17, etc., which are connected via an internal bus 18.

[0013] The smartphone 101 performs various processes by causing the CPU 10 to read and execute various programs stored in the ROM 11 .

[0014] The moving object data is a video V (see FIG. 3) captured while the moving object is moving.

[0015] 2 is a diagram for explaining how to capture a moving image V. FIG. 3 is a diagram for explaining the moving image V.

[0016] A video V as moving body data is captured by mounting a smartphone 101 on a moving body. As shown in FIG. 2, in this embodiment, the smartphone 101 is mounted on a vehicle 200, which is an example of a moving body. The moving body may be something other than a vehicle. For example, a bicycle, a motorcycle, an airplane, a boat, etc. are also moving bodies.

[0017] 2, camera 13 of smartphone 101 captures an image of the area ahead (traveling direction) of vehicle 200. The area indicated by the dashed line is the imaging range. During video capture, microphone 14 records sounds generated while vehicle 200 is moving.

[0018] 3, the moving image V captured by the smartphone 101 records the moving state of the vehicle 200. From the moving image V, it is possible to visually grasp, for example, the vehicle speed, road surface conditions, steering operation status, and the like.

[0019] In this embodiment, an image of the area ahead of the vehicle 200 is described, but the direction in which the video V is captured is not limited to this. When identifying the location and cause of an abnormal sound, it may be necessary to capture an image in a direction other than the front. Furthermore, for example, if the camera 13 is an omnidirectional camera, it is also useful to capture images in all directions.

[0020] Next, the information display process performed by the information display application software will be described with reference to Figures 4 and 5. The information display application software is a program stored in ROM 11. The information display process is performed by the CPU 10 reading and executing the information display application software stored in ROM 11.

[0021] Fig. 4 is a diagram for explaining how the information display process is executed, showing a start screen, a video selection screen, and a screen showing FFT (Fast Fourier Transform) analysis. Fig. 5 is a diagram for explaining how the information display process is executed, showing an analysis result display screen and a video playback screen.

[0022] 4 and 5, the information display application software is used with the touch panel display 15 oriented vertically as seen by the user. In this embodiment, the speaker 16 side of the smartphone 101 is on the upper side, and the physical button 17 side is on the lower side.

[0023] When the user of the smartphone 101 starts the information display application software, a start screen is displayed on the touch panel display 15 (see FIG. 4). On the start screen, a start button 19 for starting the information display process is displayed as a virtual button.

[0024] When the user taps the start button 19, a video selection screen is displayed on the touch panel display 15.

[0025] On the video selection screen, thumbnails of various videos V stored in the RAM 12 of the smartphone 101 are displayed.

[0026] When the user selects (tap) the thumbnail of the video V that he or she wishes to play, an FFT analysis is performed on the sound of the video V (the sound recorded simultaneously with the capture of the video V). During the FFT analysis, an FFT analysis in progress screen is displayed on the touch panel display 15.

[0027] On the video selection screen, it is also possible to select a video V stored in a cloud environment or a video V stored in a storage medium such as a memory card attached to the smartphone 101. The video V may be captured and recorded using a device other than the smartphone 101.

[0028] If the file format of the sound of the moving image V selected by the user (the sound recorded simultaneously with the shooting of the moving image V) is not a predetermined file format, it is automatically converted to the predetermined file format. This reduces the user's effort and enables data to be quickly prepared in a file format suitable for FFT analysis. The predetermined file format is preferably, for example, WAV format (Waveform Audio File Format).

[0029] The FFT analysis screen displays an hourglass, indicating that FFT analysis is currently being performed on the sound recorded simultaneously with the capture of video V. When the FFT analysis is complete, an analysis result display screen is displayed on touch panel display 15 (see FIG. 5).

[0030] On the analysis result display screen, moving image V, which is moving body data showing the moving state of vehicle 200, and graphic data G, which is sound data generated from sounds made while vehicle 200 is moving, are simultaneously displayed vertically on the vertically long touch panel display 15. "Simultaneous display" means that the moving body data and sound data are displayed side by side vertically on a single screen without any switching or other operations.

[0031] The graphic data G is a color map in which the horizontal axis represents time [sec], the vertical axis represents frequency [Hz], and the color represents sound pressure [db].

[0032] In Figure 5, the left axis of the color map represents frequency (0 Hz to 5000 Hz), the right axis represents sound pressure (-10 dB to 50 dB), and the horizontal axis represents time (sec). The color distribution is shown diagrammatically.

[0033] The right axis of the color map is a color indicator. In this embodiment, examples of colors are -10[db] to 0[db] blue, 0[db] to 10[db] light blue, 10[db] to 20[db] green, 20[db] to 30[db] yellow, 30[db] to 40[db] orange, and 40[db] to 50[db] red. The color map corresponds to this color setting, and the colors of the multiple areas shown in the diagram are blue, light blue, green, yellow, orange, and red, starting from the top area.

[0034] A playback position indicator 30 for the video V is displayed as a virtual button below the video V. Below the playback position indicator 30, a play button 31, a stop button 32, a fast-forward button 33, a rewind button 34, and a volume control 35 are displayed as virtual buttons.

[0035] As shown on the moving image playback screen, the graphic data G has an annotation bar 36 that is displayed moving along the time axis in synchronization with the playback of the moving image V.

[0036] When the user taps the play button 31, the video V is played. Furthermore, while the video V is being played, the sound recorded simultaneously with the capture of the video V is also played. The annotation bar 36 moves horizontally along the time axis in accordance with the playback position of the video V. In other words, the position of the annotation bar 36 corresponds to the playback position of the video V indicated by the playback position indicator 30.

[0037] By using the information display device 100, the user can simultaneously check the moving image V, the sound, and the color map, making it easier for the user to visually grasp the movement of the moving image V and the changes in the sound shown in the color map. In other words, by visually interpreting the frequency of the sound, the timing at which the sound occurs, the duration of the sound occurrence, changes in sound pressure, etc., it becomes easier to identify the source and cause of an abnormal sound from the vehicle 200.

[0038] Furthermore, in the information display device 100, the annotation bar 36 moves along the horizontal axis. In this regard, if the moving image V and the graphic data G were simultaneously displayed side by side on the horizontally long touch panel display 15, the user would need to follow the annotation bar 36 horizontally with their eyes while moving their eyes left and right to compare the moving image V and the graphic data G side by side. This can be said to be a significant burden on the user.

[0039] In contrast, in this embodiment, the moving image V and the graphic data G are simultaneously displayed side by side on the vertically long touch panel display 15, so that it is easy to move the eyes to the moving image V while following the annotation bar 36 horizontally. This reduces the burden on the user.

[0040] In this way, according to the information display device 100 (smartphone 101) of this embodiment, not only are the sound and video V synchronized, but the video V and graphic data G are simultaneously displayed vertically on the vertically long touch panel display 15, making it easier to identify the location and cause of abnormal noise in the vehicle 200.

[0041] Next, the procedure of the information display process will be described with reference to FIG.

[0042] FIG. 6 is a flowchart of the information display process.

[0043] When the user of the smartphone 101 starts the information display application software and taps the start button 19, the information display process starts.

[0044] When a moving image V is selected by the user (step S11), it is determined whether the file of the sound of the selected moving image V (sound recorded simultaneously with the shooting of the moving image V) is in a predetermined file format (step S12).

[0045] If it is determined that the file is formed in the predetermined file format, the process proceeds to step S13. If it is determined that the file is not formed in the predetermined file format, the process proceeds to step S14.

[0046] In step S14, the sound of the moving image V is converted into a predetermined file format.

[0047] In step S13, the sound of the moving image V is subjected to FFT analysis.

[0048] When the FFT analysis is completed, graphic data G, which is sound data, is generated from the sound of the moving image V (step S15). In this embodiment, the graphic data G is a color map.

[0049] In step S16, the moving image V and the graphic data G are displayed one above the other on the touch panel display 15.

[0050] When the user taps the play button 31 (step S17), the video V is played back, and the annotation bar 36 moves horizontally along the time axis in synchronization with the video V (step S18).

[0051] The configurations and effects of the information display device 100, the information display method, and the program according to the embodiment of the present invention will be described below.

[0052] (1) The information display device 100 (smartphone 101) synchronizes moving object data (video V) representing the moving state of the vehicle 200 with sound data (graphic data G) generated from sounds made while the vehicle 200 is moving, and displays them simultaneously, one above the other, on the vertically long touch panel display 15.

[0053] This not only synchronizes the sound and video V, but also displays the moving object data and sound data vertically at the same time on the vertically long touch panel display 15, making it easier to identify the source and cause of abnormal noises in the vehicle 200.

[0054] (2) The moving body data is a video V captured while the vehicle 200 is moving, and the sound data is graphic data G generated from sound recorded simultaneously with the capture of the video V and arranged with the time axis as the horizontal axis, and the graphic data G has an annotation bar 36 that is displayed moving along the time axis in synchronization with the playback of the moving body data.

[0055] This allows the graphic data G, which is an image of the sound data converted into a graphic, and the video V to be displayed side by side, and the video V and the annotation bar 36 to be displayed in synchronization, making it easier to identify the location and cause of the abnormal noise in the vehicle 200.

[0056] (3) The sound data is graphic data G in which the horizontal axis represents time, the vertical axis represents frequency, and the sound pressure is displayed in color.

[0057] This allows the image that graphically represents the frequency and sound pressure and the video V to be displayed side by side, and the video V and the annotation bar 36 to be displayed in sync, making it easier to identify the location and cause of abnormal noise in the vehicle 200.

[0058] (4) The moving body data is video V captured while the vehicle 200 is moving, and the sound data is graphic data G generated from sound recorded simultaneously with the capture of the video V, with the time axis arranged on the horizontal axis. If the file format of the recorded sound is not a specified file format, the sound data is generated after being converted into the specified file format.

[0059] According to this, if the file format of the sound of video V (sound recorded simultaneously with the shooting of video V) is not a predetermined file format, it is automatically converted into the predetermined file format. This reduces the user's effort and enables data to be quickly prepared in a file format suitable for FFT analysis. Therefore, sound data can be easily converted into graphics.

[0060] (5) In the information display method executed by the computer (CPU 10) of the information display device 100 (smartphone 101), moving object data (video V) representing the moving state of the vehicle 200 and sound data (graphic data G) generated from sounds made while the vehicle 200 is moving are synchronized and simultaneously displayed vertically on the vertically long touch panel display 15.

[0061] This not only synchronizes the sound and video V, but also displays the moving object data and sound data vertically at the same time on the vertically long touch panel display 15, making it easier to identify the source and cause of abnormal noises in the vehicle 200.

[0062] (6) A program (information processing application software) executable by the computer (CPU 10) of the information display device 100 (smartphone 101) causes the CPU 10 to execute a procedure for synchronizing moving object data (video V) representing the moving state of the vehicle 200 with sound data generated from sounds made while the vehicle 200 is moving and simultaneously displaying them vertically on the vertically long touch panel display 15.

[0063] This not only synchronizes the sound and video V, but also displays the moving object data and sound data vertically at the same time on the vertically long touch panel display 15, making it easier to identify the source and cause of abnormal noises in the vehicle 200.

[0064] Although the embodiments of the present invention have been described above, the above embodiments merely illustrate some of the application examples of the present invention, and it is not intended that the technical scope of the present invention be limited to the specific configurations of the above embodiments. [Explanation of symbols]

[0065] 100 Information display device 101 Smartphone (information display device) 200 Vehicles (moving objects) 10 CPU (computer) 15 Touch panel display (display unit) 36 Annotation Bar G Graphic data (sound data) V Video (moving data)

Claims

1. moving object data representing the moving state of the moving object and sound data generated from sounds generated while the moving object is moving are displayed simultaneously in upper and lower positions on a vertically long display unit in synchronization with each other; Information display device.

2. The information display device according to claim 1, the moving object data is a video captured while the moving object is moving, the sound data is graphic data generated from sounds recorded simultaneously with the video and arranged with the time axis as the horizontal axis, the graphic data has an annotation bar that is displayed moving along the time axis in synchronization with the playback of the moving object data; Information display device.

3. 3. The information display device according to claim 2, The sound data is graphic data in which the time axis is arranged as the horizontal axis, frequency is arranged as the vertical axis, and sound pressure is displayed as color. Information display device.

4. The information display device according to claim 1, the moving object data is a video captured while the moving object is moving, the sound data is graphic data generated from sounds recorded simultaneously with the video and arranged with the time axis as the horizontal axis, If the file format of the recorded sound is not a predetermined file format, the sound data is generated after being converted into the predetermined file format. Information display device.

5. An information display method executed by a computer of an information display device, comprising: moving object data representing the moving state of the moving object and sound data generated from sounds generated while the moving object is moving are displayed simultaneously in upper and lower positions on a vertically long display unit in synchronization with each other; Information display method.

6. A program executable by a computer of an information display device, causing the computer to execute a procedure of simultaneously displaying moving object data representing a moving state of the moving object and sound data generated from sounds generated while the moving object is moving, one above the other, on a vertically long display unit in synchronization with each other; program.

Images