Display control device

Abstract

To provide a display control device capable of enhancing entertainability using point cloud information.SOLUTION: A display control device 1 has a LiDAR 2 configured to acquire point cloud information comprising a plurality of points representing objects around a vehicle, and a display control unit 3 configured to display a point cloud information-based image in one of a plurality of display modes with different display styles on a display unit 4.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a display control device that displays an image on a display unit. [Background technology]

[0002] Conventionally, a technology has been proposed in which a laser radar is used to acquire the positions in three-dimensional space of a plurality of points on an object present in a detection area (i.e., an area targeted for object detection) such as in front of a vehicle, and the object is recognized. For example, Patent Document 1 discloses a technology in which the points whose positions have been acquired are clustered to acquire a cluster point cloud, and an object corresponding to the cluster point cloud is identified based on the temporal fluctuation of measurement quantities related to the cluster point cloud. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-151043 Summary of the Invention [Problem to be solved by the invention]

[0004] As described above, laser radar (hereinafter also referred to as LiDAR (Light Detection And Ranging)) is used to recognize objects present around a moving vehicle, etc. However, the information acquired by the laser radar (so-called point cloud information) is not used for purposes other than recognizing such objects.

[0005] On the other hand, in future vehicles equipped with autonomous driving functions, it is expected that the entertainment value of the travel time during autonomous driving will be improved. However, the idea of ​​using information obtained by laser radar to improve the entertainment value has not existed until now.

[0006] One example of a problem that the present invention aims to solve is to improve entertainment value by utilizing so-called point cloud information. [Means for solving the problem]

[0007] In order to solve the above problem, the invention described in claim 1 is a display control device that displays an image on a display unit arranged on a moving body, characterized by comprising: a point cloud information acquisition unit that acquires point cloud information that represents features around the moving body using a plurality of points; and a display control unit that displays an image based on the point cloud information on the display unit in one of a plurality of display modes with different display modes.

[0008] The invention described in claim 6 is a display control method executed by a display control device that controls the display of a display unit arranged on a moving body, characterized in that it includes a point cloud information acquisition process that acquires point cloud information that represents features around the moving body using a plurality of points, and a display control process that displays an image based on the point cloud information on the display unit in one of a plurality of display modes with different display modes.

[0009] The invention as set forth in claim 7 is characterized in that the display control method as set forth in claim 6 is executed by a computer. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram of a display control device according to an embodiment of the present invention; [Figure 2] 2 is a flowchart of a display operation of the display control device shown in FIG. [Figure 3] 2 is an example of an operation image in the display control device shown in FIG. 1. [Figure 4] 10 shows examples of a first mode display mode and a second mode display mode. [Figure 5] 10 is an example of another display mode in the second mode. [Figure 6] This is an example of a display mode that mixes the display modes of the first mode and the second mode. [Figure 7] 2 is a modified example of the display control device shown in FIG. DETAILED DESCRIPTION OF THE INVENTION

[0011] A display control device according to one embodiment of the present invention will be described below. In the display control device according to one embodiment of the present invention, a point cloud information acquisition unit acquires point cloud information that represents features around a moving object using a plurality of points, and a display control unit displays an image based on the point cloud information on a display unit in one of a plurality of display modes with different display modes. In this way, it is possible to display in a plurality of display modes based on the point cloud information. Therefore, it is possible to improve entertainment value by utilizing the point cloud information.

[0012] The display control unit may also switch between a plurality of display modes to display on the display unit, thereby allowing images with different display modes to be displayed by appropriately switching between the plurality of display modes.

[0013] The display control unit may also display multiple display modes on the display unit in an overlapping (or combined) manner. This allows for a display in which the display modes of multiple display modes are overlapped, thereby increasing the variety of displays. Such displays may include, for example, a gradual transition from one display mode to another.

[0014] The display control unit may also be configured to change the display mode of an image displayed in one display mode depending on the music being played, thereby enabling the image to be displayed in a display mode that matches the image of the music.

[0015] Furthermore, one of the plurality of display modes may be a display mode that is a deformed version of the display mode in the other display modes, thereby enabling display in a deformed display mode based on the display mode in the other display mode.

[0016] Furthermore, a display control method according to one embodiment of the present invention includes a point cloud information acquisition step for acquiring point cloud information representing features around a moving object using a plurality of points, and a display control step for displaying an image based on the point cloud information, the display having a plurality of display modes each having a different display manner. In this way, it is possible to display an image in a plurality of display manners based on the point cloud information. Therefore, it is possible to improve entertainment value by utilizing the point cloud information.

[0017] The above-described display control method may also be implemented as a display control program that causes a computer to execute the method. In this way, a computer can be used to display a plurality of display modes based on point cloud information. Therefore, the point cloud information can be used to improve entertainment value. [Example]

[0018] A display control device according to a first embodiment of the present invention will be described with reference to Fig. 1 to Fig. 6. As shown in Fig. 1, the display control device 1 includes a rider 2, a display control unit 3, a display unit 4, a music acquisition unit 5, a music analysis unit 6, and an operation unit 7. The display control device 1 shown in Fig. 1 is assumed to be mounted on a vehicle such as an automobile, as an example of a moving object.

[0019] The LIDAR 2, which serves as a point cloud information acquisition unit, is a sensor for recognizing objects present around the vehicle, and is also referred to as LiDAR (Light Detection and Ranging). The LIDAR 2 is a known sensor that emits light such as laser light, measures the distance to an object in the external world discretely based on the reflected light, and recognizes the position, shape, and the like of the object as a three-dimensional point cloud. Therefore, the point cloud acquired by the LIDAR 2 is information (point cloud information) that represents features, including structures such as buildings and roads present around the vehicle, in three dimensions using a plurality of points. Note that the LIDAR 2 may recognize people, other vehicles, and the like present around the vehicle in addition to features such as signs on the road on which the vehicle is traveling and buildings and roadside trees present along the road.

[0020] The display control unit 3 converts the point cloud information acquired by the LIDAR 2 from a coordinate system (vehicle coordinate system) in which the LIDAR 2 acquired the point cloud information to a coordinate system (display coordinate system) for displaying the point cloud information on the display unit 4 that displays the point cloud information. The vehicle coordinate system is a coordinate system based on the vehicle (LIDAR 2). The display coordinate system is a two-dimensional coordinate system defined on the display unit 4. The display control unit 3 converts the three-dimensional vehicle coordinate system in this point cloud information into a two-dimensional display coordinate system. This coordinate conversion can use a well-known coordinate conversion method, such as conversion from a so-called view coordinate system to a screen coordinate system.

[0021] The display control unit 3 has two display modes, a first mode and a second mode, for displaying an image based on the point cloud information on the display unit 4. The display control unit 3 also applies processing such as effects to the point cloud information displayed on the display unit 4 based on the analysis results of the music analysis unit 6, and outputs the processed image to the display unit 4 as an image based on the point cloud information. That is, the display control unit 3 has a plurality of display modes for displaying an image based on the point cloud information on the display unit 4.

[0022] The display unit 4 displays an image based on the point cloud information processed by the display control unit 3. The display unit 4 can be configured, for example, as a monitor screen or a HUD (Head-Up Display) installed in the center console of the vehicle. The display unit 4 is not limited to a device installed in the vehicle, and may display images on, for example, a mobile terminal owned by a passenger. The display unit 4 may also be a glass part of the vehicle, such as the windshield, door glass, or sunroof glass of the vehicle in which the display control device 1 is installed. In this way, the display unit 4 is not limited to a display device fixed in the vehicle or a part capable of displaying by projection, etc., but may also be a terminal brought into the vehicle, in other words, it may be located in the vehicle.

[0023] The music acquisition unit 5 acquires music data that is being played in a car by a car audio system or that is recorded on a car audio system, for example. Note that the music acquisition unit 5 may acquire music data not only from a car audio system but also from a portable music player connected via short-range wireless communication or a smartphone with a music playback function. Alternatively, the music data may be music data distributed from an external network.

[0024] The music analysis unit 6, which serves as a music information acquisition unit, analyzes the features of the music data acquired by the music acquisition unit 5. Examples of the features include tempo (BPM: Beats Per Minute), rhythm (meter, beat), etc. Analysis of these features can be performed using well-known methods. Genre information contained in the music data itself may also be acquired. Therefore, these feature values ​​and genre information become music information related to the music.

[0025] The operation unit 7 is configured with, for example, a touch panel, push buttons, etc. The operation unit 7 can switch between the first mode and the second mode described above.

[0026] Next, the operation (display control method) of the display control device 1 configured as described above will be described with reference to the flowchart in Fig. 2. The flowchart shown in Fig. 2 is executed by the display control unit 3. Therefore, by configuring the display control unit 3 with a microcomputer or the like having a CPU or the like, it can be made into a display control program that executes the display control method.

[0027] First, the display control unit 3 acquires the point cloud information detected by the rider 2 (step S11), and acquires the analysis results (tempo, rhythm, genre, etc.) analyzed by the music analysis unit 6 (step S12).

[0028] Next, the display mode selected by the operation unit 7 or the like is determined (step S13). An example of the operation unit 7 is shown in FIG. 3. FIG. 3 shows an operation image 40 displayed on, for example, the display unit of the center console. This display unit is assumed to have a touch panel. Alternatively, if the display unit 4 is configured as the display unit of the center console, the display area of ​​the display unit 4 may be divided into a display area for an image for operation and a display area for an image based on point cloud information.

[0029] In Fig. 3, an image 40 includes a song display section 41, a slide section 42, and a button 44. In the song display section 41, the titles and jacket images of two songs (or a group of songs such as an album or playlist) are arranged side by side. In the example of Fig. 3, the title "AAAAA" and its jacket image J1 are displayed on the left side, and the title "BBBBB" and its jacket image J2 are displayed on the right side.

[0030] The slide section 42 has a slider 43, which can be slid left and right on the slide section 42 by dragging it with a finger or the like. The button 44 is made up of a button 44L and a button 44R, and the button 44L slides the slider 43 leftward. The button 44R slides the slider 43 rightward. In other words, the slider 43 can be slid directly by a drag operation, and can also be slid by operating the button 44.

[0031] 3, when the slider 43 is slid to the left end, it is determined to be the first mode, when it is slid to the right end, it is determined to be the second mode, and when the slider 43 is stopped between the left end and the right end, it is determined to be "MIX." Therefore, in step S13, when the slider 43 is slid to the left end, it is determined to be the first mode, and the point cloud information is processed to have a display form in the first mode (step S14). Also, when the slider 43 is slid to the right end in step S13, it is determined to be the second mode, and the point cloud information is processed to have a display form in the second mode (step S15).

[0032] 3, the music played at the slider position corresponds to the display mode. That is, music is set for each of the first and second modes. For example, when the slider 43 is slid to the left end, the title "AAAAA" is played in the display mode of the first mode, and when the slider 43 is slid to the right end, the title "BBBBB" is played in the display mode of the second mode. When the slider 43 is stopped between the left end and the right end, the display mode becomes "MIX," and both music pieces are played mixed together, and the display mode is also superimposed (combined) with the two display modes.

[0033] Therefore, if the slider 43 is slid to the left end in step S13, it is determined that the first mode is selected, and the point cloud information is processed to display in the first mode (step S14). Also, if the slider 43 is slid to the right end in step S13, it is determined that the second mode is selected, and the point cloud information is processed to display in the second mode (step S15).

[0034] Here, examples of the display mode in the first mode and the display mode in the second mode are shown in FIG. 4. The upper part of FIG. 4 shows an example of the display mode in the first mode. The first mode in this embodiment is a display mode in which the point cloud is displayed as is based on the point cloud information acquired by the LIDAR 2. In other words, the image is one in which each feature is reproduced as a point cloud. However, the points may be colored for each feature or according to distance.

[0035] 4 shows an example of the display mode in the second mode. The second mode in this embodiment is an image that is displayed in a deformed manner, for example, by emphasizing the contours of features, based on the point cloud information acquired by the LIDAR 2.

[0036] In this embodiment, there are two display modes, but there may be three or more. In this embodiment, music is set for each of the first and second modes in the display control unit 3, and the display mode of the image displayed on the display unit 4 in one display mode is changed in accordance with the music being played as described below, but the display mode for each mode may be selected from multiple display modes depending on the music, or may be selected depending on the genre of the music.

[0037] FIG. 5 shows another example of the display mode in the second mode. Specifically, it shows an example of changes in the display mode of an image displayed in the second mode using music information for music set for the first mode and the second mode, analyzed by the music analysis unit 6. In FIG. 5, for example, a circular effect E1 is displayed on the wall of a building, for example, a building; an effect E2 in which a figure representing color, light, etc. moves in the direction of a moving object along a road curb or white line; and an effect E3 in which the color, light, etc. changes along the height of the building, creating a flowing effect. These effects E1 to E3 vary in the blinking speed of the circle, color, light, etc., the movement speed of the figure representing color, light, etc., and the flow speed of the color, light, etc., depending on the characteristics of the music (tempo (BPM), rhythm (time signature, beat), genre, etc.). Furthermore, like effect E1, a circular effect E4 displayed on the wall of a building, for example, varies in size depending on the rhythm of the music. In other words, the timing of the effect applied to the image changes based on the characteristics of the music. The bottom part of Figure 4 and Figure 5 show an image of projection mapping being applied to the entire city as the vehicle moves along.

[0038] Furthermore, if the slider 43 is stopped between the left and right ends in step S13, it is determined to be "MIX," and an image in the first mode display mode and an image in the second mode display mode are superimposed (combined) (step S16). An example of the display mode in the case of MIX is shown in FIG. 6. The ratio at which the two display modes are superimposed (combined) can be determined depending on the position of the slider 43. For example, if the position of the slider 43 is close to the left end, the ratio of the first mode display mode is increased. Note that this "MIX" display also includes cases where the display changes when, for example, the slider 43 is moved from the first mode display mode to gradually transition to the second mode display mode.

[0039] That is, when the slider 43 is positioned from the left end to the right end or from the right end to the left end, the display mode of one of the plurality of display modes is selected and an image is displayed on the display unit 4, and when the slider 43 is positioned between the left end and the right end, the superimposition (combination) ratio of the display modes of the plurality of display modes is changed and displayed on the display unit 4. That is, when the slider 43 is positioned from the center of the slide section 42 to the left end, an image in which an image in the display mode of the first mode and an image in the display mode of the second mode are superimposed is displayed on the display unit 4 in a state in which the proportion of the image in the display mode of the first mode is higher than the proportion of the image in the display mode of the second mode.

[0040] Then, in step S17, the image generated in one of the display modes in steps S14 to S16 is displayed on the display unit 4.

[0041] According to this embodiment, the display control device 1 acquires point cloud information in which the rider 2 represents features around the vehicle using a plurality of points, and the display control unit 3 causes the display unit 4 to display an image based on the point cloud information in two display modes each having a different display manner. In this way, an image based on the point cloud information can be displayed in two display manners. Therefore, the entertainment value can be improved by utilizing the point cloud information.

[0042] For example, images such as those shown in Figures 4 to 6 can be generated by processing images captured by a camera. However, since images captured by a camera do not contain distance information, it is necessary to estimate the distance using image processing or the like, which increases the processing load on the display control device. In contrast, point cloud information contains distance information for each point, so accurate depth information can be obtained without the need for analysis such as image processing. Therefore, the processing load is lower than that of camera images, and more precise representation is possible. As a result, images with improved entertainment value can be displayed with a low processing load.

[0043] Furthermore, the display control unit 3 switches between two display modes to display an image on the display unit 4. In this way, by appropriately switching between the two display modes, it is possible to display an image with a different display mode.

[0044] Furthermore, the display control unit 3 changes the superimposition (combination) ratio of the images in the two display modes and displays them on the display unit 4. In this way, it is possible to display an image in which the display modes of the two display modes are superimposed (combined), thereby increasing the display variation. In this embodiment, two display modes (first mode, second mode) and superimposition (combination) of images in the two display modes are used, but the present invention is not limited to this. The superimposition (combination) of images in the two display modes may be used as a third mode to switch between three display modes.

[0045] Furthermore, music is set for each of the two display modes, and the display control unit 3 changes the display mode of the image displayed on the display unit 4 in one display mode according to the music being played. In this way, the image can be displayed in a display mode that matches the image of the music.

[0046] Furthermore, the second mode of the two display modes is displayed in a display mode that is a deformed version of the display mode of the first mode. In this way, it is possible to display in a deformed display mode based on the display mode of the first mode.

[0047] Furthermore, the music information analyzed or acquired by the music analysis unit 6 includes the characteristics and genre information of the music. In this way, the display mode can be changed based on the characteristics of the music, such as tempo and rhythm, and the genre of the music.

[0048] Furthermore, the display control unit 3 changes the timing of the effects applied to the image based on the feature amount of the music. In this way, it is possible to apply or change the effects in accordance with, for example, the tempo or rhythm of the music.

[0049] If the music data includes a jacket image, the music analysis unit 6 may acquire the jacket image and change the display mode based on the jacket image. For example, the color tone may be determined based on the jacket image of the music. For example, if the jacket of the music being played uses a lot of red, the display mode may be a color tone centered around red. Alternatively, the jacket image may be included in the display mode. For example, the circular image of effects E1 and E4 in FIG. 5 may be replaced with the jacket image. Part of the text, shape, logo, etc. of the jacket image may be extracted and displayed. This allows the user to visually experience the world of the music being played.

[0050] Furthermore, effects may be changed depending on the structure of the song, such as the verse, chorus, etc., in addition to the tempo and rhythm as characteristics of the song. For example, flashy effects may be applied to the exciting parts of the chorus.

[0051] 1, the point cloud information acquired by the LIDAR 2 is processed in real time by the display control unit 3, but as shown in Fig. 7, the point cloud information may be stored in advance in the storage unit 9, and the display control unit 3A may read out the point cloud information corresponding to the current position estimated by the current position estimation unit 8 from the storage unit 9 and perform predetermined processing on the information. That is, in the case of Fig. 7, the display control device 3A functions as the point cloud information acquisition unit.

[0052] The current position estimation unit 8 estimates the current position of the vehicle in which the display control device 1A is mounted. The current position estimation unit 8 may be configured, for example, with a GPS (Global Positioning System) receiver, and may estimate position information of latitude and longitude that indicates the current position based on radio waves received from a GPS satellite.

[0053] In FIG. 7, the display control device 1A is provided with a memory unit 9, but since such 3D data has a large volume of data, it may be stored in an external server or the like, and a communication device for communicating with the external server may be provided, so that the display control unit 3A acquires point cloud information via the communication device.

[0054] Furthermore, the point cloud information acquired by the LIDAR 2 includes information indicating objects present around the vehicle, such as pedestrians and other vehicles that the driver should be aware of. Therefore, by using the point cloud information to display entertaining images, the driver can enjoy the images while also monitoring objects that require their attention. This improves the driver's sense of security.

[0055] Furthermore, the present invention is not limited to the above-described embodiments. In other words, a person skilled in the art can implement various modifications in accordance with conventionally known knowledge without departing from the gist of the present invention. As long as such modifications still comprise the configuration of the display control device of the present invention, they are of course included in the scope of the present invention. [Explanation of symbols]

[0056] 1, 1A Display control device 2. Lidar (point cloud information acquisition unit) 3, 3A Display control unit (point cloud information acquisition unit, setting unit) 4 Display 5. Music Acquisition Section 6. Music analysis unit (music information acquisition unit)

Claims

[Claim 1] A display control device that displays an image on a display unit disposed on a moving body, a point cloud information acquisition unit that acquires point cloud information representing features around the moving object using a plurality of points; a display control unit that causes the display unit to display an image based on the point cloud information in one of a plurality of display modes that have different display modes; A display control device comprising:

Images