Display System

Abstract

To provide technology for providing drive information by which a driver driving a vehicle can intuitively grasp the scenery that can be seen.SOLUTION: An exemplary moving image generation device includes an acquisition unit that acquires moving image data captured by a camera that captures a vehicle periphery, and a moving image generating unit that generates an extracted moving image of a predetermined time width by extracting a time portion of the moving image data in which scenery having a predetermined characteristic is shown, using a learnt model by machine learning in which the moving image data is input.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a moving image generating device, a display system, and a display method. [Background technology]

[0002] Conventionally, a tourist information providing device is known that provides guide information to places that a user (user) wants to visit after watching a tourist video (see, for example, Patent Document 1). Such a tourist information providing device displays, on a user terminal, a screen including a video display area for displaying a video and a map display area for displaying a map, and plays the video in response to an operation via the user terminal. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-135709 A Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, there is a need to find places where beautiful scenery can be seen while driving. Conventional methods for providing tourist information provide information on scenery that can be seen by getting out of a vehicle, such as observation decks and parking areas. For this reason, even when searching for a place to go for a drive, it was not possible for a driver (operator) to know in advance what kind of scenery the driver (driver) would see while driving and decide on a place to go.

[0005] In view of the above, an object of the present invention is to provide a technique for providing driving information that enables a driver who is driving a vehicle to intuitively grasp the scenery that can be seen. [Means for solving the problem]

[0006] An exemplary video generation device of the present invention includes an acquisition unit that acquires video data captured by a camera that captures the area around the vehicle, and a video generation unit that uses a trained model through machine learning to which the video data is input to generate an extracted video of a predetermined duration by extracting a portion of the video data in which a scene with a predetermined characteristic is shown.

[0007] An exemplary display system of the present invention comprises a moving image generating device having the above-mentioned configuration, a server device configured to be able to communicate with the moving image generating device and to acquire the extracted moving image from the moving image generating device, and a display device configured to be able to communicate with the server device and having a display screen capable of displaying image data acquired from the server device. Effect of the Invention

[0008] According to the exemplary embodiment of the present invention, it is possible to provide driving information that enables a driver who is driving a vehicle to intuitively grasp the scenery that can be seen. [Brief description of the drawings]

[0009] [Figure 1] Schematic diagram showing the configuration of a display system [Diagram 2] Block diagram showing the configuration of a video generating device [Diagram 3] A block diagram showing the configuration of a display device. [Figure 4] FIG. 1 is a first explanatory diagram for explaining an example of the operation of the display device; [Diagram 5] FIG. 2 is a second explanatory diagram for explaining an example of the operation of the display device; [Figure 6] FIG. 3 is a third explanatory diagram for explaining an example of the operation of the display device; [Figure 7] FIG. 4 is a fourth explanatory diagram for explaining an example of the operation of the display device; [Figure 8] FIG. 11 is a diagram for explaining a method of displaying a list of thumbnails in a display device according to a first modified example; [Figure 9] FIG. 11 is a diagram for explaining a further modification of the display device of the first modification. [Figure 10]FIG. 13 is a diagram for explaining a method of displaying a list of thumbnails in a display device according to a fourth modified example; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an exemplary embodiment of the present invention will be described in detail with reference to the drawings. In the following, when describing the positional relationship regarding the vehicle, the direction of the vehicle moving straight from the driver's seat to the steering wheel is referred to as the "forward direction". Also, the direction of the vehicle moving straight from the steering wheel to the driver's seat is referred to as the "rearward direction". Also, the direction perpendicular to the vehicle's moving straight and the vertical line, from the right side to the left side of the driver facing forward, is referred to as the "leftward direction". Also, the direction perpendicular to the vehicle's moving straight and the vertical line, from the left side to the right side of the driver facing forward, is referred to as the "rightward direction".

[0011] <1. Display system> Fig. 1 is a schematic diagram showing a configuration of a display system 100 according to an embodiment of the present invention. In detail, the display system 100 is a display system that displays driving information for the purpose of providing the driving information. The display system 100 may be recognized as a driving information providing system. As shown in Fig. 1, the display system 100 includes a video generating device 1, a server device 2, and a display device 3.

[0012] The video creation device 1 is mounted on a vehicle 10. The video creation device 1 acquires shooting data captured by a camera 4 mounted on the same vehicle 10 as the device itself, and creates a short movie for the purpose of providing driving information. The video creation device 1 is capable of communicating with a network 5, such as the Internet or a mobile phone network.

[0013] At least one of the video creation device 1 and the camera 4 may be configured to be permanently mounted on the vehicle 10, or may be a portable device that is not permanently mounted on the vehicle 10. The video creation device 1 and the camera 4 may be a dedicated device provided for providing driving information, or may be configured as follows. For example, at least one of the video creation device 1 and the camera 4 may be included in a drive recorder. Also, for example, at least one of the video creation device 1 and the camera 4 may be included in a mobile terminal such as a smartphone. Details of the video creation device 1 will be described later.

[0014] The server device 2 is provided so as to be able to communicate with the video production device 1. In particular, the server device 2 is provided so as to be able to communicate with the video production device 1 via a network 5. The server device 2 is able to acquire the above-mentioned short movies generated by the video production device 1 via the network 5. For the sake of convenience, FIG. 1 shows a configuration in which only the video production device 1 of one vehicle 10 is connected to the server device 2 via the network 5. However, in reality, the video production devices 1 of a plurality of vehicles 10 are connected to the server device 2 via the network 5.

[0015] The display device 3 is provided so as to be able to communicate with the server device 2. In particular, the display device 3 is provided so as to be able to communicate with the server device 2 via a network 5. The display device 3 has a display screen capable of displaying image data acquired from the server device 2. The image data may be video data or still image data. In the present embodiment, the image data is preferably video data.

[0016] The display device 3 may be a device that can be used by a user who wants to obtain driving information, and may be a device having a display screen. The display device 3 may be, for example, a mobile terminal such as a smartphone or a tablet. The display device 3 does not have to be a mobile type, and may be, for example, a set of a PC (Personal Computer) and a monitor. The display device 3 may be, for example, an in-vehicle device such as a navigation device or a display audio. In some cases, the display device 3 may be the same device as the video generating device 1. Details of the display device 3 will be described later.

[0017] <2. Video Creation Device> Next, the configuration of the above-mentioned video generation device 1 will be further described. Fig. 2 is a block diagram showing the configuration of the video generation device 1 according to an embodiment of the present invention. Note that Fig. 2 shows only components necessary for explaining the features of the embodiment, and general components are omitted. As shown in Fig. 2, the video generation device 1 includes a control unit 11, a storage unit 12, and a communication unit 13.

[0018] The control unit 11 is configured with, for example, a processor. The control unit 11 is configured to include, for example, an integrated circuit for calculations, a RAM (Random Access Memory), and a ROM (Read Only Memory). The integrated circuit for calculations may be, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an APU (Accelerated Processing Unit).

[0019] The storage unit 12 non-temporarily stores or memorizes computer-readable programs, data, etc. The storage unit 12 is at least one type of non-transient tangible storage medium, such as a semiconductor memory, a magnetic medium, an optical medium, etc. The storage unit 12 stores various programs and various data.

[0020] In this embodiment, the storage unit 12 stores a trained model 121 based on machine learning. The trained model 121 is generated in advance using a learning device (not shown) that performs machine learning, and is stored in advance in the storage unit 12. The trained model 121 is provided by an algorithm such as a neural network or a support vector machine. Details of the trained model 121 will be described later.

[0021] The communication unit 13 is wirelessly connected to the network 5 and communicates with the server device 2 via the network 5 .

[0022] The control unit 11 includes an acquisition unit 111, a moving image generating unit 112, and a transmission processing unit 113. The acquisition unit 111, the moving image generating unit 112, and the transmission processing unit 113 are functions of the control unit 11 that are realized by an integrated circuit for calculations of the control unit 11 executing calculation processing according to a computer program. In other words, the control unit 11 includes the acquisition unit 111, the moving image generating unit 112, and the transmission processing unit 113. That is, the moving image generating device 1 includes the acquisition unit 111 and the moving image generating unit 112.

[0023] At least one of the acquisition unit 111, the video generation unit 112, and the transmission processing unit 113 included in the control unit 11 may be configured with hardware such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The acquisition unit 111, the video generation unit 112, and the transmission processing unit 113 are conceptual components. The function executed by one component may be distributed to multiple components, or the functions of multiple components may be integrated into one component.

[0024] The acquisition unit 111 acquires video data captured by the camera 4 capturing images of the periphery of the vehicle 10. The acquisition unit 111 acquires the video data from the camera 4 using a wired or wireless connection. More specifically, the acquisition unit 111 acquires captured images from the camera 4 in a time-series manner at a predetermined cycle (e.g., 1 / 30 second cycle). The captured images that are continuous in time constitute a video (video). The acquisition unit 111 also appropriately performs processes required as preparations for various processes performed using the captured data.

[0025] In this embodiment, the camera 4 that captures the surroundings of the vehicle 10 is, in detail, a camera that can capture the scenery that the driver of the vehicle 10 can see while driving. In this embodiment, the camera 4 is, in a preferred embodiment, a front camera that captures the view ahead of the vehicle 10. The number of cameras 4 is not limited to one, and may be multiple. For example, the camera 4 that captures the surroundings of the vehicle 10 may include, in addition to the front camera, a left side camera and a right side camera that capture the left and right sides of the vehicle 10.

[0026] The video generation unit 112 uses the trained model 121 to which the video data captured by the camera 4 is input to generate an extracted video of a predetermined time width by extracting a time portion of the video data in which a scene with a predetermined characteristic is shown. The extracted video corresponds to the short movie described above. The scene with the predetermined characteristic is determined by the learning data (teacher data) used to generate the trained model 121. The predetermined characteristic may be, for example, a good view. The scene with the predetermined characteristic may be, for example, a scene with a beautiful ocean, a scene with a beautiful blue sky, a scene with a beautiful row of cherry blossom trees, etc.

[0027] Trained model 121 is a trained model obtained by machine learning using various photographic data (a large amount of photographic data) showing view spots evaluated as scenery with predetermined characteristics (such as a scenery that looks beautiful of the sea) as training data. A view spot, which is a requirement of training data, may be a spot that is highly rated in, for example, word-of-mouth information that can be viewed using a network such as the Internet, or in SNS (social networking services), etc. The training data used for machine learning may be video data or still image data.

[0028] The trained model 121 recognizes a scene with a predetermined characteristic, such as "a beautiful view of the sea," when the scene has the predetermined characteristic in the video (image) captured by the camera 4. The trained model 121 may recognize only one type of scene with the predetermined characteristic, but preferably recognizes multiple types. The video generation unit 112 extracts data of a time portion of the video data captured by the camera 4 that shows the scene with the predetermined characteristic recognized by the trained model 121, and generates an extracted video (short movie) of a predetermined time width. When the scene with the predetermined characteristic does not exist in the video captured by the camera 4, the video generation unit 112 does not generate a short movie.

[0029] The predetermined time span is not particularly limited, but may be, for example, about 10 seconds. The predetermined time span may be changeable. In the short movie, there is only a time period during which a scene with a predetermined characteristic is shown, and it is not necessary that the scene with the predetermined characteristic is shown throughout the entirety of the short movie.

[0030] According to the video creation device 1 of the present embodiment, it is possible to create a video that allows a user to intuitively grasp the scenery that the driver can see while driving the vehicle 10. Also, according to the video creation device 1 of the present embodiment, it is possible to create a video that allows a user to quickly grasp the scenery that the driver can see while driving the vehicle 10.

[0031] In this embodiment, the video generation device 1 is mounted on the same vehicle as the vehicle 10 on which the camera 4 is mounted. In this embodiment, the process of generating a short movie using the video of the camera 4 mounted on the vehicle 10 can be executed in the vehicle 10 without going through the network 5. That is, in this embodiment, the configuration for generating a short movie using the trained model 121 can be realized by using so-called edge AI (Artificial Intelligence).

[0032] However, the configuration for generating a short movie using the captured data of the camera 4 mounted on the vehicle 10 may be realized using a method other than edge AI. For example, the camera 4 mounted on the vehicle 10, or an in-vehicle device capable of communicating with the camera 4, and a server device provided to be capable of communicating via the network 5 may generate a short movie using a trained model. In this case, the video generating device is configured by the server device, or is configured by the in-vehicle device and the server device described above.

[0033] When a short movie is generated by the video generation unit 112, the transmission processing unit 113 performs processing to transmit data of the short movie (video data) to the server device 2. That is, the server device 2 acquires the short movie (extracted video) from the video generation device 1. The server device 2 accumulates and manages the acquired short movie data. The server device 2 can accumulate short movies of a large number of view spots because it can acquire short movie data from the video generation device 1 mounted on each vehicle 10 so as to be able to communicate with the server device itself.

[0034] As can be seen from the above, according to the display system 100 of this embodiment, the user can obtain and display data of short movies (extracted videos generated by video generating device 1) stored in the server device 2 on the display device 3. Then, by watching the video displayed on the display device 3, the user can intuitively grasp beautiful scenery and the like that the driver can see while driving at the location of the video. The user can obtain information from the driver's perspective and select a driving course.

[0035] <3.Display device> Fig. 3 is a block diagram showing a configuration of a display device 3 according to an embodiment of the present invention. In Fig. 3, only components necessary for explaining the features of the embodiment are shown, and descriptions of general components are omitted. As shown in Fig. 3, the display device 3 includes a control unit 31, a storage unit 32, a communication unit 33, a display unit 34, and an operation unit 35.

[0036] The control unit 31 is a controller that performs overall control of the entire display device 3. The control unit 31 is configured, for example, by a processor. The control unit 31 is configured, for example, to include an integrated circuit for calculations, a RAM, and a ROM. The integrated circuit for calculations may be, for example, a CPU, a GPU, or an APU.

[0037] The storage unit 32 non-temporarily stores or memorizes computer-readable programs, data, etc. The storage unit 32 is at least one type of non-transient substantial storage medium, such as a semiconductor memory, a magnetic medium, an optical medium, etc. The storage unit 32 stores various application software (apps) and various data.

[0038] The communication unit 33 is connected to the network 5 wirelessly or via a wired connection, and communicates with the server device 2 via the network 5.

[0039] The display unit 34 is configured using, for example, a liquid crystal panel, an organic EL panel, etc. The display unit 34 has a display screen that displays image data (moving image data or still image data) acquired from the server device 2.

[0040] The operation unit 35 enables a user to operate the display device 3. The operation unit 35 may be configured with at least one of, for example, a touch panel, a button, a lever, a rotating body, a keyboard, and a voice input device.

[0041] In this embodiment, the display device 3 is provided to be capable of providing driving information and setting a driving route related to the location for which the driving information is provided. The display device 3 may be recognized as a driving information providing device. Hereinafter, the operation of the display device 3 will be described using the example in which the display device 3 is a smartphone or a tablet.

[0042] Fig. 4 is a first explanatory diagram for explaining an example of the operation of the display device 3 of this embodiment. Fig. 5 is a second explanatory diagram for explaining an example of the operation of the display device 3 of this embodiment. Fig. 6 is a third explanatory diagram for explaining an example of the operation of the display device 3 of this embodiment. Fig. 7 is a fourth explanatory diagram for explaining an example of the operation of the display device 3 of this embodiment.

[0043] 4 to 7 are front views of the display device 3 configured as a smartphone or a tablet. The operation unit 35 of the display device 3 is configured as a touch panel, and the user can operate the display device 3 by touching the display screen 340. As the operation of the display device 3 progresses, the display screen 340 changes in the order of FIG. 4, FIG. 5, FIG. 6, and FIG. 7.

[0044] 4 shows a state before a user starts using an application that provides driving information (hereinafter, referred to as a driving information providing application). A plurality of icons 341 for starting various applications are displayed on a display screen 340 of the display device 3. The plurality of icons 341 includes an icon 341a for starting the driving information providing application. When the user touches the icon 341a, the driving information providing application is started.

[0045] Fig. 5 is a diagram showing an example of a display screen 340 displayed by starting up an application for providing driving information. As shown in Fig. 5, a menu screen is displayed by starting up the driving information providing application. A plurality of selection keys (icons) 342 are displayed on the menu screen. The plurality of selection keys 342 includes a selection key 342a for "view viewing spot" that displays a screen that provides viewing spot information, which is a type of driving information. When the user touches the selection key 342a, viewing spot information is displayed on the display screen 340 according to a predetermined condition.

[0046] FIG. 6 is a diagram showing a state in which the viewing spot information is displayed on the display screen 340 according to a predetermined condition. As shown in FIG. 6, the display device 3 displays thumbnails 343 on the display screen 340. The number of thumbnails 343 displayed on the display screen 340 may be one or more. In this embodiment, the display device 3 displays a list of multiple thumbnails 343 on the display screen 340. In detail, the display device 3 displays a list of multiple thumbnails 343 on the display screen 340 so that the user can select them. In the example shown in FIG. 6, six thumbnails 343 are displayed in a list arrangement in which three are vertically arranged on the left and right sides of the display screen 340. The three thumbnails 343 on the left and right sides are arranged in symmetrical positions with respect to the bisector that divides the display screen 340 into two equal parts. The six thumbnails 343 have the same size. However, the number, size, and arrangement of the thumbnails 343 displayed on the display screen 340 may be changed as appropriate.

[0047] The predetermined condition may be a fixed condition, or may be configured to be changeable according to the user's desire. In this example, the predetermined condition is set to be changeable by a selection operation by the user. In detail, a setting section 344 for setting the predetermined condition is displayed in the upper part of the display screen 340. More specifically, the setting section 344 is disposed above the thumbnail display area in which the multiple thumbnails 343 are displayed. However, the position of the setting section 344 may be changed as appropriate. Also, the setting section 344 does not necessarily have to be displayed together with the thumbnails 343. The user can set the predetermined condition by touching the setting section 344.

[0048] In the example shown in FIG. 6, the predetermined condition is set to "display in order of proximity." In this case, images of view spots located in positions closest to the user's current location are selected as images to be displayed on the display screen 340 in order (in order of proximity). The predetermined condition may be, for example, order of proximity to the destination, recommended mountain spots, recommended sea spots, recommended cherry blossom tree-lined spots, recommended spots according to the season, etc. For example, if recommended mountain spots are set as the predetermined condition, images of places where beautiful mountain scenery can be seen from the driver's eye level while driving are preferentially displayed on the display screen 340.

[0049] That is, in this embodiment, the display device 3 is provided so that the categories of objects to be displayed as thumbnails 343 on the display screen 340 can be changed. With this configuration, the user can efficiently find a driving course that has scenic spots that match his / her preferences.

[0050] In detail, when the selection key 342a "View View Spot" is selected by the user, the control unit 31 of the display device 3 performs a process of transmitting the predetermined condition and information related thereto to the server device 2. In the example shown in FIG. 6, the information related to the predetermined condition is the user's current location information. The current location information is acquired, for example, using a GPS (Global Positioning System) sensor provided in the display device 3. The information related to the predetermined condition may be transmitted as necessary, and may not be transmitted depending on the content of the predetermined condition.

[0051] Furthermore, together with transmitting the predetermined conditions, the control unit 31 requests the server device 2 to transmit a view spot image corresponding to the predetermined conditions. In response to the request from the display device 3, the server device 2 transmits a view spot image corresponding to the predetermined conditions to the display device 3. In this embodiment, the view spot image is selected from short movies generated by the above-mentioned video generation device 1 and stored in the server device 2. The server device 2 extracts short movies that meet the predetermined conditions from the stored data and transmits them to the display device 3.

[0052] In this embodiment, the display device 3 downloads moving images from the server device 2 and displays each moving image as a thumbnail. More specifically, the display device 3 displays the downloaded moving images as thumbnails of moving images. In other words, the display device 3 displays the downloaded moving images as thumbnails in a playing state. That is, all six thumbnails 343 shown in FIG. 6 are displayed in a playing state. In this embodiment, the moving images are about 10 seconds long and are played repeatedly. Note that the moving images are, more specifically, extracted moving images (short movies) of a predetermined time width generated by the moving image generating device 1.

[0053] That is, the display device 3 displays a list of multiple thumbnails 343 showing the extracted moving images as moving images (playback state) on the display screen 340. With this configuration, a user looking at the display screen 340 of the display device 3 can intuitively grasp in a short time what kind of scenery can be seen at each of the multiple viewing spots.

[0054] In this embodiment, the multiple thumbnails 343 are displayed in a selectable manner as described above. That is, the user can touch any one of the thumbnails 343 displayed on the display screen 340 to view details of the drive information displayed in the touched thumbnail 343. Fig. 7 is a diagram showing the state of the display screen 340 when one of the multiple thumbnails 343 in Fig. 6 is selected by the user.

[0055] As shown in Fig. 7, when a certain thumbnail 343 is selected, a video area R1 and a map area R2 are displayed on the display screen 340. In Fig. 7, the video area R1 is displayed on the upper side of the screen and the map area R2 is displayed on the lower side of the screen, but this up-down relationship may be reversed. Also, the video area R1 and the map area R2 may be arranged horizontally instead of vertically.

[0056] A video relating to the view spot displayed by the selected thumbnail 343 is displayed in the video area R1. The video may be, for example, a video from the driver's perspective when driving a section longer than the short movie displayed by the thumbnail 343. The data for the video is obtained by downloading from the server device 2. In this example, the server device 2 has a configuration in which, for each driving course, a short movie and a long movie that is longer than the short movie. The long movie may be, for example, the entire duration of the video used by the video generation device 1 when generating the short movie.

[0057] In video area R1, playback of the video may be started in conjunction with the selection of thumbnail 343, or a still image may be displayed first, and the video may be played when the user touches a play icon displayed superimposed on the still image. Also, in video area R1, the same video as that displayed in thumbnail 343 may be played by enlarging the display area.

[0058] A map of the location including the view spot displayed by the selected thumbnail 343 is displayed in the map area R2. It is preferable that the map has an indication of the location introduced in the short movie and the location of the driving course superimposed on the map. The indication of the location may be, for example, an indication of a pin pointing to the location. The indication of the location of the driving course may be, for example, an indication of a colored line superimposed on a road on the map.

[0059] As can be seen from the above description, in this embodiment, when one of the multiple thumbnails 343 displayed on the display screen 340 is selected, the display device 3 displays a confirmation screen that enables confirmation of the location of the scenery shown in the selected thumbnail 343. With this configuration, the user can easily check the details of a driving course that interests the user after viewing the video of the thumbnail 343. It is possible to provide the user with information that makes it easier to decide whether or not to go on the driving course that interests him or her. The display screen 340 shown in FIG. 7 is an example of a confirmation screen that enables confirmation of the location of the scenery shown in the selected thumbnail 343.

[0060] In the example shown in Fig. 7, the road names of the driving course are displayed on the upper side of the screen of the video area R1, and the address of the end point of the driving course, etc. is displayed on the upper side of the screen of the map area R2 (the lower side of the screen of the video area R1). These displays are not necessarily required, but by displaying them, more information can be provided to the user.

[0061] 7, a registration icon 345 that allows the driving course displayed in the map area R2 (more specifically, for example, the end point of the driving course) to be registered as a destination is displayed at the bottom of the display screen 340. When the user touches the registration icon 345, route information is transferred to a navigation device (not shown) installed in a vehicle used by the user of the display device 3. This allows the route to be automatically set in the navigation device installed in the vehicle.

[0062] That is, when one of the thumbnails 343 displayed on the display screen 340 is selected, the display device 3 displays a navigation setting screen that enables a user to set, as a guide route, a route that passes through a location with a view shown in the selected thumbnail 343. With this configuration, the user can easily set navigation for a driving course that interests him or her after watching the video of the thumbnail 343.

[0063] 7 is an example of a navigation setting screen that allows a route that passes through the location of the scenery shown in the selected thumbnail 343 to be set as a guide route. In the present embodiment, when the user touches the registration icon 345, route information is transferred to a device (navigation device) that exists separately from the display device 3, but this is an example. When the user touches the registration icon 345, route setting in the navigation function of the display device 3 may be automatically performed. For example, such a configuration can be adopted when the display device 3 also functions as a navigation device.

[0064] <4. Modifications> (4-1. First modified example) In the first modified example, the display device 3A also displays a list of multiple thumbnails 343A on the display screen 340A. However, the multiple thumbnails 343A include a moving image thumbnail 3431 that displays the extracted moving image (short movie) as a moving image (playback state), and a still image thumbnail 3432 that displays some of the still images that make up the extracted moving image. In this respect, it differs from the above-mentioned embodiment.

[0065] If multiple thumbnails 343A are all simultaneously played back as moving images, there is a concern that the amount of data communication will increase, it will take a long time to download, and it will take a long time until all of the thumbnails 343A can be played back as moving images. There is also a concern that the load on the server device 2 will increase. Furthermore, there is a concern that the amount of data communication will increase, and the user's available data capacity will be consumed more quickly. In this regard, if still image thumbnails 3432 are included in the thumbnails 343A displayed in a list as in this modified example, the amount of data communication can be reduced, and the above concerns can be alleviated.

[0066] A specific example of a configuration in which a plurality of thumbnails 343A includes a video thumbnail 3431 and a still image thumbnail 3432 will be described with reference to Fig. 8. Fig. 8 is a diagram for explaining a method of displaying a list of thumbnails 343A in a display device 3A of a first modified example. More specifically, Fig. 8 is a diagram for explaining a display method for displaying a list of a plurality of thumbnails including video thumbnails and still image thumbnails on a display screen. Note that in this modified example as well, the display device 3A is configured as a smartphone or a tablet as an example.

[0067] In the example shown in Fig. 8, multiple thumbnails 343A are displayed in a vertical line on the display screen 340A. In detail, three thumbnails 343A are displayed in a vertical line on the display screen 340A. However, the number of thumbnails 343A displayed on the display screen 340A may be changed as appropriate. Also, in the example shown in Fig. 8, of the three thumbnails 343A displayed in a vertical line, the middle thumbnail 343A is displayed larger than the other thumbnails 343A. However, this is merely an example, and the thumbnails 343A displayed on the display screen 340A may all be the same size.

[0068] 8, the thumbnail 343A located at the center of the multiple thumbnails 343A is a video thumbnail 3431. This configuration makes it possible to reduce the amount of communication required to obtain image data, while arranging the video in a playing state at an easy-to-view position on the display screen 340. In other words, it is possible to achieve both a reduction in the amount of data communication and user convenience.

[0069] The thumbnail 343A located at the center among the multiple thumbnails 343A may be, for example, one thumbnail 343A located at the center of the row when the number of thumbnails 343A lined up in a row is odd. Also, the thumbnail 343A located at the center among the multiple thumbnails 343A may be, for example, two thumbnails 343A located at the center of the row when the number of thumbnails 343A lined up in a row is even. Also, the thumbnail 343A located at the center among the multiple thumbnails 343A may be, for example, a thumbnail 343A located at a predetermined one side of two thumbnails 343A located at the center of the row when the number of thumbnails 343A lined up in a row is even. Also, the thumbnail 343A located at the center among the multiple thumbnails 343A may be, for example, one thumbnail 343A located at a position overlapping with the center position of the display screen 340A in the direction in which the thumbnails 343A are lined up. Furthermore, the thumbnail 343A located in the center of the multiple thumbnails 343A may be, for example, at least one thumbnail 343A that overlaps with a predetermined range including the center position of the display screen 340A in the direction in which the thumbnails 343A are arranged.

[0070] 8, of the three thumbnails 343A arranged in a vertical row, only the middle thumbnail 343A is a video thumbnail 3431, and the thumbnails 343A above and below are still image thumbnails 3432. That is, only the middle thumbnail 343A is displayed in a state in which a video is being played. The still image displayed in the still image thumbnail 3432 is one frame image among a plurality of frame images constituting a short movie (video), for example, the first frame image among the plurality of frame images.

[0071] Also, in the example shown in Fig. 8, the screen can be scrolled up and down in the area where thumbnail 343A is displayed. That is, thumbnail 343A located in the center changes due to scrolling. For example, when the screen is scrolled from the state shown in Fig. 8 and thumbnail 343A located at the top in Fig. 8 is moved to the center, thumbnail 343A moved to the center is displayed in a playback state. Thumbnail 343A that was in the center before scrolling and has been moved to the bottom of display screen 340A is displayed as a still image.

[0072] 8, multiple thumbnails 343A are displayed in a vertical row, but multiple thumbnails 343A may be displayed in a horizontal row as shown in Fig. 9. In this case as well, the thumbnail 343A located in the center of the multiple thumbnails 343A may be set as a video thumbnail 3431.

[0073] Fig. 9 is a diagram for explaining a further modified example of the display device 3A of the first modified example. In the configuration shown in Fig. 9, it is preferable that the screen can be scrolled left and right in the area where the thumbnail 343A is displayed, similarly to the configuration shown in Fig. 8. The screen state shown in Fig. 9 may be automatically changed by rotating the display device 3A (display screen 340A) by 90° from the state shown in Fig. 8.

[0074] (4-2. Second Modification) In the second modified example, the display device also displays at least one thumbnail on the display screen. In detail, the display device displays a list of multiple thumbnails on the display screen. In the second modified example, the display screen on which the list of multiple thumbnails is displayed may be substantially similar to the display screen 340 shown in FIG. 6.

[0075] However, each of the multiple thumbnails displays a part of still images extracted from multiple still images (frame images) constituting the extracted video (short movie) by switching them in order. This point differs from the above-mentioned embodiment. By configuring in this way, it is possible to reduce the amount of data communication, for example, to reduce the processing load of the server device 2 and to reduce the consumption rate of the communication capacity owned by the user.

[0076] In detail, the image data constituting each thumbnail is a capture of only a few frame images among all the frame images constituting the short movie. For example, the frame images are captured at intervals of one second. In each thumbnail, the captured frame images are displayed in chronological order in a picture-story show format. In this configuration, since there is a loss of data compared to a short movie, each thumbnail displays a video with somewhat unnatural movements. However, even in this case, the user can intuitively grasp the beautiful scenery that the driver can see while driving by looking at the thumbnail images. In other words, it is possible to achieve both reduction in data communication volume and convenience for the user.

[0077] (4-3. Third Modification) In the third modified example, the display device also displays at least one thumbnail on the display screen. In detail, the display device displays a list of multiple thumbnails on the display screen. In the third modified example, the display screen on which the list of multiple thumbnails is displayed may be substantially similar to the display screen 340 shown in FIG. 6.

[0078] However, each of the multiple thumbnails displays a short video obtained by extracting a portion of the extracted video (short movie) as a video. This differs from the above-described embodiment. Displaying as a video means displaying in a playback state. This configuration can reduce the amount of data communication, and can reduce, for example, the processing load on the server device 2 and the consumption rate of the communication capacity owned by the user.

[0079] In detail, the image data constituting each thumbnail is short video data obtained by extracting only a part of the time portion from all the image data constituting the short movie. For example, if the short movie is about 10 seconds long, the short video may be about 1 second long. The time portion extracted from the short movie is preferably a portion showing a beautiful landscape (a portion showing a view spot). With this configuration, the amount of data to be downloaded can be reduced compared to downloading a short movie, so that the amount of data communication can be reduced. Even in this case, the user can intuitively grasp the beautiful landscape that the driver can see while driving. In other words, it is possible to achieve both the suppression of the amount of data communication and the convenience of the user.

[0080] The short video displayed in each thumbnail may be simply played back repeatedly, or may be played back and reversed repeatedly. This configuration can prevent frequent blank periods in the thumbnail area, where no image is displayed, from occurring, making it easier for the user to view the screen.

[0081] (4-4. Fourth Modification) 10 is a diagram for explaining a method of displaying a list of thumbnails 343B in a display device 3B of the fourth modified example. In the fourth modified example, the display device 3B also displays a list of multiple thumbnails 343B on a display screen 340B. However, each of the multiple thumbnails 343B is displayed as a still image. That is, each thumbnail 343B is a still image thumbnail 3432. In this respect, it differs from the above-mentioned embodiment.

[0082] In detail, the still image displayed on each thumbnail 343B is one of a plurality of frame images constituting a short movie (video), for example, the first frame image of the plurality of frame images. A play icon 346 is displayed superimposed on each thumbnail 343B. When the user touches the play icon 346, the entire data of the short movie is downloaded from the server device 2 and the short movie is put into a play state. When a video is being downloaded, a display showing the progress of the download may be displayed superimposed on the thumbnail 343B.

[0083] This modified example configuration can reduce the amount of data to be downloaded compared to the case where all the short movies of the thumbnails are downloaded at once, and therefore the amount of data communication can be reduced. Even in this case, the user can intuitively grasp the beautiful scenery that the driver can see while driving by switching the still image to the video. In other words, it is possible to achieve both the suppression of the amount of data communication and the convenience of the user.

[0084] <5. Points to note> In addition to the above embodiment, various technical features disclosed in this specification can be modified in various ways without departing from the spirit of the technical creation. In other words, the above embodiment should be considered to be illustrative in all respects and not restrictive, and the technical scope of the present invention is indicated by the claims, not the description of the above embodiment, and should be understood to include all modifications that fall within the meaning and scope of the claims. In addition, multiple embodiments and modifications shown in this specification may be implemented in appropriate combinations to the extent possible.

[0085] In the above, the short movie is configured to be generated by the video production device 1, but the short movie may be generated by the server device 2. In addition, the short movie may be generated by a user and uploaded to the server device 2. [Explanation of symbols]

[0086] 1. Video generation device 2. Server device 3, 3A, 3B...display device 4. Camera 100···Display System 111...Acquisition part 112...Video generation unit 121...Pre-trained model 340, 340A, 340B...display screen 343, 343A, 343B...Thumbnail 3431···Video thumbnail 3432...Still image thumbnail

Claims

1. a video generation device that acquires video data captured by a camera that captures the periphery of a vehicle, and uses a trained model obtained by machine learning various captured data showing a view spot evaluated as a view with a predetermined characteristic as training data to generate an extracted video of a predetermined duration by extracting a time portion of the video data in which the view with the predetermined characteristic is shown; a display device that displays a thumbnail of the extracted moving image as a moving image on a display screen; A display system comprising:

2. The display system described in claim 1, wherein the video generating device is mounted on the same vehicle as the vehicle in which the camera is mounted.

3. A server device capable of communicating with the video generation device and acquiring the extracted video from the video generation device, The display system according to claim 1 , wherein the display device is provided so as to be able to communicate with the server device, and acquires the extracted moving image from the server device.

4. The display system according to claim 1 , wherein the display device displays a list of the thumbnails on the display screen.

5. The display system according to claim 1 , wherein the display device displays a list of multiple thumbnails on the display screen, the thumbnail being a video thumbnail and a still image thumbnail displaying a portion of a still image that constitutes the extracted video.

6. The plurality of thumbnails are displayed in a vertical or horizontal row on the display screen; The display system according to claim 5 , wherein a thumbnail located at a center of the plurality of thumbnails is the video thumbnail.

7. A display system described in any one of claims 1 to 3, wherein the thumbnails display a selection of still images extracted from a plurality of still images constituting the extracted video by switching between them in sequence.

8. A display system as described in any one of claims 1 to 3, wherein the thumbnail displays a short video obtained by extracting a portion of the extracted video as a video.

9. The display system according to claim 1 , wherein when a thumbnail displayed on the display screen is selected, the display device displays a confirmation screen that enables confirmation of the location of the scenery shown in the thumbnail.

10. 10. The display system according to claim 1, wherein when a thumbnail displayed on the display screen is selected, the display device displays a navigation setting screen that enables a route that passes through a location with a view shown in the thumbnail to be set as a guide route.

11. The display system according to claim 1 , wherein the display device is provided so that a category of the subject to be displayed as the thumbnail on the display screen can be changed.

Images