Video distribution server and video distribution system

Abstract

To provide a moving image distribution server and a moving image distribution system capable of distributing in real time a landscape viewed by a passenger of a train, on a terminal of a viewer in a proper state.SOLUTION: A server 100 is a moving image distribution server which distributes in real time a distribution moving image being a moving image of a landscape viewed by a passenger of an electric train 400 as a train, on a terminal 300 of a viewer. The server 100 comprises: acquisition means which acquires a captured moving image being a moving image obtained by capturing the moving image of the landscape viewed by the passenger of the electric train 400, using a camera 200 loaded on the electric train 400; generation means which generates the distribution moving image on the basis of the captured moving image; distribution image which distributes the distribution moving image to a terminal; and sudden braking detection means which detects sudden braking of the electric train 400. The generation means includes braking processing means for applying a process of invisualizing at least a part of the captured moving image when the sudden braking detection means detects sudden braking.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a video distribution server and a video distribution system, and more particularly to a video distribution server and a video distribution system that distribute landscape videos to a viewer's terminal in real time. [Background technology]

[0002] Passengers on a train see the scenery outside the train window, or the scenery inside the train, such as the seats, advertisements, instruments at the driver's seat, and other passengers. By seeing these scenery, passengers experience riding on a train.

[0003] People cannot experience riding a train unless they travel to an area where the train runs on that line, making it difficult for people to experience riding a train that runs on a line in an area far away from where they live.

[0004] Train lines with low service are called local lines. Generally, local trains operate in areas far from urban areas and operate only a few times per day. This makes it particularly difficult for people to experience riding a train that runs on a local line. On the other hand, the scenery outside the train window is often scenic, and the facilities inside the train are also unique. For this reason, local lines are popular with tourists, and many people would like to ride a train that runs on a local line.

[0005] A technique for distributing captured video to a viewer's terminal is disclosed in, for example, Patent Document 1 below. Patent Document 1 below discloses an imaging device that streams video captured by an imaging unit in real time to a management server via a mobile terminal device and a communication network. When a specific operation is performed during streaming, this imaging device replaces the video being distributed with an image corresponding to the specific operation for a specific period of time. The management server transmits the received video to a playback terminal. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2013-243531 A Summary of the Invention [Problem to be solved by the invention]

[0007] A conventional method for simulating the experience of riding a train involves train passengers recording the scenery they see with a camera in advance and providing the recorded video to the viewer's terminal. However, the video scenery provided by this method is a train scenery recorded before the viewing time. For this reason, the viewer cannot experience the season or weather around the train in real time.

[0008] Another possible method is for train passengers to record the scenery they see with a camera and provide the recorded video to viewers' devices in real time. However, there is a risk that the video provided in this way may include inappropriate video that should not be provided. Inappropriate video that should not be provided is typically a video of a train accident or the confusion that accompanies an accident. If this type of video were distributed, it would cause psychological shock to viewers.

[0009] For this reason, with conventional technology, it has been difficult to deliver the scenery seen by train passengers in an appropriate condition to the viewer's terminal in real time.

[0010] The present invention is made to solve the above-mentioned problems, and its object is to provide a video distribution server and a video distribution system that can distribute the scenery seen by train passengers in an appropriate condition to the viewer's terminal in real time. [Means for solving the problem]

[0011] A video distribution server according to one aspect of the present invention is a video distribution server that distributes a distribution video, which is a video of the scenery seen by train passengers, to a viewer's terminal in real time, and is equipped with an acquisition means for acquiring a shot video, which is a video of the scenery seen by train passengers captured by a camera mounted on the train, a creation means for creating a distribution video based on the shot video, a distribution means for distributing the distribution video to the terminal, and a sudden braking detection means for detecting sudden braking of the train, and the creation means includes a braking processing means for performing a process to make at least a portion of the shot video invisible when sudden braking is detected by the sudden braking detection means.

[0012] In the video distribution server, the captured video preferably includes a video of a scene outside the train.

[0013] The above-mentioned video distribution server preferably further includes a location information acquisition means for acquiring train location information, and the creation means further includes an external information addition means for adding information related to the scenery outside the train contained in the filmed video based on the train location information.

[0014] Preferably, the above-mentioned video distribution server further includes an object storage means for storing position information of a specific object, and the creation means further includes an object processing means for applying processing to a specific object included in the captured video to make it invisible based on the train position information and the position information of the specific object.

[0015] In the video distribution server, the shot video preferably includes a video of the interior of a train.

[0016] Preferably, the above-mentioned video distribution server further includes an interior information storage means for storing interior information which is information describing objects inside the train contained in the filmed video, and the creation means further includes an information addition means for adding the interior information to the filmed video.

[0017] The video distribution server preferably further comprises a face storage means for storing human faces, and a face recognition means for recognizing human faces included in the captured video, and the creation means further includes a face processing means for applying processing to make invisible human faces that are different from the faces stored in the face storage means among the human faces recognized by the face recognition means.

[0018] In the above video distribution server, the filmed video preferably includes a plurality of filmed video fragments each of which is video shot by a plurality of cameras, and the creation means creates the distribution video by combining the plurality of filmed video fragments together.

[0019] A video distribution system according to another aspect of the present invention includes the video distribution server described above, a camera, and a terminal.

[0020] In the above moving image distribution system, the terminal preferably includes operation receiving means for receiving an operation, and display means for displaying the distributed moving image in a mode according to the operation received by the operation receiving means. Effect of the Invention

[0021] According to the present invention, it is possible to provide a video distribution server and a video distribution system that can distribute the scenery seen by train passengers in an appropriate state to a viewer's terminal in real time. [Brief description of the drawings]

[0022] [Figure 1] 1 is a diagram conceptually illustrating a configuration of a video distribution system in one embodiment of the present invention. [Diagram 2] 1 is a block diagram showing a configuration of a video distribution system according to an embodiment of the present invention. [Diagram 3] 10 is a flowchart showing the operations of the server 100 and the terminal 300 when viewing of a distributed moving image is started in an embodiment of the present invention. [Figure 4] 2 is a diagram showing a schematic viewer information table stored in storage unit 130 of server 100. FIG. [Diagram 5]4 is a flowchart showing the operation of the server 100 when creating a moving image to be distributed in one embodiment of the present invention. [Figure 6] FIG. 2 is a diagram showing a schematic diagram of a forward-viewing video captured by a camera 200a. [Figure 7] 2 is a diagram showing a schematic diagram of a first portion of a video shot in a passenger compartment by a camera 200b. FIG. [Figure 8] 13 is a diagram showing a schematic diagram of a second portion of the passenger compartment shot video created by camera 200b. FIG. [Figure 9] This is a subroutine of the process (S107) of creating a video to be distributed in FIG. [Figure 10] FIG. 10 is a diagram showing a schematic diagram of a moving image after mosaic is added to the forward-shot moving image (FIG. 6) in step S157 in FIG. [Figure 11] 2 is a diagram showing a schematic view information table stored in a storage unit 130 of a server 100. FIG. [Figure 12] FIG. 10 is a diagram illustrating a schematic diagram of a video after scenery information is added to the video shot in the passenger compartment (FIG. 8) in step S161 of FIG. 9. [Figure 13] FIG. 10 is a diagram showing a schematic diagram of a video after scenery information is added to a video shot in a passenger compartment (different from the videos in FIGS. 7 and 8) in step S161 in FIG. 9. [Figure 14] 2 is a diagram showing a mosaic information table stored in a storage unit 130 of a server 100. FIG. [Figure 15] FIG. 10 is a diagram showing a schematic diagram of a moving image after mosaic is added to the forward-shot moving image (FIG. 6) in steps S165 and S167 in FIG. [Figure 16] 9. FIG. 10 is a diagram showing a schematic diagram of a moving image obtained by adding interior information to the forward-shooting moving image (FIG. 6) in step S169 in FIG. [Figure 17] 8 is a diagram showing a schematic diagram of interior information added to the hanging advertisement AD of the video shot in the passenger compartment of FIG. 7. FIG. [Figure 18] 2 is a diagram illustrating a face table stored in a storage unit 130 of a server 100. FIG. [Figure 19]FIG. 10 is a diagram showing a schematic diagram of a video after mosaic is added to the video shot in the passenger compartment (FIG. 7) in step S175 of FIG. 9. [Figure 20] 10A to 10C are diagrams showing a schematic diagram of changes in a distributed moving image displayed on display 340 of terminal 300. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0024] [Video distribution system configuration]

[0025] FIG. 1 is a diagram conceptually showing the configuration of a moving image distribution system in one embodiment of the present invention.

[0026] With reference to Fig. 1, the video distribution system in this embodiment includes a server 100 (an example of a video distribution server), a camera 200 (an example of a camera), and a plurality of terminals 300 (terminals 300). The server 100, the camera 200, and each of the plurality of terminals 300 can communicate with each other by wireless communication. The wireless communication between the server 100, the camera 200, and each of the plurality of terminals 300 may use a fifth generation mobile communication system (5G), a wireless LAN (Local Area Network), or the like. The number of each of the cameras 200 and the terminals 300 is arbitrary.

[0027] Camera 200 is mounted on train 400 (an example of a train). Camera 200 creates a captured video by capturing the scenery seen by passengers on train 400. The captured video is a video captured by camera 200 mounted on train 400 of the scenery seen by passengers on train 400. Camera 200 transmits the created captured video to server 100 in real time. Note that train 400 may be any train. Trains include electric trains, diesel railcars, passenger cars, battery trains, hybrid trains, diesel locomotives, steam locomotives, and electric locomotives.

[0028] In this embodiment, a plurality of cameras 200a and 200b are mounted on train 400 as camera 200. Camera 200a is provided inside driver's cab 401 of train 400. Camera 200b is provided inside passenger compartment 402 of train 400. Note that camera 200c may also be mounted on train 400 as camera 200. Camera 200c is provided on the exterior front of the body of train 400.

[0029] Hereinafter, the captured video created by camera 200a or camera 200c may be referred to as a “forward captured video,” and the captured video created by camera 200b may be referred to as a “cabin captured video.”

[0030] When camera 200b is installed inside passenger compartment 402 of train 400, there is a risk that passengers inside passenger compartment 402 may be included in the captured video. For this reason, the operator of train 400 may set the fare collected from passengers riding in passenger compartment 402 where camera 200b is installed lower than the fare collected from passengers riding in passenger compartment 403 where camera 200b is not installed. In this case, passengers who agree to be filmed can be given the benefit of a fare discount, while the wishes of passengers who do not want to be filmed can be respected.

[0031] Server 100 distributes a distribution video, which is a video of the scenery seen by passengers on train 400, to destination terminal 300 in real time. Server 100 acquires captured video from camera 200 and creates a distribution video based on the acquired captured video. Server 100 creates a distribution video based on the acquired captured video and distributes the created distribution video to destination terminal 300 in real time.

[0032] Each of the multiple terminals 300 is a terminal owned by a viewer who watches the distributed video. The destination terminal 300 displays the distributed video distributed by the server 100. Each of the multiple terminals 300 is a mobile terminal, a smartphone, a PC (Personal Computer), a VR (Virtual Reality) headset, or the like.

[0033] FIG. 2 is a block diagram showing a configuration of a moving image distribution system according to an embodiment of the present invention.

[0034] 2, server 100 is a mobile terminal, a smartphone, a PC, or the like. Server 100 includes control unit 110, image memory 120, storage unit 130 (an example of object storage means, in-vehicle information storage means, and face storage means), communication unit 140 (an example of acquisition means and distribution means), recognition unit 150, and image processing unit 160 (an example of creation means). Control unit 110, image memory 120, storage unit 130, communication unit 140, recognition unit 150, and image processing unit 160 are each connected to each other and can communicate with each other.

[0035] The control unit 110 controls the overall operation of the server 100. The control unit 110 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). The CPU controls the entire server 100. The CPU executes a control program stored in the ROM. The ROM stores various programs executed by the CPU and various fixed data. The RAM is used to temporarily store data required when the CPU executes the control program.

[0036] The image memory 120 temporarily stores captured moving images received from the camera 200, moving images created by the image processing unit 160, and the like.

[0037] The storage unit 130 stores various information such as a server program 131 which is a program for controlling operations related to video distribution performed by the server 100.

[0038] Communication unit 140 communicates with each of camera 200 and terminal 300. Communication unit 140 receives various information such as captured video from camera 200. Communication unit 140 distributes distribution video to terminal 300, which is the distribution destination.

[0039] The recognition unit 150 recognizes what an object is included in the captured video based on the features of the object obtained by deep learning, etc. The recognition unit 150 may recognize what an object is by using AI (Artificial Intelligence).

[0040] The image processing unit 160 performs various image processing operations and creates a moving image to be distributed based on the captured moving image.

[0041] Camera 200 includes control unit 210, imaging unit 220, position information acquisition unit 230 (an example of position information acquisition means), communication unit 240, audio recording unit 250, and sudden braking detection unit 260 (an example of sudden braking detection means). Control unit 210, imaging unit 220, position information acquisition unit 230, communication unit 240, audio recording unit 250, and sudden braking detection unit 260 are each connected to each other and can communicate with each other.

[0042] Control unit 210 controls the overall operation of camera 200. Control unit 210 includes a CPU, a ROM, and a RAM. The CPU controls the entire camera 200. The CPU executes a control program stored in the ROM. The ROM stores various programs executed by the CPU and various fixed data. The RAM is used to temporarily store data required when the CPU executes the control program.

[0043] The photographing unit 220 creates a photographed video by photographing the scenery seen by passengers on the train 400.

[0044] The location information acquisition unit 230 acquires location information of the train 400 using a GPS (Global Positioning System) function, the running distance of the train 400, and the like.

[0045] The communication unit 240 communicates with external devices such as the server 100. In particular, the communication unit 240 transmits the captured video to the server 100.

[0046] The sound recording unit 250 records sounds around the camera 200 .

[0047] The sudden braking detection unit 260 detects sudden braking of the train 400. The sudden braking detection unit 260 may include an acceleration sensor, and may detect sudden braking based on the magnitude of negative acceleration applied to the train 400 during deceleration. The sudden braking detection unit 260 may include a sensor that detects the rotation of the wheels of the train 400, and may detect sudden braking based on a negative change in the rotation speed of the wheels of the train 400 during deceleration. Furthermore, the sudden braking detection unit 260 may include a sound level meter, and may detect sudden braking based on sounds generated by sudden braking (for example, friction sounds between the tracks and the wheels).

[0048] Each of the multiple terminals 300 includes a control unit 310, a storage unit 320, an operation unit 330 (an example of an operation receiving means), a display unit 340 (an example of a display means), and a communication unit 350. The control unit 310, the storage unit 320, the operation unit 330, the display unit 340, and the communication unit 350 are each connected to each other and can communicate with each other.

[0049] The control unit 310 controls the overall operation of the terminal 300. The control unit 310 includes a CPU, a ROM, and a RAM. The CPU controls the entire terminal 300. The CPU executes a control program stored in the ROM. The ROM stores various programs executed by the CPU and various fixed data. The RAM is used to temporarily store data required when the CPU executes the control program.

[0050] The storage unit 320 stores a viewing application 321, which is an application for viewing distributed videos, and various information such as a user ID and a password.

[0051] The operation unit 330 accepts various operations.

[0052] The display unit 340 displays various information such as streaming video.

[0053] The communication unit 350 communicates with external devices such as the server 100. In particular, the communication unit 350 transmits information necessary for viewing a distributed video to the server 100, and receives the distributed video from the server 100.

[0054] [Server and device behavior when starting to watch a streaming video]

[0055] FIG. 3 is a flowchart showing the operations of the server 100 and the terminal 300 when starting to view a distributed moving image in one embodiment of the present invention.

[0056] 3, when a touch operation or the like is received on an icon of viewing application 321 displayed on display unit 340 (S1), control unit 310 of terminal 300 starts viewing application 321 and transmits the ID and password registered in storage unit 320 to server 100 (S3). As a result, terminal 300 requests server 100 to distribute a distribution video.

[0057] When the control unit 110 of the server 100 receives the ID and password (S5), it refers to the viewer information table (FIG. 4) to perform authentication processing for the user of the terminal 300 (S7). As a result of the authentication processing, the control unit 110 determines whether or not the authentication has been successful (S9).

[0058] When it is determined in step S9 that the authentication is successful (YES in S9), the control unit 110 sets the terminal 300 as the distribution destination and starts distribution of the distribution video to the terminal 300 (S11). The terminal 300 receives the distribution video from the server 100 (S13). The terminal 300 starts displaying the distribution video on the screen of the viewing application (S15), and ends the process.

[0059] In step S9, when it is determined that the authentication has failed (NO in S9), the control unit 110 does not set the terminal 300 as a delivery destination, and notifies the terminal 300 that the authentication has failed (S17). The terminal 300 displays the fact that the authentication has failed on the screen of the viewing application (S19), and ends the process.

[0060] FIG. 4 is a diagram showing a schematic viewer information table stored in storage unit 130 of server 100. As shown in FIG.

[0061] With reference to Figures 3 and 4, the viewer information table is a table in which information on users (viewers) who have the authority to view distributed videos is recorded. The viewer information table includes information such as the viewer's ID, password, grade, billing destination, and distribution status. The information included in the viewer information table can be changed, added, or deleted by an administrator of the video distribution system. There are two grades: Grade A and Grade B. Grade A users are charged a higher amount than Grade B users, and can view distributed videos in more ways than Grade B users. Here, credit card information is recorded as billing destination information. Information for electronic payment services (such as account and password) may also be recorded as billing destination information.

[0062] When performing authentication processing in step S7 of FIG. 3, the control unit 110 of the server 100 judges whether or not the ID and password received from the terminal 300 match the ID and password registered in the viewer information table. If the ID and password received from the terminal 300 match the ID and password registered in the viewer information table, the control unit 110 determines that the authentication has been successful. The delivery status column in the viewer information table is set to "delivery in progress." If the ID and password received from the terminal 300 do not match the ID and password registered in the viewer information table, the control unit 110 of the server 100 determines that the authentication has failed. If viewing is stopped, the delivery status column in the viewer information table is left blank.

[0063] Note that the control unit 110 may charge the billing destination of the user of the terminal 300 set as the distribution destination every time the terminal 300 is set as the distribution destination. The control unit 110 may charge the billing destination of the user recorded in the viewer information table every time a certain period of time has elapsed. Furthermore, the control unit 110 may not perform the authentication process as described above, and may distribute the distribution video without charging all the terminals 300 that have accessed the server 100.

[0064] [Server behavior when creating a distribution video]

[0065] Referring to FIG. 2, the outline of the flow of distributing a video to be distributed is as follows. Camera 200 encodes the video captured by image capture unit 220 and the audio data recorded by audio capture unit 250 to create a captured video. Camera 200 transmits the captured video to server 100. Upon receiving the captured video from camera 200, server 100 creates a video to be distributed in a predetermined distribution format based on the captured video. Server 100 temporarily stores the captured video and the video being processed in image memory 120. Server 100 creates a video to be distributed by processing the captured video data in packet units. Server 100 distributes the created video to destination terminal 300 by streaming. Here, the operation of server 100 when creating a video to be distributed based on packets of data of one captured video will be described.

[0066] FIG. 5 is a flowchart showing the operation of the server 100 when creating a moving image to be distributed in one embodiment of the present invention.

[0067] With reference to FIG. 5, control unit 110 of server 100 acquires a captured video from camera 200 (S101). Control unit 110 stores the acquired captured video in image memory 120 (S103). Next, control unit 110 acquires position information of train 400 from camera 200 using communication unit 140, and stores the acquired position information in image memory 120 (S105). Next, control unit 110 creates a distribution video based on the captured video using image processing unit 160, and stores the created distribution video in image memory 120 (S107). Details of the process of creating a distribution video will be described later. Control unit 110 distributes the created distribution video (the video stored in image memory 120) to destination terminal 300 using communication unit 160 (S109). The destination terminal 300 is a user terminal 300 whose distribution status column in the viewer information table is "distributing". The processes of steps S101 to S109 are repeated until the distribution is completed.

[0068] The distribution video created by the control unit 110 includes a forward distribution video and a guest room distribution video. The control unit 110 creates a forward distribution video based on the forward shot video created by the camera 200a, and creates a guest room distribution video based on the guest room shot video created by the camera 200b. The forward distribution video and the guest room distribution video are different from each other.

[0069] FIG. 6 is a diagram showing a schematic diagram of a forward-viewing moving image captured by camera 200a.

[0070] 1 and 6, the camera 200a mainly captures the external scenery seen from the driver's seat window WD1 and the scenery inside the driver's cab 401 including the instruments for driving, and creates a forward-captured video including these. The forward-captured video may also include the track, the streetscape, the driver DR, and the like. The forward-captured video may be created by the camera 200c provided on the exterior front of the body of the train 400, or may include only the scenery in front of the train 400.

[0071] There may be multiple cameras 200a and 200b. As an example, when the camera 200a is composed of four cameras, the first camera among the four cameras captures the upper right portion SR1 of the forward-shooting video to create the first captured video fragment. The second camera among the four cameras captures the lower right portion SR2 of the forward-shooting video to create the second captured video fragment. The third camera among the four cameras captures the lower left portion SR3 of the forward-shooting video to create the third captured video fragment. The fourth camera among the four cameras captures the upper left portion SR4 of the forward-shooting video to create the fourth captured video fragment. The forward-shooting video includes each of the first to fourth captured video fragments. When creating a distribution video based on the captured video (S107 in FIG. 5), the image processing unit 160 of the server 100 creates a forward distribution video by combining the first to fourth captured video fragments with each other.

[0072] The control unit 110 may create a distribution video by replacing the scenery inside the window in the video captured by a camera installed inside the train 400 with the scenery outside the train 400 captured by a camera installed outside the train 400. In this case, the process of creating a distribution video (S107) of FIG. 5 is performed on the video after replacement. As an example, when a part of the video captured by the camera 200a is replaced with a part of the video captured by the camera 200c, the forward distribution video includes the scenery inside the driver's cab 401 created by the camera 200a and the scenery outside seen from the driver's seat window WD1 created by the camera 200c. As a result, it is possible to delete unnecessary objects reflected in the window from the distribution video while conveying the atmosphere inside the train 400 to the viewer.

[0073] Fig. 7 is a diagram showing a first part of the video taken by the camera 200b. Fig. 8 is a diagram showing a second part of the video taken by the camera 200b. Fig. 7 is a part of the video taken by the passenger compartment in a direction from the front to the rear of the passenger compartment 402. Fig. 8 is a part of the video taken by the passenger compartment in a direction rotated about 180 degrees to the right from the direction of Fig. 7.

[0074] 1, 7, and 8, the camera 200b mainly captures the interior scenery of the passenger compartment 402 of the train 400 and the exterior scenery seen from the window WD2 of the passenger compartment 402 of the train 400, and creates a passenger compartment captured video including these. The passenger compartment captured video may also include hanging advertisements AD, the conductor CR, the passengers PG, seats ST, or the display ND of the car number.

[0075] Each of the cameras 200a and 200b may be an omnidirectional camera that captures video in all directions, up, down, left, and right. As an example, when the camera 200b is an omnidirectional camera, the video captured in the passenger compartment of a portion rotated about 90 degrees to the right from the portion in FIG. 7 mainly includes the outside scenery seen through the window WD2 as shown in FIG.

[0076] Fig. 9 shows a subroutine of the process (S107) for creating a video to be distributed of Fig. 5. The subroutine of Fig. 9 is performed for each of the forward-shot video created by camera 200a and the passenger compartment shot video created by camera 200b.

[0077] Referring to FIG. 9, in the process of creating a video to be distributed, the control unit 110 adds the location information of the train 400 obtained from the camera 200 to the videos (front-shot video and rear-shot video) stored in the image memory 120, and stores the videos after addition in the image memory 120 (S151).

[0078] Next, the control unit 110 recognizes what objects are included in the video stored in the image memory 120 (S153). Through the process of step S153, the control unit 110 recognizes objects such as a window WD1, a house, a car, a car license plate, a railroad track, a streetscape, instruments, and the figure of the driver DR in the forward-shooting video. The control unit 110 recognizes objects such as a window WD2, a hanging advertisement AD, the figure of the conductor CR, the figure of the passenger PG, a seat ST, a display ND of the car number, and a mountain in the rearward-shooting video. In addition, in the process of step S153, the control unit 110 may recognize unnecessary objects reflected in the window WD1 or WD2 in the video stored in the image memory 120 as noise, and perform a process of deleting the recognized noise from the video stored in the image memory 120.

[0079] Next, the control unit 110 determines whether or not sudden braking has been detected (S155). If it is determined in step S153 that sudden braking has been detected (YES in S155), the control unit 110 performs pixelization on the scenery outside the train 400 in the video stored in the image memory 120, and stores the pixelized video in the image memory 120 (S157). The control unit 110 then proceeds to the process of step S159.

[0080] FIG. 10 is a diagram that illustrates a moving image after mosaic is added to the forward-shot moving image (FIG. 6) in step S157 in FIG.

[0081] 10, mosaic processing is video processing that makes a target area invisible on a pixel-by-pixel basis. In the forward-shooting video, the inside of window WD1 corresponds to the exterior scenery of train 400. For this reason, in step S157 in FIG. 9, mosaic processing is performed on the area including the inside of window WD1. As a result of the mosaic processing, a mosaic MC1 is added to the area including the inside of window WD1.

[0082] The mosaic process may be performed on at least a portion of the captured video. Instead of an external landscape, a specific object such as a railroad or a person may be the target of the mosaic process. The entire captured video may be the target of the mosaic process.

[0083] When sudden braking of train 400 is detected, there is a possibility that train 400 has been involved in an accident. In such a case, the captured video may include a video of a train accident or confusion accompanying the accident, and there is a risk that an inappropriate video that should not be provided may be distributed as a distribution video. Therefore, when sudden braking of train 400 is detected, a mosaic process is performed on at least a portion of the captured video, thereby making it possible to avoid a situation in which an inappropriate distribution video is distributed.

[0084] Referring to FIG. 9, in step S155, when it is determined that sudden braking has not been detected (NO in S155), control unit 110 proceeds to the process of step S159.

[0085] In step S159, the control unit 110 refers to the scenery information table (FIG. 11) and determines whether scenery information corresponding to the acquired position information exists (S159). Scenery information is information related to the scenery outside the train 400 included in the captured video. In step S159, if it is determined that scenery information corresponding to the acquired position information exists (YES in S159), the control unit 110 adds the scenery information to the video stored in the image memory 120 based on the acquired position information, and stores the video after the addition in the image memory 120 (S161). Thereafter, the control unit 110 proceeds to the process of step S163.

[0086] FIG. 11 is a diagram showing a schematic view information table stored in the storage unit 130 of the server 100. As shown in FIG.

[0087] Referring to FIG. 11, the scenery information table is a table in which scenery information is recorded. In the scenery information table, location information of the object to which scenery information is added, the type of object to which scenery information is added, scenery information, and whether donation is possible are recorded in association with each other. The location information here is information of latitude and longitude. Whether donation is possible is information indicating whether donation to the object to which scenery information is added is possible. The information included in the scenery information table can be changed, added, and deleted by an administrator of the video distribution system.

[0088] FIG. 12 is a diagram that illustrates the video after scenery information is added to the video shot in the passenger compartment (FIG. 8) in step S161 of FIG.

[0089] 12, here, information IF1, which is scenery information "Mt. Iwaki, a volcano with an elevation of 1625m. It is also known as Tsugaru Fuji." is added to a mountain included in a video shot in a cabin having location information of "N40.39.322, E140.18.194." By adding information IF1, it is possible to provide the viewer with information about the outside scenery that can be seen from window WD2.

[0090] FIG. 13 is a diagram that illustrates a video after scenery information is added to the video shot in a passenger compartment (different from the videos in FIGS. 7 and 8) in step S161 in FIG.

[0091] 13, here, information IF2, which is scenery information "Hirosaki Station, you can buy souvenirs at the following URL: https: / / **** / ***** / " is added to a station included in a video shot in a cabin having location information of "N40.40.596, E140.18.478" (part of the URL is omitted). When the URL is touched while the distribution video shown in FIG. 13 is displayed, control unit 310 of terminal 300 starts a browser and displays the site indicated by the URL (a site selling items related to the area where the station is located) on the browser. By adding information IF2, the viewer can have a simulated experience of stopping by a station visible from window WD2 and buying souvenirs at the station.

[0092] The terminal 300 displaying the distributed video may always display the scenery information, or may display the scenery information when a predetermined operation (such as a touch operation on an object to which scenery information has been added) is received.

[0093] Referring to FIG. 9, in step S159, if it is determined that there is no scenery information corresponding to the acquired position information (NO in S159), control unit 110 advances the process to step S163.

[0094] In step S163, the control unit 110 refers to the mosaic information table (FIG. 14) and determines whether or not there is mosaic information corresponding to the acquired position information (S163). Mosaic information is information including position information of an object to be subjected to mosaic processing. In step S163, when it is determined that there is mosaic information corresponding to the acquired position information (YES in S163), the control unit 110 performs mosaic processing on the object included in the moving image stored in the image memory 120 based on the position information of the train and the position information of the object indicated by the mosaic information, and stores the moving image after mosaic processing in the image memory 120 (S165). Thereafter, the control unit 110 proceeds to the process of step S167.

[0095] In step S163, when it is determined that there is no mosaic information corresponding to the acquired position information (NO in S163), control unit 110 proceeds to the process of step S167.

[0096] In step S167, control unit 110 performs mosaic processing on a specific type of object among recognized objects in the moving image stored in image memory 120, and stores the moving image after the mosaic processing in image memory 120 (S167). After that, control unit 110 proceeds to the process of step S169.

[0097] The specific type of object is, for example, an object that may violate a person's privacy if the video is distributed. The specific type of object may include a license plate of a car or a nameplate on a house.

[0098] FIG. 14 is a diagram showing a schematic view of a mosaic information table stored in the storage unit 130 of the server 100. As shown in FIG.

[0099] Referring to Fig. 14, the mosaic information table is a table in which mosaic information is recorded. The mosaic information includes location information of the object to be subjected to the mosaic processing, the type of the object to be subjected to the mosaic processing, the size of the mosaic, and the like. The object to be subjected to the mosaic processing is, for example, an object requested by its owner to be mosaicked, or a facility where filming is prohibited. The mosaic information included in the mosaic information table can be changed, added, or deleted by an administrator of the video distribution system.

[0100] FIG. 15 is a diagram that illustrates a moving image after mosaic is added to the forward-shot moving image (FIG. 6) in steps S165 and S167 in FIG.

[0101] 14 and 15, in step S165, a mosaic process is performed on a house having location information of "N40.39.451, E140.18.102". As a result of the mosaic process, a mosaic MC2 is added to the house. In addition, in step S167, a mosaic process is performed on the license plate of a car recognized as a specific type of object. As a result of the mosaic process, a mosaic MC3 is added to the license plate. By adding the mosaic MC2 or MC3, it is possible to make objects that should not be displayed invisible, and to prevent a situation in which an inappropriate video that should not be provided is distributed as a distribution video.

[0102] 9, in step S169, control unit 110 adds the interior information to the video stored in image memory 120, and stores the video after the addition in image memory 120 (S169). After that, control unit 110 proceeds to the process of step S171.

[0103] The interior information is information that describes objects inside the train 400 that are included in the captured video. The storage unit 130 stores the interior information. The control unit 110 adds the interior information to objects inside the train 400 that are included in the video stored in the image memory 120, based on the recognition result by the recognition unit 150.

[0104] Fig. 16 is a diagram that illustrates a moving image in which vehicle interior information is added to the forward-shooting moving image (Fig. 6) in step S169 in Fig. 9. Note that Fig. 16 shows an enlarged portion of the moving image.

[0105] 16, in the forward-shooting video, information IF3, IF4, and IF5 are added to driving instruments such as a speedometer, brake, and accelerator. Information IF3, IF4, and IF5 are in-vehicle information in the form of text, such as "speedometer," "brake," and "accelerator," respectively. Terminal 300 displaying the forward-delivered video may always display the in-vehicle information in the form of text, or may display it when a predetermined operation (such as touching the object) is received.

[0106] FIG. 17 is a diagram showing in-car information added to the hanging advertisement AD of the video shot in the passenger compartment of FIG.

[0107] With reference to FIG. 17, an image IM is added to the hanging advertisement AD (FIG. 7) of the video shot in the guest room.

[0108] The image IM is in-car information corresponding to the image of the hanging advertisement AD. The terminal 300 displaying the cabin delivery video may always display the image IM superimposed on the hanging advertisement AD. In addition, when the terminal 300 displaying the cabin delivery video receives a predetermined operation (such as a touch operation on the hanging advertisement AD), the terminal 300 may display the image IM superimposed on the hanging advertisement AD or may switch the display from the cabin delivery video to the image IM.

[0109] By adding interior information to objects inside train 400, it is possible to provide viewers with information about objects inside train 400 that cannot be understood from the distributed video.

[0110] 9, at step S171, control unit 110 determines whether or not a human face is included in the moving image stored in image memory 120, based on the recognition result by recognition unit 150 (S171).

[0111] In step S171, when it is determined that the moving image stored in image memory 120 contains a human face (YES in S171), control unit 110 refers to the face table (FIG. 18) and performs mosaic processing on the human faces recognized from the moving image stored in image memory 120 that differ from the faces of the people shown in the images recorded in the face table. Control unit 110 stores the moving image after the mosaic processing in image memory 120 (S173). Thereafter, control unit 110 returns.

[0112] In step S171, when it is determined that the moving image stored in image memory 120 does not contain a human face (NO in S171), control unit 110 returns without performing pixelization on the face.

[0113] FIG. 18 is a diagram showing a schematic diagram of a face table stored in the storage unit 130 of the server 100. As shown in FIG.

[0114] 18, the face table is a table in which images of the faces of people to be excluded from the mosaic processing are recorded. People to be excluded from the mosaic processing include the conductor and driver of train 400, station staff, and on-board sales staff circulating inside train 400. The face table records a registration ID, information (file name) that identifies the face image of a person to be excluded from the mosaic processing, and the occupation of that person, in association with each other. The information included in the face table can be changed, added, and deleted by an administrator of the video distribution system.

[0115] FIG. 19 is a diagram that illustrates the video after mosaic is added to the video shot in the passenger compartment (FIG. 7) in step S175 of FIG.

[0116] Referring to Fig. 19, the video captured in the passenger compartment includes the face of the conductor CR and the face of the passenger PG. Of these faces, the face of the conductor CR is the same as the face shown in the image recorded in the face table. Therefore, in step S175 of Fig. 9, mosaic processing is not performed on the face of the conductor CR, but is performed on the face of the passenger PG. As a result of the mosaic processing, a mosaic MC4 is added only to the face of the passenger PG. By adding the mosaic MC4, it is possible to protect the privacy of the passenger PG while minimizing the area to which the mosaic is added.

[0117] Note that the process for creating a video to be distributed shown in FIG. 9 only needs to include at least the mosaic process (S157 in FIG. 9) that is performed when sudden braking is detected, and other processes are optional.

[0118] [Device behavior when viewing streaming video]

[0119] Operation unit 330 of terminal 300 accepts operations from a user (viewer) of terminal 300. When control unit 310 of terminal 300 starts receiving the distribution video distributed from server 100, it displays the distribution video received from server 100 on display unit 340 in a manner according to the operation accepted by operation unit 330. An example of the relationship between the accepted operation and the display manner of the distribution video is described below. Note that the relationship between the accepted operation and the display manner of the distribution video is arbitrary.

[0120] FIG. 20 is a diagram showing a schematic diagram of changes in the distributed moving image displayed on display unit 340 of terminal 300. As shown in FIG.

[0121] 20, in the default state, control unit 310 displays video MV1 on display unit 340 (FIG. 20(a)). Video MV1 is a video equivalent to the guest room distribution video shown in FIG. 19, and is accompanied by a mosaic MC4.

[0122] When a flick operation is received while video MV1 is being displayed (FIG. 20(a)), control unit 310 moves the location of the room-delivered video being displayed in the direction of the flick operation. As an example, when a rightward flick operation is received while video MV1 is being displayed (FIG. 20(a)), control unit 310 moves the location of the room-delivered video being displayed from video MV1 to video MV2 (FIG. 20(b)). Video MV2 is a video equivalent to the room-delivered video shown in FIG. 13, and is a video of a location rotated 90 degrees to the right from the location of video MV1.

[0123] When a pinch-in operation is received while video MV1 is being displayed (Fig. 20(a)), control unit 310 reduces and displays video MV1 with the midpoint of the pinch-in operation as the center position. When a pinch-out operation is received while video MV1 is being displayed (Fig. 20(a)), control unit 310 enlarges and displays video MV1 with the midpoint of the pinch-out operation as the center position. As an example, when a pinch-out operation is received with display ND as the midpoint while video MV1 is being displayed (Fig. 20(a)), control unit 310 displays video MV3 (Fig. 20(c)). Video MV3 corresponds to a video that is an enlargement of the part of display ND in video MV1.

[0124] When a touch operation on the hanging advertisement AD is received while the video MV1 is being displayed (FIG. 20(a)), the control unit 310 displays an image IM (FIG. 20(d)). The image IM is the image of the hanging advertisement AD shown in FIG. 17. When a touch operation on the image IM is received, the control unit 310 displays the video MV1 again.

[0125] When a double click operation is received on the display unit 340 while video MV1 is being displayed (FIG. 20(a)), the control unit 310 switches the displayed video from the cabin broadcast video to the front broadcast video, and displays video MV4 (FIG. 20(e)). Video MV4 is a video equivalent to the front broadcast video shown in FIG. 15, and has mosaics MC2 and MC3 added.

[0126] If train 400 applies sudden braking while video MV4 is being displayed (Fig. 20(e)), the distributed video received from server 100 will have at least a portion of it blurred. As a result, control unit 310 displays video MV5 (Fig. 20(f)). Video MV5 is a video equivalent to the forward distributed video shown in Fig. 10, and has mosaic MC1 added to it.

[0127] When a touch operation is received on an instrument included in the forward distribution video while video MV4 is being displayed (FIG. 20(e)), the control unit 310 displays video MV6 (FIG. 20(g)). Video MV6 is a video equivalent to the forward distribution video shown in FIG. 16, and is accompanied by information IF3, IF4, and IF5.

[0128] The control unit 310 may display a menu button 341 together with the distributed video. When the control unit 310 receives a touch on the menu button 341, the control unit 310 may display a menu screen including items such as rotation, switching between the forward distributed video and the cabin distributed video, enlarging the video, reducing the video, displaying scenic information and in-car information, displaying an image of a hanging advertisement, adjusting the volume, or displaying the current location on a map. The control unit 310 may display the distributed video in a mode according to the operation received on the menu screen. It is preferable that the menu screen is semi-transparent so as not to interfere with viewing of the distributed video.

[0129] When distributing the distribution video to the terminal 300, the control unit 110 may set a viewing restriction on the distribution video for the terminal 300 of a B-grade user. As an example, both the forward distribution video and the guest room distribution video may be distributed to the terminal 300 of the A-grade user, and the A-grade user may be permitted to view both the forward distribution video and the guest room distribution video. The control unit 110 may distribute only the forward distribution video to the terminal 300 of the B-grade user, and permit the B-grade user to view only the forward distribution video. The control unit 310 may also distribute the omnidirectional guest room distribution video to the terminal 300 of the A-grade user, and permit the A-grade user to view the omnidirectional guest room distribution video. The control unit 310 may distribute only one direction of the guest room distribution video to the terminal 300 of the B-grade user, and permit the B-grade user to view only one direction of the guest room distribution video.

[0130] [Effects of the embodiment]

[0131] According to the above-described embodiment, when sudden braking of a train is detected, a mosaic process is performed on at least a part of the captured video, thereby making it possible to make inappropriate parts of the captured video invisible. As a result, it is possible to prevent inappropriate videos that should not be provided from being distributed as streaming videos, and it is possible to distribute the scenery seen by train passengers in an appropriate state to the viewer's terminal in real time.

[0132] In addition, viewers can view the scenery seen by train passengers in real time under appropriate conditions, allowing them to virtually experience riding a train without having to travel to an area where the train runs, and to experience the season and weather in the area around the train at the time of viewing in real time.

[0133] Furthermore, train operators can earn revenue from train fares as well as from charging viewers of streamed videos. Local lines, in particular, have low fare revenues and their survival is in question. However, revenue from charging viewers of streamed videos can improve profits and help local lines survive.

[0134] [others]

[0135] The control unit 110 may accept donations from viewers of the terminals 300. This type of donation is also called "tipping (gifting)." When the control unit 110 accepts a donation from a viewer of the terminals 300, it charges the billing destination of the viewer registered in the viewer information table (FIG. 4). The money collected through this charging is donated to the recipient of the donation at a later date.

[0136] For example, when a predetermined operation (e.g., a long press on the scenery information displayed on display unit 340 of terminal 300) is received on terminal 300 while a streaming video containing an object that is identified as eligible for donation in the scenery information table (Figure 11) is being streamed, control unit 110 may accept a donation to an entity related to that object (such as an environmental protection group for Mt. Iwaki in the case of Figure 12).

[0137] In addition, when the control unit 110 receives a predetermined operation on the terminal 300 while streaming a video that includes crew members of the train 400, such as the driver DR (Figure 6) or the conductor CR (Figure 7), the control unit 110 may accept a donation to the operator of the train 400.

[0138] Furthermore, when a predetermined operation is received at the terminal 300 while a streaming video including an image IM (Figure 17) of the hanging advertisement AD is being distributed, the control unit 110 may receive a donation to the advertiser of the hanging advertisement AD (in the case of Figure 17, "Sankan Foods").

[0139] When an object that is available for donation is included in the forward-shot video or the video shot in the cabin, the control unit 110 may add an indication (such as a "$" sign) indicating that donation is available to the part of the object in the forward-shot video or the video shot in the cabin.

[0140] The timing of accepting donations is arbitrary. When the control unit 110 accepts a predetermined operation on the terminal 300 at an arbitrary timing, the control unit 110 may display options of donation recipients (such as the operator of the train 400, or companies or organizations along the line of the train 400) on the display unit 340 of the terminal 300 and accept the selection of the donation recipient.

[0141] It is assumed that some viewers of the distributed video have a special attachment to Train 400 and its line and would like to financially support them. By accepting donations, the wishes of such viewers can be met. The operator of Train 400, or companies or organizations along the line of Train 400, can receive financial support from such viewers.

[0142] In the above-described embodiment, the captured video and the distributed video include audio, but the captured video and the distributed video do not necessarily have to include audio.

[0143] The mosaic processing in the above-mentioned embodiment may be any processing that makes the target portion invisible. The processing that makes the target portion invisible may be, in addition to mosaic processing, a mask processing that does not display the target portion and displays only the portion other than the target portion, a blur processing that blurs the target portion, or an image addition processing that overwrites a specific image on the target portion. Furthermore, the processing that makes the target portion invisible may be a processing that stops the distribution video, or a processing that replaces the distribution video with an image that includes a caption such as "Image transmission is currently stopped". Mosaic processing is a processing that makes the target portion invisible while conveying part of the atmosphere of the target portion, and is therefore more suitable than other processing as a processing that makes the target portion invisible.

[0144] The terminal 300 may be a device (such as a VR headset) that displays three-dimensional video, and the server 100 may distribute three-dimensional images to the terminal 300.

[0145] The above-described embodiments and modifications can be combined as appropriate.

[0146] The above-described embodiments and modifications should be considered as illustrative and not restrictive in all respects. The scope of the present invention is defined by the claims, not the above description, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0147] 100 Server (Example of a video distribution server) 110, 210, 310 Control unit 120 Image Memory 130, 320 Memory unit (an example of an object memory means, an in-vehicle information memory means, and a face memory means) 131 Server Programs 140, 240, 350 Communication unit (an example of acquisition means and distribution means) 150 Recognition part 160 Image processing unit (an example of a creating means) 200, 200a, 200b, 200c Camera (example of camera) 220 Photography Department 230 Location information acquisition unit (an example of location information acquisition means) 250 Recording section 260 Sudden braking detection unit (an example of a sudden braking detection means) 300 terminals (example of terminals) 321 Viewing App 330 Operation unit (an example of an operation reception means) 340 Display unit (an example of a display means) 341 Menu button 400 Train (an example of a train) 401 Cab 402 Rooms 403 Rooms IF1, IF2, IF3, IF4, IF5 Information MC1,MC2,MC3,MC4 Mosaic MV1,MV2,MV3,MV4,MV5,MV6 Videos SR1, SR2, SR3, SR4 Front-view video locations ST seat WD1,WD2 Windows

Claims

1. A video distribution server that distributes a video of a scene seen by a train passenger to a viewer's terminal in real time, An acquisition means for acquiring a captured video, which is a video of a scene seen by passengers of the train, captured by a camera mounted on the train; A creating means for creating the distribution video based on the captured video; A distribution means for distributing the distribution video to the terminal; a sudden braking detection means for detecting sudden braking of the train, The creation means is a video distribution server including a brake processing means that performs a process to make at least a portion of the captured video invisible when the sudden braking detection means detects sudden braking.

2. The video distribution server according to claim 1 , wherein the captured video includes a video of a scene outside the train.

3. A position information acquisition means for acquiring position information of the train is further provided. The video distribution server according to claim 2 , wherein the creation means further includes an external information adding means for adding information related to the scenery outside the train contained in the filmed video based on the position information of the train to the filmed video.

4. Further comprising an object storage means for storing position information of a specific object, 4. The video distribution server according to claim 3, wherein the creation means further includes an object processing means for performing processing on the specific object included in the captured video to make it invisible based on the position information of the train and the position information of the specific object.

5. 5. The video distribution server according to claim 1, wherein the captured video includes a video of the interior of the train.

6. The train information storage means further includes an interior information storage means for storing interior information that describes objects inside the train and is included in the captured video. The video distribution server according to claim 5 , wherein the creating means further includes information adding means for adding the vehicle interior information to the captured video.

7. A face storage means for storing an image of a person's face; A face recognition unit is further provided for recognizing a face of a person included in the captured video. The video distribution server according to claim 6, wherein the creation means further includes face processing means for applying processing to make invisible faces of people recognized by the face recognition means that differ from the faces of people represented by the images stored in the face memory means.

8. The captured video includes a plurality of captured video segments each of which is a video captured by each of the plurality of cameras; 8. The video distribution server according to claim 1, wherein the creation means creates the video to be distributed by combining the plurality of shot video segments together.

9. A video distribution system comprising the video distribution server according to any one of claims 1 to 8, the camera, and the terminal.

10. The terminal includes: An operation receiving means for receiving an operation; 10. The moving image distribution system according to claim 9, further comprising: a display means for displaying the moving image to be distributed in a manner according to the operation accepted by the operation accepting means.

Images