Distribution apparatus, distribution method, and computer program product

Abstract

The present application provides a distribution device, a distribution method, and a computer program product, which reduce dizziness of a user who views a distribution image. A distribution server (200) has a second control section (230) that acquires a user ID and device information included in a distribution request, receives movement information indicating movement of a head of a user wearing a first display device (400A) from the first display device (400A) in a case where a display device indicated by the device information is the first display device (400A), decides a range of a captured image to be distributed as a distribution image based on the received movement information, and distributes the captured image of the decided range as the distribution image to the first display device (400A). Even if the movement information includes movement of the head of the user in a vertical direction, the second control section (230) does not change the range of the captured image to be distributed as the distribution image in the vertical direction in correspondence with the movement of the head of the user in the vertical direction.

Description

Technical Field

[0001] This invention relates to a distribution device, a distribution method, and a computer program product. Background Technology

[0002] In recent years, initiatives to provide sustainable transportation systems that take into account vulnerable groups among transportation participants, such as the elderly, people with disabilities, and children, have become increasingly active. To realize these initiatives, research and development efforts are focused on further improving the safety and convenience of transportation through the development of mobility options for vulnerable groups.

[0003] For example, when vulnerable individuals use transportation systems, they consider not only their own journey but also the methods of viewing animated images distributed by video distributors. By watching these animated images, they can experience the feeling of going out or traveling.

[0004] For example, Patent Document 1 discloses an image processing device that processes images of objects in the real world that the observer is observing, captured by a camera worn on the observer's head along with a head-mounted display and displayed on the head-mounted display.

[0005] [Existing Technical Documents]

[0006] [Patent Literature]

[0007] Patent Document 1: Japanese Patent Application Publication No. 2012-160898 Summary of the Invention

[0008] [The problem the invention aims to solve]

[0009] However, when images displayed on a head-mounted display are transferred to other display devices so that the images can also be viewed on those devices, there is a problem that users viewing the images on other display devices sometimes experience dizziness due to the shaking of the images.

[0010] This invention was made in view of the above circumstances, and its purpose is to reduce dizziness in users viewing distributed images. Furthermore, by connecting remote locations and vehicles, it helps to support good connectivity between urban areas and local regions, represented by urban peripheries, and contributes to the development of sustainable transportation systems.

[0011] [Methods used to solve problems]

[0012] The distribution apparatus disclosed herein includes: a communication unit that receives a distribution request; a storage unit that stores user identification information identifying a user in correspondence with device information identifying a display device used by the user, and stores a captured image for distribution received from a distributor terminal; and a control unit that acquires the user identification information and device information included in the distribution request, receives from the display device movement information indicating movement of the display device as indicated by the acquired device information, determines the distribution range of the captured image as a distribution image based on the received movement information, and distributes the captured image within the determined range as the distribution image to the display device. If the movement information includes vertical movement, the control unit makes the distribution range of the captured image as the distribution image smaller than the vertical movement range indicated by the movement information, or does not change the range of the captured image in the vertical direction.

[0013] [Invention Effects]

[0014] According to this disclosure, even if the structure that determines the range of the image to be captured in accordance with the movement of the user's head does not change the range of the image to be captured in accordance with the vertical movement of the user's head, it is possible to suppress dizziness in users who view the distributed images. Attached Figure Description

[0015] Figure 1 This is a diagram showing the system architecture of the distribution system.

[0016] Figure 2 A block diagram illustrating the structure of the vehicle-mounted device and the distribution server.

[0017] Figure 3 This is a diagram showing the structure of the user registration database.

[0018] Figure 4 This is a diagram showing the structure of the distribution management database.

[0019] Figure 5 This is a block diagram of the distributor terminal.

[0020] Figure 6 This is a block diagram showing the structure of the first display device.

[0021] Figure 7 This is a block diagram showing the structure of the second display device.

[0022] Figure 8 This is a flowchart illustrating the actions of the distribution server.

[0023] Figure 9 This is a flowchart illustrating the actions of the distribution server. Detailed Implementation

[0024] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0025] [Implementation Method]

[0026] [1. Structure of the distribution system]

[0027] Figure 1 This is a diagram showing the system architecture of distribution system 1.

[0028] The distribution system 1 includes a vehicle-mounted device 30, a distributor terminal 100 used by the distributor S, a distribution server 200, and a display device 400 used by the viewer U. It is a system for live streaming content containing images captured by the distributor terminal 100.

[0029] Live streaming refers to the real-time distribution of images captured by the distributor's terminal 100 to the display device 400, with the participation of both the distributor and an unspecified audience. The images can be still images or moving images.

[0030] The vehicle-mounted device 30 is mounted on the vehicle 3 and is wirelessly connected to the distributor terminal 100 held by the distributor S, who is an occupant of the vehicle 3, via near-field wireless communication or the like.

[0031] The vehicle-mounted device 30 receives content data from the distributor terminal 100, which includes images captured by the camera of the distributor terminal 100 and sound input from the microphone of the distributor terminal 100. The vehicle-mounted device 30 uploads the received content data to the distribution server 200.

[0032] The distributor terminal 100 can be a smartphone or tablet PC (Personal Computer), or a digital camera or a camera capable of capturing a 360-degree view of the surroundings. In this case, the microphone and camera can also be configured separately.

[0033] The distribution server 200 stores the content received from the vehicle-mounted device 30 in its server storage unit. The distribution server 200 distributes the uploaded content data to the display device 400 of the viewer U. Live streaming technologies can be implemented using well-known technologies such as HLS (HTTP Live Streaming), CMAF (Common Media Application Format), WebRTC (Web Real-Time Communications), and RTMP (Real-Time Messaging Protocol).

[0034] Display device 400 is a terminal device held by a viewer U who is watching content data distributed from distribution server 200. Figure 1 In the diagram, N audience members U are represented by U1, U2, ..., UN. N is an arbitrary natural number. Furthermore, in... Figure 1 In the middle, as a display device 400, a first head-mounted display device 400A worn on the head by viewers U1 and U2 and a second display device 400B held by viewer UN are shown.

[0035] Display device 400 includes a first display device 400A and a second display device 400B. The first display device 400A is a head-mounted display device that displays images and is worn on the head of the viewer U. The first display device 400A can be either transmissive or non-transmissive, as long as it can display content distributed from the distribution server 200.

[0036] The second display device 400B is a non-head-mounted display device whose image display unit is not worn on the head of the viewer U. The second display device 400B can be, for example, a portable or mobile terminal device such as a smartphone, tablet PC, or laptop PC, or a fixed terminal device such as a desktop PC.

[0037] Communication network 5 is a wired or wireless transmission path for information sent from devices connected to communication network 5. For example, communication network 5 may also include public networks such as the Internet, mobile communication networks, and satellite communication networks, as well as various LANs (Local Area Networks) and WANs (Wide Area Networks) including Ethernet. Additionally, communication network 5 may also include private networks such as IP-VPN (Internet Protocol-Virtual Private Network).

[0038] [2. Structure of the vehicle-mounted device]

[0039] Figure 2 A diagram illustrating the structure of the vehicle-mounted device 30 and the distribution server 200.

[0040] First, refer to Figure 2 The structure of the vehicle-mounted device 30 will be described.

[0041] The vehicle-mounted device 30 includes an external communication interface 31, an in-vehicle short-range wireless communication interface 32, an in-vehicle communication interface 33, a touch panel 34, a sound processing unit 35, a speaker 36, and an in-vehicle ECU (Electronic Control Unit) 300.

[0042] The external communication interface 31 is a communication module that performs wireless communication according to mobile communication standards such as LTE (Long Term Evolution), 4G (Generation), 5G, and Wi-Fi. The external communication interface 31 is connected to the communication network 5 via an antenna (not shown) and exchanges data with the distribution server 200.

[0043] The vehicle-mounted near-field wireless communication interface 32 is, for example, a communication module that communicates according to NFC (Nearfield communication) communication standards such as Bluetooth (registered trademark) and Wi-Fi. The vehicle-mounted near-field wireless communication interface 32 communicates wirelessly with a terminal device held by the occupant, and receives images and sounds captured by the terminal device.

[0044] The in-vehicle communication interface 33 is an input / output interface connected to an in-vehicle network (not shown). The in-vehicle network may use a communication bus corresponding to communication standards such as CAN (Controller Area Network), CAN-FD (CAN with Flexible Data Rate), or Ethernet. The in-vehicle communication interface 33 communicates with vehicle ECU 40, other relay devices, and other in-vehicle equipment via the in-vehicle network.

[0045] The touch panel 34 includes a display panel such as a liquid crystal panel or an organic EL (Electro-Luminescence) panel, and a touch sensor. The touch sensor detects touch operations performed by the distributor S on the touch panel 34 and outputs position information indicating the location of the detected touch operation to the first control unit 330.

[0046] The sound processing unit 35, for example, converts the digital sound data output by the first control unit 330 into an analog sound signal. The sound processing unit 35 amplifies the converted analog sound signal and outputs the amplified sound signal through the speaker 36.

[0047] The vehicle-mounted ECU 300 has an input / output interface 310 and a first control unit 330.

[0048] The input / output interface 310 functions as a connection between the external communication interface 31, the in-vehicle short-range wireless communication interface 32, the in-vehicle communication interface 33, the touch panel 34, the sound processing unit 35, and the first control unit 330. The input / output interface 310 includes connectors and interface circuitry, and outputs data input from the external communication interface 31, the in-vehicle short-range wireless communication interface 32, the in-vehicle communication interface 33, the touch panel 34, and the sound processing unit 35 to the first control unit 330. Furthermore, the input / output interface 310 outputs data input from the first control unit 330 to the external communication interface 31, the in-vehicle short-range wireless communication interface 32, the in-vehicle communication interface 33, the touch panel 34, or the sound processing unit 35.

[0049] The first control unit 330 is a computer device having a first memory 340 and a first processor 350.

[0050] The first memory 340 may include, for example, volatile semiconductor memory and non-volatile semiconductor memory. The first memory 340 may also include only non-volatile semiconductor memory. Alternatively, the first memory 340 may have an auxiliary storage device such as an SSD (Solid State Drive). The first memory 340 stores control programs 345 executed by the first processor 350, setting data specifying the operation of the vehicle-mounted device 30, etc.

[0051] The first processor 350 is, for example, a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit) or other computing device. The first processor 350 may consist of a single processor or multiple processors. Alternatively, the first processor 350 may also consist of a System-on-a-Chip (SoC) that integrates part or all of the first memory 340 with other circuitry.

[0052] The vehicle-mounted ECU 300 receives captured images and sounds from the distributor terminal 100 and uploads the received images and sounds to the distribution server 200.

[0053] [3. Structure of the distribution server]

[0054] Next, the structure of the distribution server 200 will be explained.

[0055] The distribution server 200 includes a network communication interface 210 and a second control unit 230.

[0056] The network communication interface 210 is equipped with a communication card such as a NIC (Network Interface Card) and communicates with the vehicle-mounted device 30, the distributor terminal 100, and the display device 400 via the communication network 5.

[0057] The second control unit 230 is a computer device equipped with a second memory 240 and a second processor 250.

[0058] The second memory 240 may include, for example, volatile semiconductor memory and non-volatile semiconductor memory. The second memory 240 may also be a structure that only includes non-volatile semiconductor memory. Alternatively, the second memory 240 may be a structure that includes auxiliary storage devices such as HDD (Hard Disk Drive) or SSD.

[0059] The second memory 240 stores the control program 241, the user registration database 243, the distribution management database 245, and the content database 247 executed by the second processor 250.

[0060] Figure 3 This is a diagram showing the structure of the user registration database 243.

[0061] A record in the user registration database 243 includes the user ID, the user's personal information, the APP_ID, and device information. APP is an abbreviation for application.

[0062] The user ID is identification information used to identify users who use the image distribution service provided by the distribution server 200. Users include distributors S and viewers U.

[0063] Personal information refers to the personal information of the distributor S and the viewer U as users, such as name, address, gender, age, etc.

[0064] APP_ID is identification information that identifies an application that has been downloaded to the distributor terminal 100 or display device 400 and installed.

[0065] Distributor S and viewer U download the APP from the designated website and install it on distributor terminal 100 and display device 400 respectively.

[0066] The device information indicates whether the display device 400 with the APP_ID installed is the first display device 400A or the second display device 400B. In other words, the device information indicates whether it is a head-mounted display device or a smartphone, tablet PC, or laptop PC other than a head-mounted display device. The device information also includes communication information with the display device 400, such as its IP address.

[0067] Figure 4 This is a diagram showing the structure of the distribution management database 245.

[0068] A record in the distribution management database 245 contains the distribution ID that identifies the distribution, the user ID of the distributor S, the user ID of the audience U, and the distribution start date and time.

[0069] return Figure 2 Next, we will explain the structure of the distribution server 200.

[0070] The second processor 250 is, for example, a processing unit such as a CPU or an MPU. The second processor 250 can be composed of a single processor or multiple processors. Alternatively, the second processor 250 can also be composed of a System-on-a-Chip (SoC) that integrates part or all of the second memory 240 with other circuitry.

[0071] The second control unit 230 registers the content data, including the distributed image and sound, uploaded from the distributor terminal 100 into the content database 247. Additionally, if the second control unit 230 receives a distribution request, it reads the content data specified by the received distribution request from the content database 247 and distributes the read content data to the display device 400, which is the source of the distribution request.

[0072] [4. Structure of the Distributor Terminal]

[0073] Figure 5 The structural block diagram of the distributor terminal 100.

[0074] The distributor terminal 100 includes a wireless communication interface 110, a near-field wireless communication interface 120, a camera 130, a touch panel 140, a microphone 151, a speaker 153, and a third control unit 170.

[0075] The wireless communication interface 110 is a communication module that performs wireless communication according to mobile communication standards such as LTE, 4G, 5G, and Wi-Fi. The wireless communication interface 110 is connected to the communication network 5 via an antenna (not shown) and exchanges data with the distribution server 200.

[0076] The near-field wireless communication interface 120 is, for example, a communication module that communicates according to NFC communication standards such as Bluetooth and Wi-Fi. The near-field wireless communication interface 120 transmits the images captured by the camera 130 and the sound input by the microphone 151 to the vehicle device 30 through near-field wireless communication with the vehicle device 30.

[0077] The camera 130 is a shooting device equipped with an image sensor and a data processing circuit. The image sensor is composed of, for example, a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal-Oxide-Semiconductor). The data processing circuit generates a captured image based on the output of the image sensor. The captured image generated by the camera 130 is sent to the vehicle-mounted device 30 under the control of the third control unit 170.

[0078] The touch panel 140 includes a display panel such as an LCD panel or an OLED panel, and a touch sensor. The touch panel 140 functions as a display unit for displaying images and a receiving unit for receiving operations from the distributor S.

[0079] Microphone 151 is a sound input device for inputting the sound of the transmitter S, and speaker 153 is a sound output device for outputting the sound of the audience U, etc.

[0080] The third control unit 170 is a computer device equipped with a third memory 180 and a third processor 190.

[0081] The third memory 180 is a non-volatile semiconductor memory. Alternatively, the third memory 180 may also have an auxiliary storage device such as an SD card. The third memory 180 stores control programs such as the OS (operating system) 181 and APP 183. APP 183 is an application that communicates data with the distribution server 200, displays dynamic images contained in the content distributed by the distribution server 200 on the touch panel 140, and outputs the sound contained in the content to the speaker 153.

[0082] The third processor 190 is, for example, a CPU, MPU, or other computing device. The third processor 190 can consist of a single processor or multiple processors.

[0083] [5. Structure of the display device]

[0084] Figure 6 This is a block diagram showing the structure of the first display device 400A.

[0085] The first display device 400A includes a near-field wireless communication interface 401A, an image display unit 403A, a sensor unit 405A, and a fourth control unit 410A.

[0086] The near-field wireless communication interface 401A is a communication module that communicates according to NFC communication standards such as Bluetooth and Wi-Fi. The near-field wireless communication interface 401A can wirelessly connect to the communication network 5 via a Wi-Fi router (not shown) installed in the home of the viewer U.

[0087] The image display unit 403A includes a display panel such as a liquid crystal panel or an organic EL panel (not shown), and a light guide plate that directs the image light emitted from the display panel toward the eyes of the viewer U, and is worn on the head of the viewer U.

[0088] Sensor unit 405A, for example, is mounted on image display unit 403A to detect the movement of the viewer U's head. Sensor unit 405A is, for example, a motion sensor including a three-axis accelerometer and a three-axis gyroscope. Alternatively, sensor unit 405A may also be a structure equipped with a three-axis geomagnetic sensor. In this case, sensor unit 405A may also be an IMU (Inertial Measurement Unit) modularly composed of a three-axis accelerometer, a three-axis gyroscope, and a three-axis geomagnetic sensor.

[0089] Microphone 407A collects the sound emitted by the audience U, and speaker 409A outputs the sound.

[0090] The fourth control unit 410A is a computer device equipped with a fourth memory 420A and a fourth processor 430A.

[0091] The fourth memory 420A is a non-volatile semiconductor memory. Alternatively, the fourth memory 420A can also be a structure combining volatile and non-volatile semiconductor memory. The fourth memory 420A stores OS 421A and APP 423A. APP 423A is an application that: communicates data with the distribution server 200, displays dynamic images contained in the content distributed by the distribution server 200 on the image display unit 403A, and outputs the sound contained in the content to the speaker 409A.

[0092] The fourth processor 430A is, for example, a CPU, MPU, or other arithmetic processing device. The fourth processor 430A can consist of a single processor or multiple processors.

[0093] Viewer U launches APP 423A, for example, by accessing a webpage provided by distribution server 200, and selects the channel and program they wish to watch from the programs provided by distribution server 200. When viewer U selects a channel and program, the fourth control unit 410A sends a distribution request containing user ID, APP_ID, channel information, and program information to distribution server 200.

[0094] Figure 7 This is a block diagram showing the structure of the second display device 400B.

[0095] like Figure 7 As shown, the second display device 400B includes a wireless communication interface 451B, a near-field wireless communication interface 452B, a camera 453B, a touch panel 454B, a microphone 455B, a speaker 456B, and a fifth control unit 460B. The structure of the second display device 400B is similar to... Figure 5 The structure of the distributor terminal 100 shown is the same, so the description is omitted.

[0096] [6. Actions of the distribution server]

[0097] Next, the operation of the distribution server 200 will be explained.

[0098] When the second control unit 230 of the distribution server 200 receives a distribution request from the display device 400, it obtains the user ID, the channel information requested for distribution, and the APP_ID contained in the received distribution request. The second control unit 230 then obtains the device information of the display device with the obtained APP_ID installed from the user registration database 243.

[0099] When the acquired device information indicates that the type of display device is a first display device 400A, the second control unit 230 begins distributing the first display mode. The display modes include a first display mode corresponding to the first display device 400A and a second display mode corresponding to the second display device 400B.

[0100] The first display mode is a mode in which the range of the captured image distributed as a distributed image changes in response to the movement of the head of the viewer U wearing the first display device 400A.

[0101] When the display mode is the first display mode, the second control unit 230 determines the range of the captured image to be distributed as the distribution image based on the movement information received from the first display device 400A indicating the movement of the viewer U's head, and distributes the captured image within the determined range as the distribution image.

[0102] In the first display mode, the second control unit 230 continuously receives sensor data representing the movement of the viewer U's head from the first display device 400A. The movement of the viewer U's head represented by the sensor data includes up-down and left-right movements of the user's head. Based on the sensor data received from the first display device 400A, the second control unit 230 determines a range to be cropped from the captured image as the distributed image.

[0103] For example, when the viewer U turns his head to the right, the second control unit 230 moves the area captured from the captured image to the right by a movement amount corresponding to the determined amount of head rotation.

[0104] Furthermore, when the viewer U turns their head to the left, the second control unit 230 moves the area captured from the captured image to the left by an amount corresponding to the determined amount of head rotation.

[0105] The second display mode is a display mode in which the range of the distributed image distributed to the display device 400 is fixed. When the display mode is the second display mode, the second control unit 230 always cuts a fixed range of the captured image uploaded from the vehicle-mounted device 30 and distributes the image of the cut range as the distributed image.

[0106] When the type of display device indicated by the acquired device information is the first display device 400A, the second control unit 230 distributes the distribution image of the channel corresponding to the channel information to the first display device 400A corresponding to the APP_ID. Additionally, the second control unit 230 receives sensor data indicating the movement of the viewer U's head from the first display device 400A at predetermined intervals. Based on the head movement of the viewer U indicated by the received sensor data, the second control unit 230 changes the range captured from the captured image as the distribution image.

[0107] For example, if other display devices 400 are connected to the first display device 400 on a network consisting of Wi-Fi routers, the first display device 400A can transfer the distributed image received from the distribution server 200 to the other display devices 400 via mirroring.

[0108] When a distributed image is mirrored to another display device, the first display device 400A obtains device information from the display device being mirrored and sends a mirroring notification containing the obtained device information to the distribution server 200.

[0109] When the first display device 400A receives a mirroring notification, the second control unit 230 does not incorporate data from the sensor data received from the first display device 400A indicating the vertical movement of the user's head into the range of the image to be distributed. The sensor data received from the first display device 400A also includes data indicating the user's head movement. When receiving the mirroring notification from the first display device 400A, the second control unit 230 removes the user's head movement indicated by the sensor data to determine the range of the captured image to be distributed. Alternatively, the second control unit 230 may determine the range of the image to be distributed in a way that the range of the image to be distributed is smaller than the range of the user's head movement indicated by the sensor data.

[0110] Furthermore, when the second control unit 230 receives a mirroring notification from the first display device 400A, it calculates the user's head rotation speed based on sensor data. The second control unit 230 determines whether the calculated head rotation speed is above or below a threshold. This head rotation speed may also include vertical movement. When the second control unit 230 determines that the head rotation speed is above or below a preset threshold, it reduces the frame rate of the distributed image sent to the first display device 400A by a predetermined value. The second control unit 230 reduces the frame rate by a predetermined value before the head rotation speed exceeds the preset threshold.

[0111] Furthermore, the second control unit 230 continuously calculates the user's head rotation speed based on sensor data and determines whether the head rotation speed is above a preset threshold for a preset time or longer.

[0112] If the user's head rotation speed remains above a threshold for a set time or longer, the second control unit 230 causes the display device, as indicated by the device information received from the first display device 400A, to display a prompt asking whether to begin image distribution to the display device. Upon receiving a response from the display device indicating that distribution has begun, the second control unit 230 sends a mirroring termination instruction to the first display device 400A and begins distributing the image to the display device.

[0113] Furthermore, when the user's head rotation speed is less than a threshold, the second control unit 230 causes the display device performing image distribution to display a prompt asking whether to end image distribution. Upon receiving a response from the display device allowing the distribution to end, the second control unit 230 sends a mirroring start instruction to the first display device 400A, thus ending image distribution to the display device.

[0114] Figure 8 and Figure 9 This is a flowchart illustrating the operations of the distribution server 200. (See attached diagram) Figure 8 and Figure 9 The flowchart shown illustrates the actions of the distribution server 200.

[0115] First, the second control unit 230 determines whether a distribution request has been received (step S1). If no distribution request has been received (step S1 / No), the second control unit 230 waits until a distribution request is received.

[0116] When the second control unit 230 receives a distribution request (step S1 / Yes), it obtains the APP_ID contained in the received distribution request (step S2). The second control unit 230 then refers to the user registration database 243 to obtain device information of the display device that has the APP corresponding to the obtained APP_ID installed.

[0117] Next, the second control unit 230 determines, based on the acquired device information, whether the source of the distribution request is the first display device 400A (step S3). If the second control unit 230 determines that the source of the distribution request is not the first display device 400A (step S3 / No) but the second display device 400B, it begins distributing the distribution image in the second display mode to the source of the distribution request (step S4). That is, the second control unit 230 always clips a fixed range of the captured image uploaded from the vehicle-mounted device 30 and distributes the clipped range as the distribution image.

[0118] Next, the second control unit 230 determines whether the distribution of the channel that is currently distributing the image has ended (step S5). If the distribution has not ended (step S5 / No), the second control unit 230 waits until the distribution ends. Alternatively, when the distribution ends (step S5 / Yes), the second control unit 230 terminates the processing flow.

[0119] Next, if the source of the distribution request is the first display device 400A (step S3 / Yes), the second control unit 230 starts distributing the distribution image to the first display device 400A, which is the source of the distribution request, in the first display mode (step S6).

[0120] Next, the second control unit 230 determines whether a mirroring notification has been received from the first display device 400A that started the distribution (step S7). If no mirroring notification has been received from the first display device 400A (step S7 / No), the second control unit 230 determines whether the distribution of the channel that is currently distributing the image has ended (step S8). If the distribution has not ended (step S8 / No), the second control unit 230 returns to the determination in step S7. Furthermore, if the distribution ends (step S8 / Yes), the second control unit 230 terminates the processing flow.

[0121] Next, when the second control unit 230 receives a mirroring notification from the first display device 400A (step S7 / Yes), it determines whether it has received sensor data sent from the first display device 400A (step S9). If the second control unit 230 does not receive sensor data from the first display device 400A (step S9 / No), it determines whether the distribution has ended (step S13).

[0122] When the distribution ends (step S13 / Yes), the second control unit 230 terminates the processing flow. Alternatively, if the distribution has not ended (step S13 / No), the second control unit 230 returns to the determination step S9 to determine whether sensor data has been received.

[0123] Next, when the second control unit 230 receives sensor data from the first display device 400A that is distributing images (step S9 / Yes), it determines whether the viewer U's head has moved based on the received sensor data (step S10). If the second control unit 230 determines that the viewer U's head has not moved (step S10 / No), it proceeds to the determination in step S13.

[0124] Furthermore, when the second control unit 230 determines that the viewer U's head has moved (step S10 / Yes), it calculates the direction and amount of the viewer U's head movement (step S11). Based on the calculated direction and amount of head movement, the second control unit 230 changes the range captured from the captured image as a distribution image (step S12). At this time, the second control unit 230 determines the range captured as a distribution image in a way that makes the range captured as a distribution image smaller than the range of the user's head movement indicated by the sensor data. Additionally, if the head movement includes vertical movement of the user's head, the second control unit 230 changes the range captured as a distribution image in the vertical direction without corresponding to the vertical movement of the user's head. The second control unit 230 then distributes the distribution image with the changed range to the first display device 400A.

[0125] Next, refer to Figure 9 The flowchart shown illustrates the actions of the distribution server 200.

[0126] Next, the second control unit 230 determines whether the rotation speed of the viewer U's head is above a preset threshold (step S14). If the user's head rotation speed is above the preset threshold (step S14 / Yes), the second control unit 230 determines whether the state of rotation speed above the threshold has lasted for a set time or more (step S15).

[0127] If the second control unit 230 determines that the head rotation speed is above a threshold for a set time that has not been completed (step S15 / No), it determines whether the frame rate of the distributed image distributed to the first display device 400A has been reduced by a predetermined value (step S16). If the frame rate of the distributed image has been reduced by the predetermined value (step S16 / Yes), the second control unit 230 returns to the previous state. Figure 8 The determination of step S9 in the flowchart shown.

[0128] If the frame rate of the distributed image is not reduced by a predetermined value (step S16 / No), the second control unit 230 reduces the frame rate of the distributed image distributed to the first display device 400A by a predetermined value (step S17). Afterwards, the second control unit 230 returns to... Figure 8 The determination of step S9 in the flowchart shown.

[0129] When the second control unit 230 determines that the head rotation speed has been above a threshold for a set time or longer (step S15 / Yes), it acquires the device information contained in the mirror notification received in step S7. Here, the case where the device information of the second display device 400B is acquired as device information will be described, but the device information may also be the first display device 400A. The second control unit 230 displays a prompt on the second display device 400B corresponding to the device information, asking whether to start the distribution of the distribution image to the first display device 400A (step S18).

[0130] Next, the second control unit 230 determines whether it has received a response allowing distribution from the second display device 400B (step S19). If it has not received a response allowing distribution (step S19 / No), the second control unit 230 returns to the previous state. Figure 8 The determination of step S9 in the flowchart shown.

[0131] When the second control unit 230 receives a reply allowing distribution from the second display device 400B (step S19 / Yes), it sends a mirror interruption instruction to the first display device 400A (step S20) and begins distribution to the display device (step S21).

[0132] Furthermore, if the second control unit 230 determines in step S14 that the head rotation speed is less than a threshold (step S14 / No), it determines whether the distribution server 200 is executing a sharing mode that distributes the same image to multiple display devices (step S22). That is, the second control unit 230 determines in step S21 whether distribution to the second display device 400B has started.

[0133] If the second control unit 230 is not in shared mode (step S22 / No), it returns. Figure 8 The determination in step S9 of the flowchart shown. Furthermore, if the second control unit 230 is in sharing mode (step S22 / Yes), it causes the second display device 400B to display a distribution end guide indicating the end of distribution of the distributed image (step S23).

[0134] Next, the second control unit 230 determines whether it has received a response from the second display device 400B allowing the distribution to end (step S24). If the second control unit 230 does not receive a response allowing the distribution to end (step S24 / No), it returns to... Figure 8 The determination of step S9 in the flowchart shown.

[0135] When the second control unit 230 receives a response from the display device allowing the distribution to end (step S24 / Yes), it sends a restart instruction for the image to the first display device 400A (step S25). Afterwards, the second control unit 230 ends the distribution of the image to the second display device 400B (step S26).

[0136] [7. Variations]

[0137] The above embodiments are preferred embodiments of the present invention. However, the invention is not limited to the above embodiments, and various modifications can be made without departing from the spirit of the invention.

[0138] For example, in the above embodiment, when distributing the image to the first display device 400A, the range of the captured image distributed as the distribution image is smaller than the range of vertical movement indicated by the motion information, or the range of the captured image is not changed in the vertical direction. Similarly, when distributing the image to the second display device 400B, it can also be controlled so that the range of the captured image distributed as the distribution image is smaller than the range of vertical movement indicated by the motion information, or the range of the captured image is not changed in the vertical direction. In this case, a motion sensor is mounted on the second display device 400B, just like on the first display device 400A, and the movement of the second display device 400B is detected by this motion sensor.

[0139] Alternatively, the first display device 400A and the second display device 400B can be directly connected, for example, via Bluetooth or Wi-Fi Direct, so that the first display device 400A functions as a distribution server 200. In this case, the first display device 400A receives movement information from the second display device 400B indicating the movement of the second display device 400B, determines the range of the captured image to be distributed as a distribution image based on the received movement information, and distributes the captured image within the determined range as a distribution image to the second display device 400B.

[0140] Alternatively, the first display device 400A or the second display device 400B may not change the range of the distributed image displayed on the second display device 400B in the vertical direction.

[0141] For example, if the motion information obtained from the second display device 400B includes vertical movement, the first display device 400A can control the range of the captured image distributed as a distribution image to be smaller than the range of vertical movement indicated by the motion information, or not to change the range of the captured image in the vertical direction. Furthermore, if the distribution image distributed from the first display device 400A includes vertical movement, the second display device 400B can also crop a certain range from the center of the distribution image to generate a distribution image without vertical movement, and display the generated distribution image.

[0142] For example, in the above embodiment, the situation of photographing the interior of vehicle 3 by one distributor terminal 100 was described, but it is also possible to configure multiple distributor terminals 100 to be installed in vehicle 3, and to photograph the interior of vehicle 3 by these multiple distributor terminals 100.

[0143] Furthermore, in the above embodiment, the following structure is shown: the distributor terminal 100 is connected to the vehicle-mounted device 30 via short-range wireless communication, and the distributor terminal 100 sends captured images to the vehicle-mounted device 30. However, it can also be configured such that the distributor terminal 100 uploads the captured images to the distribution server 200, and the distribution server 200 uploads the captured images to the vehicle-mounted device 30. The distributor S visually confirms the preview image of the captured image displayed on the touch panel 34 of the vehicle-mounted device 30, thereby adjusting the viewing angle of the distributor terminal 100, which serves as the capturing device.

[0144] Furthermore, in the above embodiment, the case where the captured image taken by the distributor terminal 100 is sent to the distribution server 200 via the vehicle-mounted device 30 has been described. However, it is also possible for the distributor terminal 100 to send the captured image taken by the distributor terminal 100 to the distribution server 200.

[0145] in addition, Figure 8 as well as Figure 9 The processing units in the flowchart shown are divided according to the main processing content for easy understanding of the processing of the distribution server 200. This invention is not limited to... Figure 8 and Figure 9 The flowchart illustrates the method and name for dividing the processing units. Furthermore, the processing of the distribution server 200 can be divided into more processing units based on the processing content, or it can be divided into a single processing unit containing more processing. Additionally, the processing order in the flowchart is not limited to the example shown.

[0146] In addition, Figure 9In steps S19 and S24 of the flowchart shown, the start and end of the distribution to the second display device 400B are queried. However, it is also possible to start the distribution to the second display device 400B or end the distribution without querying the start and end of the distribution.

[0147] [8. Structure supported by the above embodiments]

[0148] The above implementation supports the following structures.

[0149] (Structure 1)

[0150] A distribution device includes: a communication unit that receives a distribution request; a storage unit that stores user identification information identifying a user in correspondence with device information identifying a display device used by the user, and stores a captured image for distribution received from a distributor terminal; and a control unit that acquires the user identification information and device information included in the distribution request, receives from the display device movement information indicating movement of the display device as indicated by the acquired device information, determines the distribution range of the captured image as a distribution image based on the received movement information, and distributes the captured image within the determined range as the distribution image to the display device. If the movement information includes vertical movement, the control unit either makes the distribution range of the captured image as the distribution image smaller than the vertical movement range indicated by the movement information, or does not change the vertical range of the captured image.

[0151] According to the distribution device of Structure 1, when distributing a distribution image to a display device, the range for distributing the captured image as a distribution image is determined based on movement information indicating the movement of the display device, and the captured image within the determined range is distributed to the display device as a distribution image. Furthermore, if the movement information includes vertical movement of the display device, the range for distributing the captured image as a distribution image is made smaller than the vertical movement range indicated by the movement information, or the range of the captured image is not changed in the vertical direction. Therefore, dizziness experienced by users viewing the distribution image can be reduced.

[0152] (Structure 2)

[0153] According to the distribution device of structure 1, when the display device indicated by the device information is a head-mounted first display device, the control unit receives movement information from the first display device indicating the movement of the head of a user wearing the first display device. If the movement information includes the vertical movement of the user's head, the control unit makes the range of the captured image distributed as the distribution image smaller than the range of the vertical movement indicated by the movement information, or does not change the range of the captured image in the vertical direction.

[0154] According to the distribution device of structure 2, when the motion information includes the vertical movement of the user's head, the range of the captured image distributed as the distributed image is made smaller than the range of vertical movement indicated by the motion information, or the range of the captured image is not changed in the vertical direction. Therefore, it is possible to reduce dizziness for the user viewing the distributed image.

[0155] (Structure 3)

[0156] According to the distribution device of structure 2, when the first display device transmits the distribution image to a second display device other than a head-mounted device and the movement information includes the vertical movement of the user's head, the control unit makes the range of the captured image distributed as the distribution image smaller than the range of vertical movement indicated by the movement information, or does not change the range of the captured image in the vertical direction.

[0157] According to the distribution device of structure 3, when the first display device transmits the distributed image to the second display device, and the motion information includes the vertical movement of the user's head, the range of the captured image distributed as the distributed image is made smaller than the vertical movement range indicated by the motion information, or the range of the captured image is not changed in the vertical direction. Therefore, it is possible to suppress dizziness in the user viewing the distributed image using the second display device.

[0158] (Structure 4)

[0159] According to the distribution device of structure 1, when the speed of movement indicated by the movement information is above a preset threshold, the control unit changes the frame rate of the distributed image distributed to the display device to be lower than when the speed of movement is less than the threshold.

[0160] According to the distribution device of structure 4, when the speed of movement indicated by the motion information is above a preset threshold, the frame rate of the distributed image distributed to the first display device is reduced. Therefore, it is possible to suppress dizziness in users who view the distributed image transmitted from the first display device using the second display device.

[0161] (Structure 5)

[0162] According to the distribution device of structure 3, when multiple device information corresponds to the acquired user identification information, and the first display device and the second display device are included in the display devices corresponding to the device information, the control unit executes a shared distribution mode to distribute the distribution image to multiple display devices corresponding to the multiple device information. When the user's head movement speed is above a threshold for a preset time or more, the control unit causes the second display device to display guidance information guiding the distribution in the display mode of the second display device, which does not change the range of the captured image used as the distribution image.

[0163] According to the distribution device in structure 5, if the user's head movement speed exceeds a threshold for a preset time or longer, guidance information is displayed on the second display device. This guidance information guides the second display device to distribute images in a display mode that does not change the range of the captured image. Therefore, it is possible to switch to a sharing mode according to the user's instructions using the second display device, and it is possible to suppress dizziness in users who are viewing images on the second display device.

[0164] (Structure 6)

[0165] According to the distribution device of structure 5, in the shared distribution mode, if the speed of the user's head movement is less than the threshold, the control unit instructs the first display device to transfer the distributed image to the second display device, thereby ending the distribution of the distributed image to the second display device.

[0166] According to the distribution device of structure 6, if the speed of the user's head movement is less than a threshold, the first display device is instructed to transfer the distributed image to the second display device, thus ending the distribution of the image to the second display device. Therefore, even if the user views the distributed image using the second display device, the transfer of the distributed image from the first display device to the second display device can be restarted if the user is unlikely to experience dizziness.

[0167] (Structure 7)

[0168] According to the distribution device of structure 5, in the shared distribution mode, if the speed of the user's head movement is less than the threshold, the control unit causes the second display device to display guidance information indicating the end of the distribution of the distributed image. When the control unit receives a notification from the second display device allowing the end of the distribution of the distributed image, it instructs the first display device to transfer the distributed image to the second display device and ends the distribution of the distributed image to the second display device.

[0169] According to the distribution device in structure 7, if the user's head movement speed is less than a threshold, guidance information indicating the end of image distribution is displayed on the second display device. If a notification allowing the end of image distribution is received, the first display device is instructed to transfer the image distribution to the second display device, thus ending the image distribution to the second display device. Therefore, even if the user views the image distribution on the second display device, the transfer of the image distribution from the first display device to the second display device can be restarted if the user is unlikely to experience dizziness.

[0170] (Structure 8)

[0171] A distribution method in which a distribution device performs the following steps: receiving a captured image for distribution from a distributor terminal and storing the received captured image in a storage unit provided by the distribution device; receiving a distribution request; obtaining user identification information and device information included in the distribution request; receiving movement information from a display device indicating movement of the display device as indicated by the obtained device information; and determining the distribution range of the captured image as a distribution image based on the received movement information, and distributing the captured image within the determined range as the distribution image to the display device. In the distribution step, if the movement information includes vertical movement, the distribution range of the captured image as the distribution image is made smaller than the vertical movement range indicated by the movement information, or the range of the captured image is not changed in the vertical direction.

[0172] According to the distribution method of structure 8, the same effect as the distribution device of structure 1 can be obtained.

[0173] (Structure 9)

[0174] A computer program product enables a computer in a distribution device to perform the following processes: receiving a captured image for distribution from a distributor terminal and storing the received captured image in a storage unit in the distribution device; receiving a distribution request; obtaining user identification information and device information included in the distribution request; receiving movement information from a display device indicating movement of the display device as indicated by the obtained device information; and determining, based on the received movement information, a distribution range of the captured image as a distribution image, and distributing the captured image within the determined range as the distribution image to the display device. In the distribution process, if the movement information includes vertical movement, the distribution range of the captured image as the distribution image is made smaller than the vertical movement range indicated by the movement information, or the range of the captured image is not changed in the vertical direction.

[0175] The computer program product based on Structure 9 can achieve the same effect as the distribution device based on Structure 1.

[0176] Explanation of reference numerals in the attached figures

[0177] 1…distribution system, 3…vehicle, 5…communication network, 30…in-vehicle device, 31…external communication interface, 32…in-vehicle short-range wireless communication interface, 33…in-vehicle communication interface, 34…touch panel, 35…sound processing unit, 36…speaker, 40…vehicle ECU, 100…distributor terminal, 110…wireless communication interface, 120…short-range wireless communication interface, 130…camera, 140…touch panel, 151…microphone, 153…speaker, 170…third control unit, 180…third memory, 181… …OS, 183…APP, 190…Third Processor, 200…Distribution Server, 210…Network Communication Interface, 230…Second Control Unit, 240…Second Memory, 241…Control Program, 243…User Registration Database, 245…Distribution Management Database, 247…Content Database, 250…Second Processor, 290…Third Processor, 300…Vehicle ECU, 310…Input / Output Interface, 330…First Control Unit, 340…First Memory, 345…Control Program, 350…First Processor, 400A…First display device, 400B…Second display device, 401A…Near-field wireless communication interface, 403A…Image display unit, 405A…Sensor unit, 407A…Microphone, 409A…Speaker, 410A…Fourth control unit, 420A…Fourth memory, 421A…OS, 423A…APP, 430A…Fourth processor, 451B…Wireless communication interface, 452B…Near-field wireless communication interface, 453B…Camera, 454B…Touch panel, 455B…Microphone, 456B…Speaker, 460B…Fifth control unit, 470B…Fifth memory, 471B…OS, 473B…APP, 480B…Fifth processor.

Claims

1. A dispensing device comprising: The communications department receives and distributes requests; The storage unit stores user identification information that identifies the user in relation to device information that identifies the display device used by the user, and also stores images captured for distribution received from the distributor's terminal; and The control unit acquires user identification information and device information included in the distribution request, receives movement information from the display device indicating movement of the display device as represented by the acquired device information, determines the range for distributing the captured image as a distribution image based on the received movement information, and distributes the captured image within the determined range as the distribution image to the display device. When the movement information includes vertical movement, the control unit makes the range of the captured image distributed as the distributed image smaller than the range of vertical movement indicated by the movement information, or does not change the range of the captured image in the vertical direction.

2. The dispensing device according to claim 1, wherein, When the display device indicated by the device information is a head-mounted first display device, the control unit receives movement information from the first display device indicating the movement of the head of the user wearing the first display device. When the movement information includes vertical movement of the user's head, the range of the captured image distributed as the distributed image is made smaller than the range of vertical movement indicated by the movement information, or the range of the captured image is not changed in the vertical direction.

3. The dispensing device according to claim 2, wherein, When the first display device transmits the distributed image to a second display device other than a head-mounted display and the movement information includes the vertical movement of the user's head, the control unit makes the range of the captured image distributed as the distributed image smaller than the range of vertical movement indicated by the movement information, or does not change the range of the captured image in the vertical direction.

4. The dispensing device according to claim 1, wherein, If the speed of movement indicated by the movement information is above a preset threshold, the control unit will change the frame rate of the distributed image distributed to the display device to be lower than if the speed of movement is less than the threshold.

5. The dispensing device according to claim 3, wherein, When multiple device information pieces correspond to the acquired user identification information, and the display devices corresponding to the device information include both the first display device and the second display device, the control unit executes a shared distribution mode to distribute the distribution image to the multiple display devices corresponding to the multiple device information pieces. When the user's head movement speed is above a threshold for a preset time or longer, the control unit causes the second display device to display guidance information for distribution in the display mode of the second display device, which does not change the range of the captured image used as the distributed image.

6. The dispensing device according to claim 5, wherein, In the shared distribution mode, if the speed at which the user's head moves is less than the threshold, the control unit instructs the first display device to transfer the distributed image to the second display device, thereby ending the distribution of the distributed image to the second display device.

7. The dispensing device according to claim 5, wherein, In the shared distribution mode, if the user's head movement speed is less than the threshold, the control unit causes the second display device to display guidance information indicating the end of the distribution of the distributed image. When the control unit receives a notification from the second display device that the distribution of the distributed image is allowed to end, it instructs the first display device to forward the distributed image to the second display device and ends the distribution of the distributed image to the second display device.

8. A distribution method, wherein a distribution device performs the following steps: The device receives images captured for distribution from the distributor's terminal and stores the received images in the storage unit of the distribution device. Receive distribution request; Obtain the user identification information and device information contained in the distribution request; Receive from the display device movement information representing the movement of the display device as indicated by the acquired device information; and Based on the received motion information, the range within which the captured image is distributed as a distribution image is determined, and the captured image within the determined range is distributed as the distribution image to the display device. In the distribution step, if the movement information includes vertical movement, the range of the captured image distributed as the distribution image is made smaller than the range of vertical movement indicated by the movement information, or the range of the captured image is not changed in the vertical direction.

9. A computer program product that causes a computer included in a distribution device to perform the following processes: The device receives images captured for distribution from the distributor's terminal and stores the received images in the storage unit of the distribution device. Receive distribution request; Obtain the user identification information and device information contained in the distribution request; Receive from the display device movement information representing the movement of the display device as indicated by the acquired device information; and Based on the received motion information, the range within which the captured image is distributed as a distribution image is determined, and the captured image within the determined range is distributed as the distribution image to the display device. In the distribution process, if the movement information includes vertical movement, the range of the captured image distributed as the distributed image is made smaller than the range of vertical movement indicated by the movement information, or the range of the captured image is not changed in the vertical direction.

Images