Distribution device, distribution system, and distribution method

Abstract

To reduce traffic without interrupting live video display which is already being displayed.SOLUTION: The distribution device 2 distributes the video stream data to a reproducer 3 for simultaneously displaying the respective video stream data in the respective areas. The distribution device 2 receives a region number change request for increasing the number of regions from the reproducer 3, continues distribution of the video stream data without changing the bit rate for the region already displayed, and starts distribution of the video stream data at the bit rate corresponding to the number of regions after the increase for the region to be newly displayed. The distribution device 2 receives a region number change request for reducing the number of regions from the reproducer 3, and terminates the distribution of the video stream data for a region which is no longer required to be displayed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a distribution device, a distribution system, and a distribution method. [Background technology]

[0002] Video distribution services that deliver images captured by network cameras in real time are available. As the resolution of display devices increases, some services divide a single display into multiple areas, and simultaneously display images captured by different network cameras in each area. For example, Patent Document 1 describes a video distribution device that includes a combined data generation unit that generates combined data, which is data generated from videos from each network camera.

[0003] Furthermore, as network cameras become more powerful, the bit rate of the data being distributed is also increasing. Therefore, when a request for video stream data exceeds the network's allowable bandwidth, it is important to adjust the bit rate to fit within the network's allowable bandwidth. Therefore, Patent Document 2 describes a network camera equipped with a control means that reduces the bit rate of wide-angle shot video stream data or zoom shot video stream data within the same group when the packet loss rate exceeds a certain value. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2023-139337 [Patent Document 2] Japanese Patent Application Laid-Open No. 2007-325109 Summary of the Invention [Problem to be solved by the invention]

[0005] When distributing video from multiple network cameras to the same display device as in Patent Document 1, an increase in the number of video stream data leads to an increase in communication volume, which becomes a problem. In this case, if control is performed to lower the bit rate of the video stream data as in Patent Document 2, reconnection is required at that time, and the live video that is already being displayed is interrupted. On the other hand, there are also cases where customer operations such as surveillance require that live video already being displayed be maintained without interruption. In such cases, it becomes impossible to control the bit rate of the video stream data already being displayed, and the system is unable to handle the increased traffic volume.

[0006] The present invention has been made in consideration of the above circumstances, and its main object is to reduce the amount of communication traffic without interrupting the display of live video that is already being displayed. [Means for solving the problem]

[0007] In order to solve the above problems, a distribution device of the present invention has the following features. The present invention provides a distribution device that distributes video stream data to a player that simultaneously displays each video stream data in each area, the distribution device comprising: The distribution device receives a request from the playback device to increase the number of areas, and continues to distribute the video stream data for areas already being displayed without changing the bit rate, and starts to distribute the video stream data for areas to be newly displayed at a bit rate corresponding to the increased number of areas. Other features will be described later. [Effects of the Invention]

[0008] According to the present invention, it is possible to reduce the amount of communication traffic without interrupting the display of live video that is already being displayed. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a configuration diagram of a distribution system according to an embodiment of the present invention. [Figure 2] 10 is a table illustrating an example of a setting storage unit according to the present embodiment. [Figure 3] FIG. 2 is a hardware configuration diagram of each device in the distribution system according to the present embodiment. [Figure 4] FIG. 2 is a sequence diagram showing the operation of the distribution system according to the present embodiment. [Figure 5] FIG. 10 is an explanatory diagram showing a process for increasing the number of divisions in a comparative example related to the present embodiment. [Figure 6] 10A and 10B are explanatory diagrams showing the process of increasing or decreasing the number of divisions in the present embodiment. [Figure 7] FIG. 10 is a diagram showing a screen divided into four parts according to the present embodiment. [Figure 8] FIG. 10 is a diagram showing a nine-division screen according to this embodiment. [Figure 9] FIG. 10 is a diagram showing a screen when specifying a camera change according to the present embodiment. [Figure 10] FIG. 10 is a diagram showing a screen after a camera change is designated from the state shown in FIG. 9 according to this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0011] FIG. 1 is a diagram showing the configuration of a distribution system 100. The distribution system 100 is configured by connecting a camera 1, a distribution device 2, and a playback device 3 via a network. The camera 1 transmits the captured video as video stream data to the distribution device 2. The distribution device 2 distributes the video stream data received from the multiple cameras 1 to the playback device 3. The playback device 3 has a display control unit 31 and an operation unit 32, and is operated by the user 9 to simultaneously display each piece of video stream data in each area. The distribution device 2 distributes the video stream data to the playback device 3. The distribution device 2 includes a video receiving unit 21, a video control unit 22, a video distribution unit 23, a setting unit 24, and a setting storage unit 25.

[0012] When the player 3 receives a video playback start operation from the user 9 via the operation unit 32, it sends a video distribution request created from that start operation to the video control unit 22. The video distribution request includes information on the current (here, initial) number of screen divisions. The number of screen divisions (hereinafter sometimes abbreviated as "number of divisions") is the number of live videos that are simultaneously displayed in each window when the display of the player 3 is divided into multiple areas (hereinafter "windows"). One piece of video stream data is displayed per window. The video control unit 22 notifies the setting unit 24 of the screen division number information of the received video distribution request.

[0013] FIG. 2 is a table showing an example of the setting storage unit 25. The setting unit 24 refers to the setting storage unit 25, finds the bit rate (BPS: Bits Per Second) of the video stream data corresponding to the combination of the notified screen division number information and the recommended resolution of the playback device 3, and sends the result back to the video control unit 22. For this reason, the setting storage unit 25 has registered in advance the bit rate [Mbps] to be set in the camera 1 and the layout information to be set in the playback device 3 for each combination of the recommended resolution and the number of divisions. For example, for player 3, which has a medium display size and a division count of 4, the recommended resolution is full HD (1920 x 1080), and the bit rate per video stream data is 2 Mbps. In other words, since the division count is 4, the total bit rate for the four windows is 2 x 4 = 8 Mbps. On the other hand, for player 3, which has a larger display size and a division number of 4, the recommended resolution is 4K (3840 x 2160) and the bit rate per window is 8 Mbps. In other words, the total bit rate for the four windows is 8 x 4 = 32 Mbps.

[0014] Furthermore, the layout information in the setting storage unit 25 is information indicating that, for example, when the number of divisions is 9, three vertical windows by three horizontal windows are arranged in a grid pattern on the screen. This layout information is read by the setting unit 24 from the setting storage unit 25, and then transmitted from the setting unit 24 to the display control unit 31 via the video distribution unit 23. The display control unit 31 arranges the video stream data from the video distribution unit 23 in each window on the screen according to the received layout information.

[0015] Returning to FIG. 1, the video control unit 22 transmits a video transmission command specifying the bit rate notified by the setting unit 24 to each camera 1 that captures the video to be played back in each window. Each camera 1 transmits video stream data obtained by encoding the captured video according to the bit rate of the video transmission command to the video receiving unit 21. The video receiving unit 21 notifies the video distributing unit 23 of each received video stream data. The video distribution unit 23 distributes each notified video stream data to the display control unit 31 of the playback device 3 in accordance with a streaming distribution standard such as HLS (HTTP Live Streaming). As a result, the display control unit 31 plays back each distributed video stream data in each window divided according to the division number.

[0016] When the playback device 3 receives an operation to change the number of screen divisions from the user 9 via the operation unit 32, it sends a division change request created from the change operation to the setting unit 24. The division change request includes new screen division number information. The setting unit 24 stores a history of previously corresponding screen division number information and compares it with the new screen division number information to classify each window as follows: (a) A "continued window" that is an "existing window" that is currently being distributed and will continue to be displayed even after a request to change the division is received. (b) An "increased window" that was not previously displayed but begins to be displayed upon receiving a division change request. (c) "Reduced windows" are existing windows that will no longer need to be displayed following a request to change the division. On the other hand, the window formed according to the initial number of screen divisions is referred to as the "initial window."

[0017] Then, the distribution device 2 receives a region count change request (division change request) from the playback device 3 to increase the number of regions (screen division number), and (a) continues distribution of video stream data for regions already being displayed (continuation window) without changing the bit rate. Also, (b) for regions to be newly displayed (increased window), the distribution device 2 starts distribution of video stream data at a bit rate according to the increased number of regions. Meanwhile, the distribution device 2 receives a region number change request from the playback device 3 to decrease the number of regions, and (c) terminates distribution of video stream data for regions that will no longer need to be displayed (decreased windows).

[0018] Specifically, the setting unit 24 uses different bit rate settings for the video stream data being played back and the video stream data before playback, as follows. (a) The setting unit 24 does not change the bit rate of the continuation window and does not interrupt the video playback. (b) The setting unit 24 calculates a new bit rate for the increased window based on the new number of screen divisions. To do this, the setting unit 24 obtains a bit rate for the increased window that matches the combination of the recommended resolution and the new number of screen divisions from the setting storage unit 25. The setting unit 24 then transmits a video transmission command specifying the bit rate for the increased window to each of the corresponding cameras 1 via the video control unit 22. (c) The setting unit 24 transmits a command to stop the video to the reduction window to each of the corresponding cameras 1 via the video control unit 22 .

[0019] For the increased window (b), the setting unit 24 maintains a minimum bit rate assuming the recommended resolution, thereby preventing video jerks that may occur when the bit rate is lowered too much, or preventing the resolution from falling below the recommended resolution of the playback device.

[0020] Then, the video distribution unit 23 distributes the video stream data corresponding to the division change request as follows. (a) The video stream data in the continuation window continues to be delivered without interruption without changing the bit rate. (b) The video stream data of the increased window starts to be delivered according to the bit rate of the new number of screen divisions (generally at a lower bit rate than that of the continuous window). (c) The video stream data of the reduced window is cut off from the distribution. As a result, the display control unit 31 of the playback device 3 can reflect, on the display screen, each window of the new screen division number information to which the division change request has been applied.

[0021] FIG. 3 is a diagram showing the hardware configuration of each device (camera 1, distribution device 2, and playback device 3) in distribution system 100. Each device of the distribution system 100 is configured as a computer 900 having a CPU 901 , a RAM 902 , a ROM 903 , a HDD 904 , a communication I / F 905 , an input / output I / F 906 , and a media I / F 907 . The communication I / F 905 is connected to an external communication device 915. The input / output I / F 906 is connected to an input / output device 916. The media I / F 907 reads and writes data from a recording medium 917. Furthermore, the CPU 901 executes a program (also called an application or an app for short) loaded into the RAM 902 to improve and control each processing unit. This program can be distributed via a communication line or recorded on a recording medium 917 such as a CD-ROM and distributed.

[0022] FIG. 4 is a sequence diagram showing the operation of the distribution system 100. First, the first playback process (S110) will be described. The playback device 3 transmits a video distribution request specifying the initial screen division number to the distribution device 2 (S111). The distribution device 2 calculates the BL (bit rate) of the initial window (S112). The distribution device 2 starts distribution of video stream data to be displayed in the initial window according to the calculated bit rate to the playback device 3 (S113). The playback device 3 plays the distribution of S113 in the initial window (S114).

[0023] Next, the playback process (S120) when the number of divisions is increased due to a division change request will be described. When the playback device 3 receives a request to increase the number of screen divisions through an operation by the user 9 (S121), it notifies the distribution device 2 of the new number of screen divisions as a division change request. The distribution device 2 forms new added windows while making all existing windows into continuing windows so that "new number of screen divisions = number of continuing windows + number of added windows" holds. Then, the distribution device 2 calculates the BL of the added windows based on the new number of screen divisions (S122). The distribution device 2 continues distribution of the continuous window and starts distribution of the increased window at the BL determined in S122 (S123). The playback device 3 receives the distribution of S123 and plays it in each window (S124).

[0024] Next, the playback process (S130) when the number of divisions is reduced in response to a division change request will be described. When the playback device 3 receives a command to reduce the number of screen divisions through an operation by the user 9 (S131), it notifies the distribution device 2 of the new number of screen divisions as a division change request. The distribution device 2 classifies each existing window into a continuing window or a decreasing window so that the "new number of screen divisions = number of continuing windows" holds. Then, the distribution device 2 continues distribution of the continuation window (S133) while disconnecting the decrease window (BL of the decrease window=0) (S132). The playback device 3 receives the distribution of S133 and plays it in the continuation window (S134).

[0025] Also, the playback process (S140) when camera 1, which is the source of the video stream data captured through an existing window, is changed in response to a camera change request will be described. When the playback device 3 receives a request from the user 9 to change the camera 1 of a specified existing window (specified window) (S141), the playback device 3 notifies the distribution device 2 of this request as a camera change request. The distribution device 2 calculates the BL of the specified window based on the current number of screen divisions (S142). The distribution device 2 continues distribution to existing windows other than the specified window, and starts distribution to the specified window at the BL determined in S142 (S143). The playback device 3 receives the distribution of S143 and plays it in each window (S144).

[0026] FIG. 5 is an explanatory diagram showing the process of increasing the number of divisions in the comparative example. In the initial state at time t11, four windows F1 to F4 are being delivered at BL=5 [Mbps] each, and the total communication volume is 20 [Mbps]. At time t12, the next time after time t11, a request is received to increase the number of segments from 4 to 9, and the distribution device in the comparative example starts distribution in five new windows F5 to F9, each with a BL of 5 Mbps, for a total communication volume of 45 Mbps. However, 45 Mbps exceeds the network's permissible bandwidth per person (for example, 40 Mbps). At time t13, the time following time t12, the distribution device of the comparative example reduces the BL of each window F1-F9 from 5 Mbps to 2 Mbps while keeping the number of divisions at 9, resulting in a total communication volume of 18 Mbps. 18 Mbps is within the network's permissible bandwidth per person. However, as the BL of each window F1-F9 is changed, the live video display that was already being displayed is interrupted, causing disruption to monitoring operations.

[0027] FIG. 6 is an explanatory diagram showing the process of increasing or decreasing the number of divisions in this embodiment. In the initial state at time t21, four windows F1 to F4 are being delivered at BL=5 [Mbps] each, and the total communication volume is 20 [Mbps]. At time t22, the time following time t21, a partition change request to increase the number of partitions from 4 to 9 is received (S121 in FIG. 4), and the distribution device 2 starts distribution for five new windows F5 to F9. The difference from time t12 in FIG. 5 is that the distribution device 2 recalculates the BL for the new windows F5 to F9 based on the new number of partitions = 9. As a result, the total communication volume becomes 20 + 10 = 30 [Mbps], which is within the network's allowable bandwidth per person (e.g., 40 [Mbps]). At time t23, the time following time t22, a segmentation change request is received to reduce the number of segments from nine to four (S131 in FIG. 4). The distribution device 2 keeps the first five segments F1 to F4 and reduces the last five segments F5 to F9, resulting in a total communication volume of 20 Mbps. The distribution device 2 cuts off the distribution of the video stream data for each of the segments F5 to F9. The playback device 3 hides each of the segments F5 to F9 and resizes the display of the remaining segments F1 to F4.

[0028] FIG. 7 shows a screen divided into four parts. 7 is, for example, a screen on a browser of the playback device 3, and the browser's operation sections (URL input field, back button, page reload button) are not shown. The four windows F1 to F4 are implemented as "frames" that represent parts of the display area within the browser, for example. Each of the four windows F1 to F4 displays an image captured by a different camera 1. Therefore, when the radio button for screen division number=4 (2 vertical windows×2 horizontal windows) is clicked, the operation unit 32 transmits a video distribution request for screen division number=4 to the distribution device 2 (S111). The user 9 selects which window to display the image captured by which camera 1 for each window through the operation unit 32. The video distribution request also includes information specifying the camera 1 for each selected window.

[0029] FIG. 8 shows a nine-split screen. 7, when the radio button for screen division number=9 (3 windows vertically×3 windows horizontally) is clicked, operation unit 32 transmits a video distribution request for screen division number=9 to distribution device 2 (S121). Distribution device 2 starts distribution of new windows F5 to F9 while continuing distribution of windows F1 to F4 whose BL is not changed (S123). In response to the selection of 9 screen divisions, the display control unit 31 reduces the size of the windows F1 to F4 from the state in Figure 7 to the same size as the other new windows F5 to F9. This allows the display control unit 31 to increase the number of divisions without requiring a browser page reload operation, and therefore allows the video display in the windows F1 to F4 to continue without causing a temporary disconnection due to a page reload.

[0030] 8, when the radio button for the number of screen divisions=4 (2 vertical windows×2 horizontal windows) is clicked, the operation unit 32 transmits a division change request to the distribution device 2 to reduce the number of divisions from 9 to 4 (S131). As a result, the display control unit 31 hides windows F5 to F9 and enlarges windows F1 to F4 from the state in FIG. 8, thereby returning to the layout in FIG. 7.

[0031] FIG. 9 is a diagram showing the screen when specifying a camera change. When the operation unit 32 receives a command to change the camera 1 of the window F4 from the state shown in Fig. 7 by double-clicking on the window F4, it displays a selection field (first camera to fourth camera) for the camera 1 to be applied to the window F4. As the initial value of the selection field for the camera 1, the second camera currently broadcasting in the window F4 is selected by the radio button.

[0032] FIG. 10 is a diagram showing the screen after the camera change is specified from the state shown in FIG. 9, when the user 9 clicks the radio button for the fourth camera, the operation unit 32 notifies the distribution device 2 of a camera change request to change the camera 1 in window F4 from the second camera to the fourth camera (S141). The distribution device 2 issues a video stop command to stop the distribution of the second camera and a video transmission command to start the distribution of the fourth camera to each camera 1. As a result, the display control unit 31 displays the video stream data of the fourth camera in window F4 (S144). In this way, the playback device 3 can switch the video stream data displayed in each area. Then, the distribution device 2 ends distribution of the video stream data before the switch and starts distribution of the video stream data after the switch at a bit rate according to the current number of areas.

[0033] Furthermore, when the display control unit 31 receives various layout change operations from the operation unit 32 for each of the screen displays in Figures 7 to 10, such as resizing a window, displaying a window, hiding a window, and changing the display position of a window, the display control unit 31 may reflect the layout change in the screen display. For example, the playback device 3 can switch each area (each window) between a display state and a non-display state. Then, upon receiving a request to change the number of areas to increase the number of areas, the distribution device 2 may switch the bit rate for areas that already exist but are in a non-display state to a bit rate that corresponds to the increased number of areas. This allows the user 9 to remain unaware of the change in bit rate for non-display areas, even if the live video display is interrupted.

[0034] As described above, when the number of screen divisions increases, the distribution device 2 of the present embodiment does not change the bit rate of the live video already being displayed, but instead changes the bit rate to match the number of screen divisions of the newly displayed live video. This allows the playback device 3 to reduce communication traffic without interrupting the display of the live video already being displayed.

[0035] Furthermore, the present invention is not limited to the above-described embodiments, and various other applications and modifications are possible without departing from the spirit of the present invention as defined in the claims. For example, the above-described embodiments provide a detailed and specific description of the configuration of the distribution system 100 in order to clearly explain the present invention, and the system is not necessarily limited to one that includes all of the components described. Furthermore, it is possible to replace part of the configuration of one embodiment with the components of another embodiment. It is also possible to add the components of another embodiment to the configuration of one embodiment. It is also possible to add, replace, or delete other components from part of the configuration of each embodiment.

[0036] Furthermore, the above-described configurations, functions, processing units, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. As the hardware, a broad processor device such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit) may be used. Furthermore, each component of the distribution system 100 according to the above-described embodiment may be implemented in any hardware as long as the respective hardware can transmit and receive information to and from each other via a network. Furthermore, the processing performed by a certain processing unit may be realized by a single piece of hardware, or may be realized by distributed processing using multiple pieces of hardware. [Explanation of symbols]

[0037] 1 camera 2. Distribution Device 3 Regenerator 100 Distribution System

Claims

1. A distribution device that distributes video stream data to a player that simultaneously displays each video stream data in each area, The distribution device receives a request from the playback device to increase the number of areas, and continues distribution of the video stream data for the areas already being displayed without changing the bit rate, and starts distribution of the video stream data for the areas to be newly displayed at a bit rate according to the increased number of areas. Distribution device.

2. The distribution device receives a request from the playback device to reduce the number of areas, and terminates distribution of the video stream data for areas that will no longer need to be displayed. The distribution device according to claim 1 .

3. A distribution system comprising the distribution device according to claim 1 or 2 and the playback device, the playback device is capable of switching each area between a display state and a non-display state; The distribution device receives a request to increase the number of areas, and switches the bit rate of the areas that already exist but are not displayed to a bit rate that corresponds to the increased number of areas. Delivery system.

4. the playback device is capable of switching the video stream data displayed in each area; the distribution device terminates distribution of the video stream data before the switching and starts distribution of the video stream data after the switching at a bit rate according to the current number of areas. The distribution system according to claim 3 .

5. A distribution method by a distribution device that distributes video stream data to a player that simultaneously displays each video stream data in each area, comprising: The delivery device receives a request from the playback device to increase the number of areas, and continues delivery of the video stream data for the areas already being displayed without changing the bit rate, and starts delivery of the video stream data for the areas to be newly displayed at a bit rate according to the increased number of areas. Delivery method.

Images