Imaging system, distribution device, imaging device, control method, and program

Abstract

To shorten a video switching time.SOLUTION: A distribution device communicates with an imaging apparatus and distributes a video of tiles corresponding to resolution and an area requested from a client on a network, and the distribution device has: a holding unit that, every time it receives a command to designate a region of attention from a client, updates and holds the number of times of designation of tiles specified by the command; and a setting unit that determines the ratio for using in-frame compression of individual tiles according to the number of times of designation of tiles held by the holding unit, and sets the ratio to the imaging apparatus to code the tiles according to the ratio.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an imaging system, a distribution device, an imaging device, a control method, and a program. [Background technology]

[0002] Conventionally, there is known a technique for increasing the ratio of intra-frame compressed frames when a predetermined object is detected (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 5082134 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present invention is to provide a technique for shortening the video switching time. [Means for solving the problem]

[0005] In order to solve this problem, for example, an imaging system according to the present invention has the following configuration. An imaging system including an imaging device and a distribution device that distributes an image captured by the imaging device via a network, The imaging device includes: an imaging means for capturing images at a predetermined frame rate and outputting images with a plurality of resolutions; Each image of each resolution obtained by the imaging means is number A dividing means for dividing the image into tiles; An encoding means for encoding the image of each tile obtained by the dividing means using intra-frame compression and inter-frame compression, The above Intraframe Compression and the interframe compression An encoding means capable of independently setting a ratio of using the an output means for outputting the encoded data of each tile obtained by the encoding means to the distribution device; The distribution device includes: a distribution means for distributing, to a client on a network, encoded data of a tile corresponding to a gaze area requested by the client as a video stream; a storage means for updating and storing the number of times a tile specified by a command specifying a gaze area is specified when the command is received from a client on the network; an encoding control means for determining a ratio of using intra-frame compression for each tile according to the number of times each tile is designated and stored in said storage means, and setting said ratio in said encoding means; has. [Brief description of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram of a system according to a first embodiment. [Diagram 2] FIG. 2 is an explanatory diagram of tile division according to the first embodiment. [Diagram 3] FIG. 1 is an explanatory diagram of a system according to a first embodiment. [Figure 4] FIG. 4 is an explanatory diagram of time-series switching according to the first embodiment. [Diagram 5] FIG. 1 is a block diagram according to a first embodiment. [Figure 6] FIG. 11 is an explanatory diagram of a weight distribution according to the second embodiment. [Figure 7] 6 is a flowchart showing intra-frame compression interval setting processing in the server according to the first embodiment. [Figure 8] FIG. 4 is an explanatory diagram of the correspondence between a fixation area and a corresponding tile in the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0008] [First embodiment] Before showing a configuration example of an imaging system according to the first embodiment, a problem with an imaging system that achieves high scalability and bandwidth management will be described with reference to FIGS. 1, 2, 3, and 4. FIG.

[0009] FIG. 1 is a conceptual diagram showing an example of the configuration of an imaging system that provides high scalability and bandwidth management.

[0010] The photographing system 100 is composed of an imaging device 102 and a server 103. The server 103 stores the scene 101 supplied from the imaging device 102 in a predetermined storage device (such as a hard disk) and distributes it to clients 104 and 105 operated by users via a network (whether wired or wireless). Note that although the clients 104 and 105 are arranged outside the imaging system 100 here, they may be members constituting the imaging system 100. Furthermore, storage in the storage device may be temporary, or in the case of real-time distribution, storage may not be necessary. Also, although the figure shows two clients, this is for convenience, and there is no particular limit to the number of clients.

[0011] The imaging device 102 converts the captured scene 101 into images at multiple resolutions and transmits them to the server 103. The images generated by the imaging device 102 are, for example, a low-resolution image 106, a medium-resolution image 107, and a high-resolution image 108. Here, the image with the lowest resolution is designated as the image of layer 1, and the image with the medium resolution is designated as layer 2, and the layer number is increased as the captured resolution increases. For example, the imaging device 101 divides the image of layer 1 into one tile (not divided), the image of layer 2 into, for example, 2×2 tiles, and the image of layer 3 into 4×4 tiles, and compresses each divided tile image using a compressor with intra-frame compression and inter-frame compression such as H.264, and transmits the compressed images to the server 103. Here, assuming that the maximum number of layers is 3 and the number of divisions in the i-th layer is n i In the illustrated example, n1=1, n2=4, and n3=16, but this is for simplicity, and the maximum number of layers and the number of divisions in each layer can be any number. Here, the tile image generated by the imaging device 102 is expressed as A, where i is the number of layers to which it belongs and j is the value indicating the position of the tile. ij Here, the position of the tiles is represented as follows: for example, as shown in FIG. 2, the top left corner is the starting point and is numbered as "1", the tile to the right is the next number, and so on in the order of raster scan. Then, the server 103 can assign a total of Σn i The tile images (tile videos) will be received and temporarily stored.

[0012] On the other hand, the client 104 or the client 105, or both of them, transmit a request to acquire video data to the server 103. For example, when the server 103 receives a video acquisition request from the client 104, the server 103 first transmits the entire captured scene video 106 of layer 1 to the client 104.

[0013] Here, reference numeral 300 in Fig. 3 denotes an image (whole image = 1 tile) of layer 1 displayed on (the display device of) client 104. Now, assume that a user operates a mouse or touch panel (not shown) to set area 301 in the displayed image 300 as an area to be gazed upon (hereinafter, gaze area). At this time, client 104 transmits an image acquisition command having information specifying the gaze area to server 103. In response to this, server 103 transmits to client 104 an image of tile A21 of layer 2, which corresponds to the position of the gaze area. As a result, the image displayed on client 104 changes to image 303.

[0014] Furthermore, when a user viewing video 300 sets area 302 as the gaze area, server 103 transmits the image of tile A32 belonging to layer 3 to client 104. As a result, the image displayed on client 104 changes to image 304. Note that hereinafter, the expression "image of tile A21" transmitted to the client will simply be referred to as tile image A21. FIG. 4 is a schematic diagram showing the above-mentioned operations and processes on a time axis. The direction from left to right in the drawing represents the flow of time. Note that since the flow along the time axis in FIG. 3 is shown here, only tile images A11, A21, and A32 held by server 104 are shown.

[0015] In FIG. 4, "I" indicates a frame that uses only intraframe compression as an encoding method, and "P" indicates a frame that also includes interframe compression.

[0016] First, video stream SA11 of tile image A11 is transmitted to client 104, and as a result, video 300 is displayed on client 104.

[0017] Suppose that the user sets area 301 as the area of ​​gaze at time t1. In this case, server 104 transmits the previous video stream SA11 until time t2 when frame I of tile image A21 becomes compressed using only intra-frame compression. Then, at time t2 when frame I of tile image A21 compressed using only intra-frame compression is transmitted, server 103 switches to transmitting video stream SA21 of tile image A21 corresponding to area of ​​gaze 301. In other words, if the user specifies area of ​​gaze 301 at time t1, tile image A21 of area of ​​gaze 301 will not be displayed on client 104 until time t2 or later.

[0018] Similarly, suppose that the user sets area 302 as the area of ​​focus at time t3. In this case, server 104 transmits the previous video stream SA11 until time t4, when frame I of tile image A32 is compressed using only intra-frame compression. It is not until time t4, when frame I of tile image A32 compressed using only intra-frame compression is transmitted, that the delivery is switched to video stream SA32 of tile image A32 corresponding to area 302. In other words, if the user specifies area 302 as the area of ​​focus at time t3, tile image A32 in area of ​​focus 302 is displayed on client 104 from time t4 onwards.

[0019] As described above, it takes a considerable amount of time from when a gaze area is requested until the image is switched to a detailed (higher resolution) image of the gaze area. The maximum time for this switching is T w Then, T w =1 / F I Here, F Iis the reciprocal of the period in which intraframe compressed frames appear, in other words, the number of times (frequency) intraframe compression is performed per second. Therefore, in order to shorten the switching time, the period in which intraframe compressed frames appear should be shortened (the frequency should be increased). However, since the amount of code generated by intraframe compression is far greater than the amount of code generated by interframe compression, the shorter the period in which intraframe compression is used, the more pressure is placed on the communication band between the imaging device 102 and the server 103, and in some cases, it may become impossible to transmit video.

[0020] In this embodiment, an imaging system is presented in which the period of intra-frame compression of the image is shortened according to how frequently the user sets the area as the gaze area, and the maximum switching time Tw to this area is shortened, while when the frequency of setting the gaze area is low, the period of the intra-frame compressed frames is lengthened (however, not exceeding the maximum period set in advance), thereby preventing an increase in the amount of code for the entire image.

[0021] 5 shows the configuration of an image capturing system 500 in the first embodiment. The configuration and operation will be described below with reference to the figure.

[0022] The imaging system 500 is composed of an imaging device 501 and a distribution server 502. The imaging device 501 is composed of an imaging unit 503, an image tile division unit 504, an encoding unit 505, and a transmission unit 506. The imaging unit 503 outputs a plurality of images with different resolutions from the captured images. Here, the image with the lowest resolution is called layer 1, and the layers with higher resolutions are called images of layer 2, layer 3, ..., layer k. The imaging unit 503 outputs the images of each layer to the image tile division unit 504. Simply put, the imaging unit 503 has an imaging element with a resolution of layer k, and generates images of each layer in stages by calculating the average value of an area corresponding to one pixel of each layer. Note that the frame rate at which the imaging unit 502 captures images is not particularly important, but here, the imaging unit 502 captures images at 30 frames per second.

[0023] The image tile division unit 504 divides the image of each layer into a number of tiles determined for each layer. Here, the number of divisions of the image of layer i is m i The image tile division unit 504 outputs the divided tile images to the encoding unit 505.

[0024] The encoding unit 505 appropriately selects intraframe compression or interframe compression to encode the tile images input from the image tile division unit 504. In the embodiment, the encoding unit 505 can set the ratio at which intraframe compression is used (the interval at which intraframe compression is used) independently for each tile. The ratio at which intraframe compression is performed can be set by the encoding control unit 507 of the distribution server 502. The compression method of the tile images by the encoding unit 505 is an encoding format using intraframe compression and interframe compression such as H.264, but is not limited to H.264. The encoding unit 505 encodes each tile image of each layer and outputs the encoded image to the transmission unit 506. Here, the total number M of tiles generated from an image obtained by one imaging by the imaging unit 503 is given by the following formula (1-1).

[0025]

number

[0026] The transmission unit 506 outputs the coded data of the tile images received from the coding unit 505 to the distribution server 502.

[0027] Next, the distribution server 502 will be described. The distribution server 502 controls the imaging device 501 and distributes the video captured by the imaging device 501 to the client on the network in response to a request from the client on the network. The distribution server 502 has a control unit 550 that controls the entire distribution server. The control unit 550 has a CPU, a ROM that stores the program executed by the CPU, and a RAM used as a work area. The control unit 550 controls the video receiving unit 511, the video storage unit 512, the video transmitting unit 513, the command receiving unit 510, the video to be distributed determination unit 509, the video frequency calculation unit 508, and the encoding control unit 507. Note that each of the processing units indicated by the reference numerals 507 to 513 may be executed by the control unit 550.

[0028] Video receiving unit 511 receives the encoded data of tiles of each layer input from imaging device 501, and outputs it to video storage unit 512. Video storage unit 512 temporarily stores the encoded data of tiles input from video receiving unit 511 in a predetermined storage medium. Here, the storage medium is, for example, an SSD (Solid State Drive). Furthermore, when a distribution video request is made by distribution video determination unit 509, video storage unit 512 outputs encoded data of a tile image corresponding to the requested video to video sending unit 513. Note that, although storage in a storage medium is described in the embodiment, the storage medium is not an essential function, and it is also possible to simply output the video for which a distribution video request has been made.

[0029] The video transmission unit 513 transmits the encoded data of the video (a group of tiles constituting the video) input from the video storage unit 512 to the requesting client.

[0030] The command receiving unit 510 receives various commands from clients on the network. If the received command is a command to specify a gaze area, the command receiving unit 510 outputs it to the distribution video determination unit 509. The gaze area specification command is the coordinates of the top left and bottom right of a rectangle representing the gaze area in the currently received video, or the coordinates of the top left of the rectangle and the horizontal and vertical sizes of the rectangle. However, if the gaze area is specified on a tile-by-tile basis, information specifying the tile representing the gaze area may be used.

[0031] The distribution video determination unit 509 determines the tile (layer and tile position) that has the largest number of pixels representing the gaze area indicated by the command received by the command receiving unit 510, and transmits this to the video storage unit 512.

[0032] The video frequency calculation unit 508 holds the number of times all tiles have been selected. Then, the video frequency calculation unit 508 increments the number of times the tile has been selected transmitted from the distribution video determination unit 509, and updates the held number. Then, the video frequency calculation unit 508 transmits the updated number of times each tile has been selected to the encoding control unit 507. The encoding control unit 507 receives the number of times all tile videos have been selected from the video frequency calculation unit 508, and determines the ratio at which intra-frame compression is performed for each tile, in other words, the number of times intra-frame compression is performed per second, as follows:

[0033] For example, the number of times Nab that intra-frame compression is performed per second for tile image Aab is determined according to the following formula (1-2).

[0034]

number

[0035]

number

[0036]

number

[0037] The number of times intra-frame compression is performed per second is N, as shown in the following equation (1-4), so it can be said that the total amount of code for all tile images per second remains roughly unchanged.

[0038]

number

[0039] Such a tile image A ab The number of times per second that intraframe compression is performed is N. ab By using this method, the number of intra-frame compressed frames can be increased for tile images that are selected frequently. For example, if k1 and k2 are fixed values, and tile image A 11 and A ab Assume that the number of times that each of the images is selected is k1 and k2 times more than the other images, and that the other images are selected with equal probability. Then, A 11 , A ab and other tiles (A αβ The number of selections for each of the above is calculated using the following equations (1-5) to (1-7).

[0040]

number

[0041] Then, tile image A 11 ,A ab , and other tile images A αβ Interval I during which intraframe compression is performed intv () is expressed as the following equations (1-8) to (1-10).

[0042]

number

[0043] In other words, tile image A 11 From tile images other than A 11 If the gaze area is specified, the tile image A 11 The interval (period) of the intraframe compression frame is changed from M / N to M / (N×k1). ab From tile images other than tile image A ab If a gaze area is specified, the tile image A ab The interval between intra-frame compressed frames is changed from M / N to M / (N×k2). Therefore, the more times a tile image is designated as being gazed upon, the smaller the interval between its intra-frame compressed frames will be.

[0044] Figure 7 is a flowchart of the process of setting the proportion of intra-frame compressed frames for each tile image, executed by control unit 550 in distribution server 502 in this embodiment. Note that the tile image distribution process does not need to be described in detail here, as it simply involves transmitting encoded data of the requested tile image to the requesting client. The process in Figure 7 starts when distribution server 502 is powered on or reset.

[0045] In step S71, the control unit 550 processes all tile images A input from the imaging device 501. ij The number of selections (i, j = 1, 2, ...) is initialized by setting a predetermined value. The predetermined value here is a positive value such as "10". This is to make the denominator on the right side of the above-mentioned equation (1-2') non-zero.

[0046] Next, the process proceeds to S72, and the control unit 550 controls all tile images A ij Interval I for each intraframe compression intv This initialization may be determined according to the communication bandwidth between the image capture device 501 and the distribution server 500.

[0047] In S73, control unit 550 determines whether or not a command has been received from a client on the network by monitoring command reception unit 510. If a command has been received, control unit 550 advances the process to S74.

[0048] In S74, the control unit 550 judges whether the received command is a command related to the designation of the gaze area. If the control unit 550 judges that the command is not related to the designation of the gaze area, the control unit 550 advances the process to S75, performs a process corresponding to the command, and returns the process to S73 in preparation for receiving the next command. Note that the process of S75 is not directly related to the present embodiment, and therefore a description thereof will be omitted here.

[0049] Now, in S74, if control unit 550 determines that the received command is a command relating to designation of a gaze area, the process proceeds to S76.

[0050] In S76, the control unit 550 controls the image frequency calculation unit 508 to identify a tile image corresponding to the specified new gaze area, and increments the selection function of that tile image by "1". The control unit 550 then advances the process to S77. Here, a method for identifying a tile image corresponding to a new gaze area will be described with reference to FIG. 8. The user specifies a gaze area 801 from an image showing the entire image such as 106. When the gaze area 801 is specified, a gaze circumscribing rectangle 802 is calculated. Now, assuming that the width and height of the gaze circumscribing rectangle 802 are w and h, respectively, and the width and height of the image 106 of layer 1 are W and H, respectively, the respective ratios Rw=w / W and Rh=h / H are calculated. On the other hand, the ratio between the width and height of one tile in layer i and the width and height of the entire image of layer i is Rw i , Rh i Then, Rw j+1 <Rw<=Rwj , Rw k+1 <Rw<=Rw k If the larger of j and k is r, the image quality desired by the user can be guaranteed by selecting the tile image corresponding to the new gaze area from layer r. In addition, the coordinates within the gaze circumscribed rectangle are normalized by the width and height of the image 106 in layer 1, and the maximum and minimum values ​​of the x and y coordinates are calculated as (x max , y max ),(x min ,y min ) for each vertex of each tile s in layer r. The maximum and minimum x and y coordinates are calculated by normalizing the coordinates by the width and height of the image in layer r. max_s , y max_s ),(x min_s ,y min_s ) Then, using the sign function sgn, (sgn(x max -x min_s )+1)*(sgn(x max_s -x min )+1)*(sgn(y max -y min_s )+1)*(sgn(y max_s -y min )+1)>0 becomes the tile image corresponding to the new gaze area.

[0051] In S76, the distribution video determination unit 209 sets information for identifying the tile image of the new gaze area in the video storage unit 512. Until the video storage unit 512 receives the intra-frame compressed encoded data of the tile image of the new gaze area from the video receiving unit 511, the video storage unit 512 outputs the tile image of the old gaze area to the video transmitting unit 513. Then, in response to receiving the intra-frame compressed encoded data of the tile image of the new gaze area from the video receiving unit 511, the video storage unit 512 outputs the encoded data of the tile image of the new gaze area to the video transmitting unit 513.

[0052] Now, in S77, the control unit 550 controls the encoding control unit 507 to calculate the interval at which to perform intra-frame compression on all tile images (the period for performing intra-frame compression). Then, in S78, the control unit 550 controls the encoding control unit 507 to set the interval at which to perform intra-frame compression on each tile image for the encoding unit 505 of the imaging device 501. After this, the control unit 550 returns the process to S73 to prepare for the next command.

[0053] As described above, according to the first embodiment, the number or ratio of intra-frame compression per unit time for all tile images is determined without changing the amount of code per unit time for all tile images. As a result, it is possible to shorten the maximum switching time to an image that is frequently selected as a gaze area. Note that the encoding control unit 507 determines the ratio of intra-frame compressed frames based on the frequency Ω. However, for example, if a system determination unit is further added to the encoding control unit 507 and the system determination unit determines that video distribution is only live distribution, the frequency Ω may be replaced with the number of videos currently being transmitted to the user.

[0054] In short, according to this embodiment, the number of times intra-frame compression is performed per unit time, which is permitted in communication between the imaging device 501 and the distribution server 500, is distributed and set to each tile according to the number of times each tile is designated, which is held in the image frequency calculation unit 508. As a result, the communication bandwidth between the imaging device 501 and the distribution server 500 is not congested, and the interval at which intra-frame compression is used becomes shorter for a gaze area (tile image) that is designated more frequently, thereby shortening the time it takes to switch images.

[0055] [Second embodiment] The second embodiment will be described with reference to FIG. 6. In the second embodiment, the ratio of intra-frame compression is increased not only for an area that is selected relatively frequently by a client user, but also for the surrounding area. As a result, an example will be described in which the response is improved with respect to an operation (panning and tilting) to view the periphery of the area selected by the user. Note that panning and tilting may be performed by the control unit 550 controlling a camera platform (not shown) on which the imaging device 501 is mounted, or the imaging device 501 itself may further include a driving unit to enable panning and tilting operations. It should be understood that this process is performed in S75 of FIG. 7.

[0056] As in the first embodiment, the total number of tiles that undergo intra-frame compression per second is N, and the number of tiles in each layer is m i , the total number of tiles is M. The number of times intraframe compression is performed per second in the i-th layer is n i is determined as shown in the following equation (2-1).

[0057]

number

[0058]

number

[0059]

number

[0060] n distributed within each layer iThe weights are determined assuming a Gaussian distribution, with the most selected area at the center (xt, yt). Figure 6 shows the weight density used to determine the proportion of intra-frame compression to be performed on a set of tile images on a certain layer. The higher the weight density, the higher the proportion of intra-frame compression to be performed. The center (xt, yt) can be found, for example, by the center position of the tile image sent as the most watched area, and the weight of each tile image is determined by integrating the weight density at the boundary of each tile as follows. The weight density w is given by the following equation (2-4).

[0061]

number

[0062]

number

[0063] Here, X1 ij ,X2 ij ,Y1 ij ,Y2 ij is, for example, as shown in the following equations (2-6) to (2-9).

[0064]

number

[0065] Furthermore, the above X ijmin ,X ijmax , y ijmin ,y ijmax is the tile image A of tile division number j of layer i ij These are the minimum and maximum x-coordinate values ​​and the minimum and maximum y-coordinate values ​​in the tile image. Note that the coordinate system can be set with the upper left corner of the tile image before division as the origin, with the rightward direction being the positive direction of the x-axis and the downward direction being the positive direction of the y-axis.

[0066] The above W ij Using tile image A ij The interval for performing intra-frame compression is calculated using the following equation (2-10), and the interval for intra-frame compression of tile images in the area close to (xt, yt) becomes shorter.

[0067]

number

[0068] By determining the proportion of intra-frame compressed frames for each image using the method described above, it is possible to reduce the maximum switching time between the image that is frequently selected as the gaze area and the image of its surrounding area, while suppressing an increase in the amount of code for the entire image.

[0069] [Third embodiment] A third embodiment will now be described. In this third embodiment, the aim is to increase the response of the zoom operation to the tile image selected by the user by weighting the tiles of each layer that include the area corresponding to the most frequently selected gaze area so as to increase the ratio of intra-frame compression. The zoom operation corresponds to changing the distributed image to a layer of tile images with higher resolution that includes the area corresponding to the selected gaze area (zooming in) or to a layer of tile images with lower resolution (zooming out). When the user selects gaze area A, ab is specified, the set T of extracted tile images that includes an area corresponding to area Aab can be defined as shown in the following formula (3-1).

[0070]

number

[0071]

number

[0072] Then, the interval for intra-frame compression of layer k is given by the following formula (3-3), and the closer a layer is to layer a containing the region of interest, the shorter the interval between intra-frame compressed frames can be. In other words, the closer the resolution of a layer is to layer a containing the region of interest, the shorter the interval between intra-frame compressed frames can be. This makes it possible to suppress delays in video distribution when distributing video of a layer whose resolution has been increased or decreased in response to a zoom operation instruction from the user.

[0073]

number

[0074] Here, for simplicity, the weights are the same for the multiple tile images in the kth layer, but as in the second embodiment, the weights may be increased as the tile image is closer to the gaze area. ab The weights of the images that include and do not include satisfy the following equation (3-4).

[0075]

number

[0076]

number

[0077]

number

[0078] By determining the proportion of intra-frame compressed frames for each image using the method described above, it is possible to reduce the maximum switching time between an image that is frequently selected as the focus area and a tile image of a different resolution that contains that area, while suppressing an increase in the amount of code for the entire image.

[0079] (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0080] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0081] 500: imaging system, 501: imaging device, 502: distribution server, 503: imaging unit, 504: image tile division unit, 505: encoding unit, 506: transmission unit, 507: encoding control unit, 508: video frequency calculation unit, 509: distribution video determination unit, 510: command receiving unit, 511: video receiving unit, 512: video storage unit, 513: video transmission unit, 550: control unit

Claims

1. An imaging system including an imaging device and a distribution device that distributes an image captured by the imaging device via a network, The imaging device includes: an imaging means for capturing images at a predetermined frame rate and outputting images with a plurality of resolutions; A division means for dividing each image of each resolution obtained by the imaging means into a preset number of tiles; an encoding means for encoding the image of each tile obtained by the dividing means using intra-frame compression and inter-frame compression, the encoding means being capable of independently setting a ratio of using the intra-frame compression and the inter-frame compression for each tile; an output means for outputting the encoded data of each tile obtained by the encoding means to the distribution device; The distribution device includes: a distribution means for distributing, to a client on a network, encoded data of a tile corresponding to a gaze area requested by the client as a video stream; a storage means for updating and storing the number of times a tile specified by a command specifying a gaze area is specified when the command is received from a client on the network; an encoding control means for determining a ratio of using intra-frame compression for each tile according to the number of times each tile is designated and stored in said storage means, and setting said ratio in said encoding means; have 1. An imaging system comprising:

2. 2. The imaging system according to claim 1, wherein said holding means increases and holds the number of times the tile is designated, the number of pixels which represents the gaze area indicated by said command being the largest.

3. The encoding control means The number of times intra-frame compression is performed per unit time, which is permitted in communication between the imaging device and the distribution device, is distributed to each tile as the number of times intra-frame compression is performed per unit time for each tile according to the designated number stored in the storage means, and set.

3. The imaging system according to claim 1, wherein the imaging device is a digital camera.

4. The encoding control means The weight for each resolution is determined according to the number of times each tile is specified. The closer to the tile having the most number of times of designation in each resolution, the more the number of times of execution of intra-frame compression based on the weight per unit time is set.

3. The imaging system according to claim 1, wherein the imaging device is a digital camera.

5. The encoding control means The resolution and position to which the tile with the maximum designated number of times stored in the storage means belongs are obtained, and the number of executions of intra-frame compression per unit time is set to be larger across a plurality of resolutions, the closer to the center the obtained resolution and position are, with the number of executions being set to be larger.

3. The imaging system according to claim 1, wherein the imaging device is a digital camera.

6. A control method for an imaging system including an imaging device and a distribution device that distributes an image captured by the imaging device via a network, comprising: In the imaging device, an imaging step of imaging at a predetermined frame rate using an imaging means and outputting images with a plurality of resolutions; a division step of dividing each image of each resolution obtained in the imaging step into a preset number of tiles; an encoding step of encoding the image of each tile obtained in the dividing step using intra-frame compression and inter-frame compression, the ratio of using the intra-frame compression and the inter-frame compression being capable of being set independently for each tile; and an output step of outputting the encoded data of each tile obtained in the encoding step to the distribution device, In the distribution device, a delivery step of delivering, to a client on a network, encoded data of tiles corresponding to a gaze area requested by the client as a video stream; a storage step of updating and storing the number of times a tile specified by a command designating a gaze area is designated each time the command is received from a client on the network; an encoding control step of determining a ratio of using intra-frame compression for each tile according to the designated number of times for each tile held in the holding step, and setting the ratio in the encoding step; have A control method comprising:

7. A distribution device controls an imaging device which captures images at a plurality of resolutions at a predetermined frame rate, divides each image into a preset number of tiles, sets a ratio of using intra-frame compression and inter-frame compression independently for each tile, performs intra-frame compression in accordance with the set ratio, and outputs coded data, and distributes the coded data of each tile obtained by the imaging device as a video stream to a client on a network, a distribution means for distributing, to a client on a network, encoded data of a tile corresponding to a gaze area requested by the client as a video stream; a storage means for updating and storing the number of times a tile specified by a command designating a gaze area is designated each time the command is received from a client on the network; a setting means for determining a ratio of using intra-frame compression for each tile according to the designated number of times for each tile stored in the storage means, and setting the imaging device so that each tile is encoded according to the determined ratio; A distribution device comprising:

8. 1. A method for controlling a distribution device, the method comprising: capturing images at a plurality of resolutions at a predetermined frame rate; dividing each image into a preset number of tiles; setting a ratio of using intra-frame compression and inter-frame compression independently for each tile; controlling an imaging device that performs intra-frame compression in accordance with the set ratio and outputs coded data; and distributing the coded data of each tile obtained by the imaging device as a video stream to a client on a network, comprising: a delivery step of delivering, to a client on a network, encoded data of tiles corresponding to a gaze area requested by the client as a video stream; a storage step of updating and storing the number of times a tile specified by a command designating a gaze area is designated each time the command is received from a client on the network; a setting step of determining a ratio of using intra-frame compression for each tile according to the designated number of times for each tile held in the holding step, and setting the imaging device so as to encode each tile according to the ratio; A method for controlling a distribution device, comprising:

9. A program for causing a computer to execute each step of the method for controlling a distribution device according to claim 8 by reading and executing the program.

10. An imaging means for generating image data having a plurality of different resolutions; A division means for dividing the image data of each resolution generated by the imaging means into a plurality of tiles; an encoding means for encoding the image data of each tile divided by the dividing means using intra-frame compression and inter-frame compression to generate encoded data; an acquisition means for acquiring information regarding a ratio of intraframe compression and interframe compression set independently for each tile; an output means for outputting the encoded data of each tile generated by the encoding means performing intra-frame compression according to the set ratio to a distribution device; An imaging device comprising:

Images