Image providing system and control method thereof

Abstract

An image providing system capable of quickly providing a plurality of different cropped images is disclosed. The image providing system includes an acquisition component for acquiring position information related to a user position specified by a user and a control component. The control component controls the image providing system to: determine a range of images captured by an image capturing unit corresponding to the position information as a cropping range before an image capturing unit starts a series of image capturing in which the image capturing unit captures a plurality of images; apply cropping that cuts out a portion of the captured images based on the cropping range after the series of image capturing starts; and provide cropped images in a manner that a user can obtain cropped images to which cropping has been applied during the series of image capturing.

Description

Technical Field

[0001] The present invention relates to an image providing system and a control method of the image providing system, and more particularly, to a technique for capturing and providing images. Background Art

[0002] There is a demand for capturing images of users enjoying sports games or tourist facilities. Patent Document 1 discloses that when a user in a spectator stand inputs a capture request such as a capture range and a capture time, a camera captures an area where spectators are present based on the input capture request.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Laid-Open No. 2007-142896 Summary of the invention

[0006] Problem that the invention aims to solve

[0007] According to the method of Patent Document 1, when the number of images required at one time due to a request from a viewer is greater than the number of cameras, there is a possibility that the image requested by the viewer cannot be provided.

[0008] According to the method of Patent Document 1, when the viewers are located at different locations, it may not be possible to respond to the shooting requests of the respective viewers. On the other hand, if it is attempted to meet the needs of all viewers, the amount of image data may increase, making it impossible to provide images quickly.

[0009] According to the method of Patent Document 1, it may be impossible to provide the target audience with an image in which the audience cannot be specified, such as an image displayed on a screen in a venue.

[0010] Solutions for solving problems

[0011] In view of any of the above problems, the present invention, in one aspect thereof, provides an image providing system or an image providing apparatus capable of providing images to more viewers.

[0012] In view of any of the above problems, the present invention, in one aspect thereof, provides an image providing system or an image providing apparatus capable of quickly providing images to more viewers.

[0013] In view of any of the above problems, the present invention, in one aspect thereof, provides an image providing system or an image providing device capable of providing an image displayed on a screen to an audience.

[0014] According to one aspect of the present invention, an image providing system is provided, comprising: an acquisition component for acquiring position information related to a user position specified by a user; and a control component for controlling the image providing system to: before an image capturing unit starts a series of image capturing in which the image capturing unit captures a plurality of images, determine a range of the image captured by the image capturing unit corresponding to the position information as a cropping range, after starting the series of image capturing, apply cropping based on the cropping range to cut out a portion of the captured image based on the cropping range, and during the series of image capturing, provide the cropped image in a manner in which a user can obtain the cropped image to which cropping has been applied.

[0015] According to another aspect of the present invention, there is provided an image providing system, comprising: an acquisition component for acquiring information related to positions among a plurality of seats to be photographed; an indication component for indicating to an image capturing device an indication related to a photographing interval for image capturing for capturing an image whose field of view includes the plurality of seats; and a control component for controlling the image providing system to change the photographing interval to be indicated to the image capturing device based on the information acquired by the acquisition component.

[0016] According to another aspect of the present invention, there is provided an image providing system, comprising: a display control unit for displaying on a display unit an image of a viewer captured in any of a plurality of seats; an acquisition unit for acquiring information indicating facial information of the viewer and personal information of each viewer; a matching unit for applying facial detection to the viewer displayed on the display unit and matching with the facial information; and a control unit for controlling to provide the image of the viewer displayed on the display unit to a user corresponding to the facial information matched by the matching unit.

[0017] Effects of the Invention

[0018] An advantage of the present invention is that images can be provided to a wider audience.

[0019] An advantage of the present invention is that images can be provided quickly to a larger audience.

[0020] An advantage of the present invention is that images can be provided to a viewer displayed on a screen.

[0021] Other features and advantages of the present invention will become apparent from the following description in conjunction with the accompanying drawings. Note that throughout the accompanying drawings, the same reference numerals represent the same or similar components. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present invention and, together with the description, serve to explain the principles of the present invention.

[0023] Figure 1A is a facility layout diagram in the image providing system according to the present embodiment.

[0024] Figure 1B is a facility layout diagram in the image providing system according to the present embodiment.

[0025] Figure 1C is a facility layout diagram in the image providing system according to the present embodiment.

[0026] Figure 2 FIG. 4 is an overall configuration diagram of the system in the image providing system according to the present embodiment.

[0027] Figure 3 is a block diagram of the image processing apparatus according to the present embodiment.

[0028] Figure 4A : is a diagram showing an example of setting the cutting position for each seat according to the present embodiment.

[0029] Figure 4B : is a diagram showing an example of setting the cutting position for each seat according to the present embodiment.

[0030] Figure 5A : is a diagram showing an example of setting the cutting position for each seat according to the present embodiment.

[0031] Figure 5B : is a diagram showing an example of setting the cutting position for each seat according to the present embodiment.

[0032] Figure 6 is a diagram showing a data structure according to the present embodiment.

[0033] Figure 7 is a flowchart showing the ticket purchase process according to the present embodiment.

[0034] Figure 8 is a flowchart related to pre-match settings according to this embodiment.

[0035] Fig. 9A This is a flowchart related to post-match start processing according to this embodiment.

[0036] Fig. 9B This is a flowchart related to post-match start processing according to this embodiment.

[0037] Figure 9CA This is a flowchart related to post-match start processing according to this embodiment.

[0038] Figure 9CB This is a flowchart related to post-match start processing according to this embodiment.

[0039] Fig.10 is a flowchart showing an example of the cropping process according to the present embodiment.

[0040] Fig.11A is a diagram showing an example of a seat arrangement according to the number of people in a group according to the present embodiment.

[0041] Fig. 11B is a diagram showing an example of a seat arrangement according to the number of people in a group according to the present embodiment.

[0042] Fig. 11C is a diagram showing an example of a seat arrangement according to the number of people in a group according to the present embodiment.

[0043] Fig.12 is a diagram showing a seat relationship in a seat arrangement according to the present embodiment.

[0044] Fig.13 is a diagram of an example of a data structure storing a crop position calculation method according to the present embodiment.

[0045] Fig.14 is a diagram of an example of a data structure storing a crop position calculation method according to the present embodiment.

[0046] Fig.15A is a diagram showing an example of calculating a cropping position according to the present embodiment.

[0047] Fig. 15B is a diagram showing an example of calculating a cropping position according to the present embodiment.

[0048] Fig.16 1 is an overall configuration diagram of a system in the case of providing an image after a game is over according to the present embodiment.

[0049] Fig.17 1 is an overall configuration diagram of a system in the case of providing an image after a game is over according to the present embodiment.

[0050] Fig.18 : is a flowchart showing an example of a series of image capturing processes according to the present embodiment.

[0051] Fig.19 : is a flowchart showing an example of a series of image capturing processes (during a game) according to the present embodiment.

[0052] Fig. 20 is a flowchart showing another example of a series of image capturing processes (during a game) according to the present embodiment.

[0053] Fig.21A is a flowchart showing another example of a series of image capturing processes (during a game) according to the present embodiment.

[0054] Fig.21B is a flowchart showing another example of a series of image capturing processes (during a game) according to the present embodiment.

[0055] Fig. 21C is a flowchart showing another example of a series of image capturing processes (during a game) according to the present embodiment.

[0056] Fig. 22 This is a flowchart related to the pre-registration process of the screen according to the present embodiment.

[0057] Fig.23 This is a flowchart related to screen registrant list creation processing according to the present embodiment.

[0058] Fig.24 It is a flowchart related to the screen image providing process according to this embodiment. DETAILED DESCRIPTION

[0059] Embodiments of the present invention will be described below with reference to the accompanying drawings. The following embodiments do not limit the claimed invention. Although multiple features are described in the embodiments, not all of these features are essential to the present invention. In addition, multiple features can be combined arbitrarily. In addition, in the accompanying drawings, the same reference numerals are assigned to the same or similar components, and their repeated description is omitted.

[0060] Hereinafter, an exemplary embodiment for implementing the present invention will be described in detail with reference to the accompanying drawings. However, the components described in this embodiment are merely illustrative, and are not intended to limit the scope of the present invention to only these components.

[0061] Figures 1A to 1C FIG. 1 is a diagram showing a facility layout for remote control photography in the image providing system according to the present embodiment. Figures 1A to 1C In the present invention, a baseball stadium is used as an example for ease of explanation, but the present invention can be applied to stadiums for hosting sports games, martial arts venues, venues for hosting concerts, theaters for performing dramas and operas, and other facilities that a large number of spectators can watch from a fixed position.

[0062] exist Figures 1A to 1C In the example of the baseball stadium shown, the shooting instruction device 101 instructs the cameras 102 to 105 to simultaneously shoot the infield seats on the first base side. In this embodiment, in order to simplify the shooting, it is assumed that the number of cameras is four, but for the present invention, the number of cameras is not limited as long as it is within the range where shooting instructions can be issued physically at the same time.

[0063] When shooting from the upper part of the opposing spectator stands in a stadium or the like, the distance to the spectators (subject) is long, so even if a telephoto lens with a long focal length is attached, a certain number of spectators will be accommodated in one frame. Therefore, it is necessary to perform a cropping process for each seat and each group. In addition, since a telephoto lens with a long focal length is expensive, when trying to reduce costs, the focal length cannot be increased, so the number of people that fall within one frame increases, and thus the necessity for a cropping process increases. In addition, for example, by accommodating multiple spectators within the angle of view of one frame, images of multiple spectators enjoying a game in one exciting scene can be acquired in one shot. At this time, since the number of people that fall within one frame increases, the need for a cropping process increases.

[0064] Incidentally, regarding the shooting instruction, two methods are provided, namely, interval shooting which is regular shooting (for example, an instruction to shoot an image every 10 seconds, etc.) and manual shooting according to the operator's shooting intention.

[0065] In addition, depending on the facility, a location where it is easier to see an event such as a competition or the like may be set to a higher price, and a location where it is difficult to see the event may be set to a lower price, and therefore, there are generally multiple types of tickets. Figures 1A to 1C Area 1 close to the ground, area 2 with the largest number of seats, and area 3 as a partition space are shown. Area 1 is for spectators who want to watch from a closer place, and since the number of seats is limited, it is an area that is usually watched by a few people such as a family, rather than a unit of dozens of people on special occasions. Therefore, providing memorable images of only members gathered together improves audience satisfaction. In area 2, many fans who frequently watch the game are watching the game, and enthusiastic fans sit in groups. In addition, in area 2, various levels of spectators are included, such as spectators who watch the game as a one-time experience and spectators who are not highly enthusiastic about the game itself. Therefore, it is an area where some spectators want images when they watch the game and some spectators think it is unnecessary, and the audience's satisfaction level is improved by providing images according to the audience's intention. In area 3, customer satisfaction is improved by providing images that are enjoying the game on the spot or specific spectators have good expressions on their faces. As mentioned above, the needs of spectators vary according to the area. In order to meet the needs of all spectators, it is necessary to prepare the captured images according to various modes, but there is also a problem that if the amount of data increases, the cost increases, resulting in the spectator not being able to easily purchase the image. In addition, although the audience's satisfaction level with the game is further improved by providing images while the audience is watching the game, due to the large amount of image data, if an attempt is made to also process the image data of viewers who are unlikely to purchase the images, it will take a long time to process the images, resulting in a long time to provide the images.

[0066] Since it can be determined that viewers in areas 1 and 3 are more likely to purchase the captured image, Figure 1A In , area 1 is covered by two cameras 102 and 103, but in Figure 1B , region 2, which is wider than region 1, is similarly covered by two units of cameras 104 and 105. As a result, the number of people accommodated in the image of region 1 will be smaller, and the audience will appear larger, so the resolution of the cropped image will be higher, thereby enabling a more marketable image to be obtained, which can achieve more purchases. Incidentally, in region 3, by capturing a narrower range than region 1 by two cameras, the resolution of the cropped image is further increased. Since a cropped image with a higher resolution can be acquired in region 3, image analysis of the captured image is also possible, and structurally, in region 3, there are usually a plurality of spaces including a space for viewing, so it is possible to determine that there are more than a predetermined number of audiences in the space for viewing, select a better image by analyzing facial expressions, etc., which makes it possible to obtain a more marketable image than in region 1. Alternatively, in region 1, the audience may not always be in a position that can be captured by the camera. Therefore, using a pan / tilt system capable of camera swinging by remote operation, a plurality of defined spaces are observed by one camera, and when the audience can be captured, a specific space can be captured.

[0067] Figure 1C FIG. 1 shows a configuration diagram for instructing each camera group to perform photography from the photography instruction device 101. The photography instruction device 101 can instruct Figure 1C A plurality of camera groups A to D are shown taking pictures. Camera group C covers a wide area 4 of outfield seats with four units, and camera group D covers a wide area 5 of outfield seats with four units.

[0068] Cameras 102 to 105 are camera units composed of an image capturing element (image capturing sensor) configured by a CCD or CMOS element or the like for converting an optical image into an electrical signal. The camera includes a lens group (capturing lens) including a zoom lens and a focus lens, a shutter having an aperture function, an image capturing element, an A / D converter for converting an analog signal output from the image capturing element into a digital signal, and a protector covering the image capturing system to prevent contamination or damage. The image processing unit in the camera performs a resizing process and a color conversion process such as predetermined pixel interpolation and reduction on the data captured and obtained by the camera. Based on the calculation result obtained by the image processing unit, the CPU in the camera performs exposure control, ranging control, and AWB (auto white balance) processing. The display image data captured by the camera and image-processed by the image processing unit is displayed on the display 04 of the PC, which will be described later. Live view display (LV display) can be realized by D / A conversion of a digital signal obtained by A / D conversion of a signal captured by a camera by an A / D converter and then stored in RAM, and sequentially transmitted to the display 04 for display. Live view can be displayed in the shooting standby state of a still image, the shooting standby state of a moving image, and the recording time of a moving image, and the captured subject image can be displayed almost in real time. The CPU controls the camera and the image processing unit to start operations such as AF (auto focus) processing, AE (auto exposure) processing, or AWB processing in response to a shooting preparation instruction based on a user operation performed in the input device 06. The CPU on the camera side controls to start a series of operations such as performing main exposure, reading out a signal from an image capturing unit element, generating an image file by performing image processing on the captured image by the image processing unit, and then storing the image file in an external storage device in response to a shooting instruction (main shooting). The shooting instruction can be performed by a user operation on the input device 06 of the PC 201a or by pressing a shutter button on the camera side. The camera is capable of capturing both still and moving images.

[0069] Incidentally, depending on the facility, due to the structure, the shooting distance and the illuminance are different for each area of ​​the seat due to the position of the camera that performs shooting, and therefore, in the shooting of the present embodiment, in which many areas are covered in a single shooting instance, there may be a difference in image quality depending on these conditions; therefore, depending on the image quality, the price may be changed or the number of images provided may be increased. In addition, rather than simply setting the image of the seat close to the ground and close to the camera at a high price, there is a case where the seat whose image quality is improved according to conditions such as brightness and the number of cameras is set at a high price.

[0070] Figure 2The image processing device 201 includes the shooting instruction device 101, an image transmission and reception unit 201b, a cropping unit 202, a selection / upload unit 203, and storage devices 204 and 205.

[0071] exist Figure 2 In the embodiment, although the respective processing units 101, 201, 202, and 203 are described in one image processing device 201, they may be physically different processing devices, and the configuration of the processing units is not limited to the present invention.

[0072] The shooting pointing device 101 is Figures 1A to 1C In the same manner as in FIG. 1 , a shooting instruction is simultaneously issued to the cameras 102 to 105. Regarding the shooting instruction, two methods are provided, namely, interval shooting as regular shooting (for example, an instruction to shoot an image every 10 seconds, etc.) and manual shooting according to the shooting intention of the operator.

[0073] The image transmission and reception unit 201b stores image data which is transmitted from the camera side and is an image captured by the cameras 102 to 105 in the storage device 204. FTP (File Transfer Protocol) or the like is used as a protocol for transmitting and receiving image data.

[0074] The cropping unit 202 performs cropping processing on each image data in response to the transmission of the image data from the cameras 102 to 105 to the storage device 204 , and stores the cropped image and the seat number in the storage device 205 .

[0075] Although storage devices 204 and 205 are represented here as separate storage devices, they may be physically identical.

[0076] When the cutting process is performed, the number of people in the pre-purchased group, the seat number and the seat arrangement are obtained from the ticket purchasing unit 206, which will be described later, and based on this information, the cutting process is performed; Fig.10 1 to 15 describe the detailed processing.

[0077] The selection / upload unit 203 uploads the cropped image data, seat number, and event date and time information stored in the storage device 205 to the allocation unit 207 (allocation server). When the shooting instruction device 101 performs interval shooting during a competition or event, a large number of cropped images are created so that the operator can extract and select scenes that the audience is excited about and upload the images to the allocation unit 207. The selection method is not shown, but the object image can be visually selected from the image list, or the data of the corresponding time can be extracted by inputting a specific time, and the selection method is not limited to the present invention.

[0078] In contrast, when the shooting instruction device 101 performs only manual shooting by the operator, shooting instructions are usually performed in scenes that are exciting to the audience, and therefore all images can be uploaded without specific selection.

[0079] The ticket purchasing unit 206 is a mechanism for users to purchase tickets, and will be referred to later. Figure 8 Describes the purchase process. The user enters the date and seat number they want to purchase and makes a payment, obtaining the desired number of seats.

[0080] The allocation unit 207 uses the event date and seat number input by the user as a key to present the image data set of the corresponding seat number of the corresponding date to the user. Regarding the user input, the user can view the ticket and manually input the ticket, or a QR code (registered trademark) or the like can be printed on the ticket and read and recognized by a smart phone, so for the present invention, the input method is not limited.

[0081] The presentation to the user may be a smartphone application, or may be in the form of a web application accessed using a browser, and for the present invention, the presentation method is not limited.

[0082] Figure 3 An example of the configuration of a PC 201a (Personal Computer) in an image processing apparatus 201 as an example of an apparatus to which the present invention can be applied is shown. The image processing apparatus 201 may include a plurality of electronic devices, or may include a single PC.

[0083] exist Figure 3 In the computer system, a CPU 01, a RAM 03, a ROM 02, a display 04, an input device 06, and an I / F 08 are connected to an internal bus 09. The respective units connected to the internal bus 09 can exchange data with each other via the internal bus 09.

[0084] The RAM 03 is composed of, for example, a RAM (volatile memory using a semiconductor device, etc.). For example, the CPU 01 controls each unit of the PC 201a using the RAM 03 as a working memory according to a program stored in the ROM 02. The ROM 02 stores image data, audio data, other data, various programs for the operation of the CPU 01, etc. The ROM 02 is composed of, for example, a hard disk (HD) or a ROM, etc.

[0085] The display 04 displays an image, a GUI screen including a GUI (Graphical User Interface), and the like based on the control of the CPU 01. The CPU 01 controls the respective parts of the PC 201a to generate a display control signal according to a program, generate a video signal to be displayed on the display 04, and output the video signal to the display 04. The display 04 displays a video based on the output video signal. The PC 201a itself may have a configuration up to an interface for outputting a video signal to be displayed on the display 04, and the display 04 may be configured by an external monitor such as a TV or the like.

[0086] The input device 06 is an input device for accepting user operations, including a character information input device such as a keyboard, a pointing device such as a mouse or a touch panel, a button, a dial, a joystick, a touch sensor, a touch pad, etc. Incidentally, the touch panel is an input device configured to be flatly superimposed on the display 04 so that coordinate information corresponding to a contact position is output.

[0087] FIG. 4A to FIG. 5B This is an example of setting the cutting position for each seat according to the present embodiment.

[0088] Figure 4A and Figure 4B is an example of setting the crop position for an image captured by a camera substantially in front of the viewer. Figure 4A As shown, a rectangular cropping position is defined for each seat, and a coordinate system is defined such that the horizontal direction is the X axis and the vertical direction is the Y axis; the coordinates of the upper left vertex position 401 and the lower right vertex position 402 are stored as Figure 6 The data structure shown will be described later.

[0089] By specifying the above operation for each seat included in the captured image, such as Figure 4B The cropping positions of all seats are specified as shown. Although a rectangle is exemplified here as a rectangle, the shape only needs to be represented by a coordinate system, such as using a polygon or an ellipse to define the shape, and the form of the shape is not limited to the present invention.

[0090] Figure 5A and Figure 5B shows the arrangement of the crop position of an image captured by a camera positioned slightly diagonally above the viewer; however, by Figure 4A and Figure 4B As in the example of FIG. 1 , a rectangular cropping position is defined for each seat, and then an X-axis and Y-axis coordinate system is defined. Even if the seat position is skewed, the coordinates of the upper left vertex position 501 and the lower right vertex position 502 can be stored as described later. Figure 6 The data structure in .

[0091] Figure 6600 is an example of a data structure for storing the cropping positions of each seat in this embodiment. The data structure 600 has a row number 601, a seat number 602, a camera number 603, an X coordinate 604 of the upper left vertex coordinate, a Y coordinate 605 of the upper left vertex coordinate, an X coordinate 606 of the lower right vertex coordinate, and a Y coordinate 607 of the lower right vertex coordinate.

[0092] In the data structure 600, the X and Y coordinates are managed as separate columns, but may be managed as list information such as (X, Y) in a single column. Figure 6 607 is digital data, but character string data such as letters etc. may be used. Digital data indicating the pixel position is stored in coordinate information 604 to 607.

[0093] In this embodiment, a description will be given of providing an image by taking a photo and cutting out an image of the audience who are watching a baseball game as a user. However, the example of a baseball game described in this embodiment is an example, and the present invention can be applied to other sports games or places such as tourist spots and performances. The present invention can also be applied to places of entertainment programs including concerts, or to a case where the audience who are watching the entertainment program is set as a user.

[0094] Reference Figure 7 The ticket purchase process in this embodiment is described.

[0095] Figure 7 The process when a ticket is purchased in an application or an Internet site where a user purchases a ticket is shown. This process is realized by the CPU 01 deploying a program stored in the ROM 02 (non-volatile memory) in the RAM 03 (system memory) and then executing the program. Although the process can be started independently regardless of the state of the camera or the operation unit, the process can be started when the camera is powered on and can accept a user operation on the operation unit.

[0096] In step S701, the CPU 01 accepts input of the number of tickets to be purchased from the user.

[0097] In step S702, the CPU 01 accepts input from the user as to whether they intend to purchase the image. Regarding the input in step S702, a configuration may be adopted such that the user can directly select whether to purchase the image on the purchase screen, or the ticket and image purchases are in a set, and the ticket purchase and image purchase can be selected at once. In addition, some seats may be purchased in a set with the image without the user making a selection, and it may be assumed that the user has an image purchase intention when the user has selected the object seat.

[0098] In step S703, the CPU 01 determines whether there is an image purchase intention in the input of step S702. If it is determined that the user has an image purchase intention, the process proceeds to step S704; otherwise, the process proceeds to step S706.

[0099] In step S704, the CPU 01 selects a seat designated by the user. At this time, it may be configured so that by selecting a representative seat by the user, a plurality of modes of seat positions corresponding to the purchase quantity input in step S701 are presented so that the user can easily select a seat. Fig.10 In the described image cropping process, the cropping range is changed according to the user's designation of the seat position.

[0100] In step S705, CPU 01 generates group information based on the information received in step S704. Group information is information that associates representative seats with the seat positions of the tickets purchased and inputted at the same time in step S701. Even if there is one user, in step S705, this information is generated in association with the seats as group information. The group information may be such that other seats may be added to the same group by inputting the information described on the tickets purchased at the same time via an application or an Internet site. However, preferably, the seats to be grouped are adjacent to each other vertically or horizontally. This is because seats that are far away from each other may be photographed by different cameras, so when cropping, the seats may not be accommodated within a single cropping range. In addition, if the cropped image contains many other seats that are not included in the group, the size of each person becomes smaller.

[0101] In step S706, the CPU 01 selects the seat designated by the user.

[0102] Next, we will refer to Figure 8 The pre-match setup process in the present embodiment is described. This process indicates the process performed by the image processing device 201 before the start of the match. This process is started when the image processing device 201 is powered on before the match and an instruction to start preparing for the target match is given from the operator of the system, or at a predetermined time before the match (such as three hours before the match or the day before). This process is realized by the CPU 01 deploying a program stored in the ROM 02 (non-volatile memory) in the RAM 03 (system memory) and then executing the program.

[0103] In step S801 , the CPU 01 connects to the allocating unit 207 and the ticket purchasing unit 206 .

[0104] In step S802, the CPU 01 acquires ticket information of the target match from the ticket purchase unit 206. The ticket information includes information about the number of purchased tickets, seat position information, and seat arrangement of the group purchased at the same time, which is necessary for the cropping process. In addition, in order to select a more marketable image, information about the seat type, information about the area (such as whether the seat is inside the venue, outside the venue, or in the VIP, etc.), information about whether the seat is in the shooting target area, information about whether the seat is unspecified or specified, information about whether the user is a frequent spectator, information about whether the user is in a fan club, etc. are also included.

[0105] In step S803, the CPU 01 calculates a cropping range based on the information acquired in step S802. Using the number of people and the arrangement pattern of the group acquired in step S802, the CPU 01 acquires the cropping range for the group. Fig.13 and Fig.14 The calculation method of the cropping position of each number of people and mode defined in (to be described later) is used to calculate the cropping range. Here, the cropping range refers to a rectangle enclosed in two vertices, namely the X coordinate and Y coordinate of the upper left vertex position of the rectangle to be cropped and the X coordinate and Y coordinate of the lower right vertex position of the rectangle. Although a rectangle defined by two vertices is used here, the rectangle only needs to be represented by a coordinate system, such as using a polygon or an ellipse to define the rectangle, and for the present invention, the form of the rectangle is not limited.

[0106] Fig.13 and 14 is a diagram of an example of a data structure storing a clipping range calculation method according to the present embodiment. Fig.13 Is for Figure 4A and Figure 4B An example of a definition of an image taken from substantially in front of a viewer in Fig.14 Is for Figure 5A and Figure 5B An example of the definition of an image in which the viewer is photographed slightly diagonally.

[0107] Fig.13 and 14 The number of people in the management group 1301, the seat arrangement pattern 1302, the upper left vertex position X coordinate calculation method 1303, the upper left vertex position Y coordinate calculation method 1304, the lower right vertex position X coordinate calculation method 1305, the lower right vertex position Y coordinate calculation method 1306 and the out-of-group seat information 1307.

[0108] The number of people in the group 1301 and the seat arrangement pattern 1302 are consistent with the number of people in the purchased group and the seat arrangement obtained in advance from the ticket purchasing unit 206. Fig.13 and 14In the example of , coordinate calculation methods 1303 to 1306 use the cropped position information of any seat position in the group, but a specific number of pixels or calculation formula may be defined.

[0109] The out-of-group seat information 1307 defines information of seat positions that fall within a rectangle in the seat arrangement pattern but are not purchased as seats in the group.

[0110] In step S804, the CPU 01 prepares facial data. As facial data, facial data of a user who is a frequent spectator (i.e., a spectator whose face is detected many times in the past or a spectator in a fan club) is prepared based on the information acquired in step S802. As a result, when there is a spectator determined to be consistent with the face in the facial data in the captured image, the image of the specific spectator can be associated and collected as data, and provided to the specific spectator together. In addition, with regard to the spectators in the fan club, even if they do not indicate the image purchase intention every time they buy tickets, the images are automatically collected so that they can be reviewed together after the end of the season. Depending on the spectators, the frequency of watching the game varies greatly, so in one game, there may be spectators who want multiple images, and there may be spectators who do not mind obtaining one image per game, but want images from all the games they watch. Therefore, by preparing facial data as described above, images can be provided without the frequent spectators having to perform an operation for indicating the image purchase intention every time. In addition, images can be provided even if the spectators forget to indicate the image purchase intention every time. In addition, images of specific spectators can be collected even if the spectators sit in non-designated seats whose seat positions are unknown. In addition, a plan for the audience to purchase images can be provided separately from the purchase of game tickets (rather than the information when purchasing tickets acquired in step S802), and when the plan is selected, facial data can be registered so that the state of the audience watching the game can be provided as an image each time. In the above case, by preparing facial data, images can be smoothly provided to the audience during the game.

[0111] In step S805 , the CPU 01 sets the clocks of the respective cameras to the same time so that the capturing times of the images captured by the cameras 102 to 105 coincide with each other.

[0112] In step S806, CPU 01 inputs a shooting trigger. The shooting trigger may be an interval shooting period such as 10 seconds or 20 seconds, or based on a detection such as a response to the cheering of the audience becoming equal to or greater than a predetermined value. In addition, a control unit for analyzing the hit status of the game or analyzing the game data may be provided, and the interval may be shortened or switched to motion image shooting according to the game score, the progress of the game, and the position of the athlete. In addition, interval shooting may be stopped during the break time, and interval shooting may be started when activities such as cheerleading or question-and-answer sessions occur during the break time. Needless to say, manual shooting instructions may be combined.

[0113] In step S807, the CPU 01 actually starts the camera and performs shooting and cropping processing as a system test. The shooting conditions that are consistent with the brightness and light entry degree of the day can be set based on the captured image, and it can be confirmed whether the target seat can be cut out when the captured image is cropped based on the coordinates associated with the seat. By the way, by placing the subject used as an index point on the seat side, it can be confirmed whether the subject in the image captured by the camera is at the predetermined coordinates, whether there is no problem with the focus position, etc. In addition, since the system is operated for each activity, the data of the previous activity can be retained in some cases, so the deletion of the camera data and the organization, deletion and backup of the data in the system can be performed at this timing.

[0114] In step S808, the CPU 01 sets shooting conditions based on the system test in step S807.

[0115] Next, use Figures 9A to 9CB This shows the process after the game starts. Fig. 9A is the processing of the image processing device 201, Fig. 9B is processing related to shooting, and Figure 9CA and 9CB It is a process for cutting and distributing images. Each process starts before the start of a game or before the start of pre-game shooting. This process is realized by the CPU 01 deploying a program stored in the ROM 02 (non-volatile memory) in the RAM 03 (system memory) and then executing the program.

[0116] In step S901, the CPU 01 obtains the in-game ticket information. The in-game ticket information is related to the tickets purchased after the game starts or Figure 8 The information related to the ticket purchased after the process of step S802 can be obtained from the ticket purchasing unit 206, or the image processing device 201 can be notified from the ticket purchasing unit 206 side periodically or every time there is an update.

[0117] In step S902, the CPU 01 obtains information about the camera corresponding to camera c = 1. In step S902, information about the photographing range of camera c = 1 and the subject seat is obtained.

[0118] In step S903, the CPU 01 determines whether the camera c to be processed now is greater than the total number of cameras K, and if it is greater, it is assumed that the processing has been completed for all cameras, and the processing proceeds to step S907. Otherwise, the processing proceeds to step S904.

[0119] In step S904, the CPU 01 acquires cropping information of the seat range to be photographed from the position of the target camera c.

[0120] In step S905, CPU 01 obtains the shooting trigger of the range of seats to be photographed from the position of the object camera c. For example, when the A team and the B team play against each other, it may be better to change the shooting interval period between the attack and defense timing. Depending on the type of seat, it may be better to shoot at a higher frequency. Therefore, by obtaining the shooting trigger according to the seats included in the shooting range of the camera, it is possible not to miss the shooting opportunity without shooting a large number of images that are very unlikely to be provided to the audience. Note that the shooting trigger of step S905 can be the same for all cameras.

[0121] If a new ticket has been purchased, or if the viewer has indicated a new image purchase intention, that is, if a new crop group has been generated, then in steps S904 and S905 Figure 8 The processing of steps S803 and S804.

[0122] In step S906 , similarly to step S902 , the CPU 01 sets camera c=c+1, and then acquires information on the next camera.

[0123] In step S907, the CPU 01 determines whether a predetermined time has elapsed. Fig. 9A In the case of the first judgment of step S907 after the start of the processing, the predetermined time is measured according to the time from step S901, and for the second and subsequent judgments, the predetermined time is measured according to the time from the time when the previous judgment of step S907 was judged as "yes". Step S907 is used to obtain the in-game ticket information at predetermined time intervals. If the judgment in step S907 is "yes", the processing proceeds to step S901; otherwise, the processing proceeds to step S908.

[0124] In step S908, the CPU 01 determines whether there is a photographing trigger that has been acquired in step S905. If the determination in step S908 is "YES", the process proceeds to step S909; otherwise, the process proceeds to step S907.

[0125] In step S909, the CPU 01 notifies the camera side of a shooting instruction.

[0126] In step S910, the CPU 01 notifies the cropping information to the cropping unit 202. Since the information on the cropping information and the shooting trigger is updated every predetermined amount of time, a spectator who purchases a ticket in the middle of the game can also obtain a cropped image starting in the middle of the game.

[0127] In step S911, the CPU 01 determines whether to end the process, that is, whether the game is over and whether the shooting is over. To end the process, the CPU 01 proceeds to step S912, otherwise proceeds to step S901.

[0128] In step S912, the CPU 01 notifies the camera of power off.

[0129] As mentioned above, in Fig. 9A In the embodiment, since the cropping information is sent to the cropping unit 202 substantially simultaneously with the shooting instruction, the registered cropping process can be performed quickly after shooting. When the image is sent from the camera side to the storage device 204, the cropping unit 202 has already identified the group to be cropped and calculated the cropping range, so that the cropping process can be performed quickly and the viewer can confirm the image.

[0130] Next, we will describe Fig. 9B Shooting related processing. Fig. 9B The processing is performed by each camera.

[0131] In step S930, the CPU 01 sets the camera side to acquire shooting conditions.

[0132] In step S931, it is determined whether a shooting instruction has been issued on the camera side, and if so, the process advances to step S932, and if not, waits for a shooting instruction.

[0133] In step S932, photographing is performed on the camera side under the set photographing conditions.

[0134] In step S933, the CPU 01 controls the image to be transmitted from the camera to the image transmission and reception unit 201b, and stores the transmitted image data in the storage device 204. Alternatively, the images captured by the camera are sequentially transmitted to the storage device 204.

[0135] In step S934, the CPU 01 determines whether to end Fig. 9B When Fig. 9A If an instruction to turn off the camera power is received in step S912, the determination in step S934 is "Yes". Fig. 9B When processing Fig. 9B The processing is completed, otherwise, the processing enters step S931.

[0136] Next, we will refer to Figure 9CA and 9CB Describes the process of cropping and distributing images.

[0137] In step S951, the CPU 01 determines whether an instruction to acquire an image has been given. The instruction to acquire an image is based on the Figure 8 The CPU 01 acquires an instruction to capture images taken around the time of an event (such as, in the case of baseball, the time when a home run occurs or the time when a wonderful play occurs) according to the clock time set in step S805. The image acquisition instruction is performed according to the occurrence of the event time. It should be noted that, for example, the CPU 01 can analyze the game even when it is not the event time, and when the event occurs, the image capture instruction and the image acquisition instruction are performed at the same time.

[0138] In step S952 , the CPU 01 acquires the target image from the storage device 204 .

[0139] In step S953 , the cropping unit 202 acquires cropping information.

[0140] In step S954 and subsequent processing, the trimming process is started.

[0141] In step S954, the cropping unit 202 acquires information of seats of n=1. Then, it is assumed that the number of seats in the shooting range of the camera that has now captured the image to be cropped is m. In the present embodiment, for simplicity, it is assumed that n=1 to m are seat numbers, but the present invention is not limited thereto, and if all seats captured by one camera and set as cropping objects are covered, these seats can be linked to seats actually used in the stadium. In order to cover the audience seats with multiple cameras, the shooting range of the camera can overlap with the shooting range of another camera. Even if the audience is captured by two cameras, an image cropped from an image captured by one of the cameras is provided. Therefore, the shooting range and the cropping range of the camera may be different. In step S954, the number of seats in all cropping ranges is acquired as m. The cropping range may be changed during the game, or may be predetermined on the stadium side.

[0142] In step S955, CPU 01 determines whether the seat is in the area to be cropped. In the stadium, there is a possibility that the camera's blind spot is caused by the net, poles, roof, etc., or there is an area where shooting is prohibited. In addition, in areas where the position of the audience is unknown (such as non-designated seats, etc.), cropping corresponding to the position of the seat is not performed first. Therefore, the judgment of step S955 is "no". As will be described later, even in non-designated seats, cropping by face detection or cropping of a wider range can be performed. If it is judged as "yes" in step S955, the processing enters step S956; otherwise, the processing enters step S963. It should be noted that it is also possible to indicate to the audience when purchasing tickets or near the seat that the area is an area where images cannot be purchased.

[0143] In step S956, the CPU 01 determines whether the seat number n currently being determined as the cutting object is smaller than the seat number m within the cutting range. In other words, it is confirmed whether the determination of whether to cut has been completed for all seats in the cutting range. If the determination in step S956 is "yes", the process proceeds to step S957; otherwise, the process proceeds to step S965.

[0144] In step S957, the CPU 01 determines whether seat n is a cutting target seat. Figure 7 Indicates the image purchase intention to determine whether the seat is a cropping object. That is, when the seat is included in the cropping group, the judgment of step S955 is "yes", otherwise it is "no".

[0145] In step S958, the CPU 01 selects a cropped image as the original image to be cropped from the image acquired in step S952. With regard to the selection of the cropped image, an image having the best facial expression or greater movement compared to the previous image and the subsequent image may be selected according to the state of the audience of the cropped group of seat n currently being the target. Alternatively, the selected image may not be changed for each cropped group, and the image may be manually selected, or the movement of the audience within the cropped range may be quantified, and then the image that appears to have the greatest movement may be selected.

[0146] In step S959, the CPU 01 performs a trimming process. Fig.10 Describes the clipping process.

[0147] In step S960, the CPU 01 sets seat n = n + 1. When seat n is included in the cropping group that has been subjected to the cropping process, n + 1 is performed again to search for a seat that has not been subjected to the judgment of step S957.

[0148] In steps S961 and S962, the processing when the audience is not in the group cropping of the audience at seat n will be described. In the present embodiment, the area associated with the seat number is cropped so that the audience can obtain the image after the game. That is, instead of determining the cropping range to include the audience who makes purchases at the same time and excluding other audiences as in the group cropping, the predetermined number of people (such as 30 or 50 people, etc.) is cropped. As a result, the time and data amount required for cropping can be reduced, and the image can be quickly provided to the audience who is the group cropping object, and the image can be provided to the audience who is not the group cropping object later.

[0149] In step S961, similar to step S958, the CPU 01 selects a cropped image. In the case of regional cropping, since cropped images are not stored according to the audience of each seat, the selection of a cropped image corresponding to the audience of seat n is not performed. In the case of regional cropping, the image can be manually selected, or the movement of the audience in the cropping range can be quantified, and then the image with the largest movement in the numerical value can be selected. Alternatively, an image with a good facial expression or a large movement of the audience whose facial data is prepared in advance can be selected. As described above, even if the audience is not the group cropping target, by selecting an image with a good facial expression or a large movement of the audience who may purchase the image, a larger number of audiences who will purchase the image in the future can be satisfied. If images with good facial expressions are stored for each audience, the amount of data increases, resulting in an increase in cost and time spent in retrieving data. By preferentially selecting images for frequent audiences who may make purchases, images can be provided faster and at a lower cost. Note that the image of the target audience whose face has been detected can be notified to the audience together every predetermined number of games (such as 10 games or 5 games, etc.), or the image can be notified to the audience when watching the next game and the audience can be prompted to purchase the image. Furthermore, by presenting the audience with images captured in the previous game when purchasing tickets for the next game, the audience can grasp what type of images will be captured, making it easier to determine whether to purchase the images.

[0150] In step S962, the CPU 01 performs area cropping. As described above, area cropping crops images separated by a predetermined number of people. When seat n is already included in an image that has been area cropped in processing for another seat n, area cropping is no longer performed, and the image data amount is reduced.

[0151] In steps S963 and S964, the same processing as step S961 and step S962 is performed. If the seat is a non-designated seat that is not a cropping target seat, processing based on face detection can be performed, and if the seat is a cropping target seat and the audience is not within the group cropping range, there is no need to perform regional cropping. In addition, with regard to seats where some enthusiastic fans sit together or sit in a group, the regional cropping range can be set wider. For example, users wearing clothes of the same color can be set as the cropping range. This makes it possible to accommodate a group wearing the uniforms of the team they support in one image.

[0152] In step S965, the CPU 01 determines whether there is face data in the cropped image. If there is face data, the process proceeds to step S966, otherwise, to step S967.

[0153] In step S966, the CPU 01 associates the ID of each spectator with the image, and sets the image flag to ON for the match the spectator is watching. If the image flag is ON, the image taken in which match can be presented to the spectator.

[0154] In step S967, the CPU 01 uploads the cropped image to the distribution unit 207 so that the audience who is the group cropping target can view the image. Note that the cropped image can be presented even if the audience is not the group cropping target. At this time, for the audience who is not the group cropping target, characters such as "sample" can be added as a watermark. The image data that has not been uploaded is deleted from the storage device 204 or stored with reduced image quality, thereby reducing the size of the image, which reduces the cost. However, the image data may be stored for a predetermined period of time after the game together with the image itself selected in steps S958, S961, and S963, or the individual audiences may be notified before deletion.

[0155] Fig.10 01 is a flowchart showing an example of the cropping process according to the present embodiment. This process is realized by the CPU 01 deploying a program stored in the ROM 02 (non-volatile memory) in the RAM 03 (system memory) and then executing the program.

[0156] The CPU 01 reads out the number of persons in the pre-purchased group, the seat numbers, and the seat arrangement acquired from the ticket purchasing unit 206 (step S1001).

[0157] FIG. 11A to FIG. 11C is an example of the seating arrangement according to the number of people in a group according to the present embodiment. Fig.11Ais an example of a seating arrangement for a group of four people; in pattern (1), four people are arranged side by side, and in pattern (2), they are arranged with two people in front and behind, respectively. The patterns are merely examples, and the combination of the shape and number of people in the pattern is not intended to limit the present invention.

[0158] Similarly, Fig. 11B is an example of a seating arrangement for a group of five people, and Fig. 11C is an example of a seating arrangement for a group of six.

[0159] Fig.12 : is a diagram showing the seat relationship in the seat arrangement according to the present embodiment. Using the lower left seat position in the group as a reference, the horizontal seat position is represented as "number" and the vertical seat position is represented as "row", and the numbers of "number" increase toward the right side in the horizontal direction, and the numbers of "row" increase toward the upper side in the vertical direction.

[0160] Fig.12 Show Fig. 11B A specific example of the seating arrangement in the mode (2) of FIG. Although there is no user of the group in the number X+2 / row Y, it is required as data when cropping the rectangle, so in the later described Fig.13 and 14 The data structure is defined as out-of-group seats.

[0161] Description Back to Fig.10 Description.

[0162] The CPU 01 acquires the trimming range and the out-of-group seat information (step S1002). Note that in step S1001, the trimming range may be calculated without calculating the trimming range in advance.

[0163] When there is out-of-group seat information in the group information acquired in step S1002 ("Yes" in step S1003), the CPU 01 fills the cropping range of the seat position defined as the out-of-group seat with a single gray color (step S1004). When there is no out-of-group seat information ("No" in step S1003), the processing of step S1004 is skipped.

[0164] In step S1004, if the entire area of ​​the crop position of the out-group seat is filled, it may overlap with the crop position of the in-group seat adjacent to the upper and lower sides, so it is desirable to slightly fill the inside of the crop position. This is processing for preventing the audience outside the group from being photographed, but for the present invention, the filling method is not limited, and if there is no specific problem of portrait rights, etc. and the user's intention is that there is no problem even if people outside the group are photographed, the processing of step S1004 may not be required.

[0165] The CPU 01 determines the trimming position according to the method of calculating the trimming position from the reference numerals 1303 to 1306 acquired in step S1002 (step S1005 ).

[0166] Fig.15A and Fig. 15B is a diagram illustrating an example of calculation of a cropping position according to the present embodiment. Fig.15A Yes Figure 4A and Figure 4B An example of a method for calculating an image taken from approximately in front of a viewer, and Fig. 15B Yes Figure 5A and Figure 5B An example of a computational method for capturing an image of a viewer slightly diagonally in . Fig.15A In , the upper left vertex of the upper left seat and the lower right vertex of the lower right seat are simply used as vertices for the entire clipping position; however, in Fig. 15B Since the upper and lower rows are slightly skewed, we define:

[0167] Top left vertex

[0168] X coordinate = X coordinate of the upper left vertex of the lower left seat

[0169] Y coordinate = Y coordinate of the upper left vertex of the upper left seat

[0170] Bottom right vertex

[0171] X coordinate = X coordinate of the lower right vertex of the upper right seat

[0172] Y coordinate = Y coordinate of the lower right vertex of the lower right seat

[0173] And use this calculation method to determine the coordinate positions of the upper left vertex and the lower right vertex.

[0174] Description Back to Fig.10 Description.

[0175] Based on the cropping position determined in step S1005, CPU 01 generates a cropped image cut out at the cropping position according to the corresponding number of people and seat arrangement of the entire image captured by the camera, and stores the cropped image together with the shooting date / seat number information in the storage device 205 (step S1006).

[0176] In the above-described embodiment, the system may operate as described below so that the viewer can acquire images more easily.

[0177] - Set up a browse request for the distribution server to read the ticket QR code (registered trademark).

[0178] -Although the configuration is such that group information is acquired in advance, the group information may be acquired each time processing is performed. In this case, since the ticket purchasing system also needs to always regenerate group information, and this is acquired from the clipping unit 202 each time, the processing of the entire system increases; however, this is effective in the case of baseball, for example, where the percentage of same-day ticket sales is large and a large amount of group information is generated after the start of the game.

[0179] Next, we will refer to Fig.16 and 17 Describes situations where a viewer does not indicate an intent to purchase an image beforehand, but attempts to purchase the image after the contest is over.

[0180] Fig.16 The image processing device 201 includes the shooting instruction device 101, the cameras 102 to 105, the image transmission and reception unit 201b, the selection / upload unit 203, and the storage device 204.

[0181] exist Fig.16 In the embodiment, although the processing units 101, 201b, 203 and 204 are described as being in one image processing device 201, they may be physically different processing devices, and the configuration of the processing units is not limited to the present invention. Similarly, the allocating unit 207 and the cropping unit 202 may be physically separate processing devices.

[0182] Since the photographing processing device 101, the cameras 102 to 105, the camera image transmission and reception unit 201b and the ticket purchasing unit 206 are Figure 2 , so their description will be omitted.

[0183] The selection / upload unit 203 uploads the captured image data to the allocation unit 207 together with the date and time information of the event triggered by the captured image data and the cut position information of each seat stored in the storage device 204 .

[0184] Since the image data captured here is high-resolution data that is not cropped, when the number of captured images increases due to interval shooting, etc., or when the number of cameras increases, a large amount of data will be uploaded to the allocation unit 207, and when group cropping is attempted later, the system must retain a larger amount of data.

[0185] Here, the cropping position information of each seat is uploaded from the image processing device 201 to the allocation unit 207, but the allocation unit 207 and the cropping unit 202 may have the information. However, since the cropping position information is only position information about the image taken by the cameras 102 to 105, it is necessary to update it every time the installation position and the viewing angle of the cameras 102 to 105 on the image processing device 201 side are changed, and therefore it is necessary for the system of the image processing device 201, the allocation unit 207, and the cropping unit 202 to process the cropping position information in a more cooperative manner.

[0186] The allocation unit 207 sends the thumbnail position information and seat number acquired from the selection / upload unit to the cropping unit 202 using the event date and seat number input by the user as a key, instructs cropping processing, and presents the acquired cropped image to the user.

[0187] Regarding user input, something like Figure 2 , the user can view the ticket and manually enter the ticket, a QR code (registered trademark) etc. can be printed on the ticket or seat and read and recognized using a smart phone, or if the ticket information is unknown, facial detection can be performed, so the input method is not limited to the above methods.

[0188] The presentation to the user may be a smartphone application, or may be in the form of a web application accessed using a browser, and for the present invention, the presentation method is not limited.

[0189] While the processing itself is similar to Figure 2 , but each time the user wishes to make a purchase, the cropping unit 202 will generate a cropped image, and since the cropping time is included in the time until the user acquires the cropped image, it is not possible to quickly provide the image. In addition, in order to quickly perform image processing, an environment capable of performing image processing at a higher speed is required.

[0190] Next, in Fig.17 In the following, the case where image data is stored in the storage device 204 will be described. Fig.17 In the processing of Fig.16 Unlike the case where a large amount of image data is not sent to the distribution unit 207, since the image processing device 201 requests and then uploads the image captured by the subject camera after receiving the image purchase trigger from the user, it takes more time than Fig.16 The time in increases the length of time it takes to acquire the image, so the user spends more time acquiring the cropped image data.

[0191] Fig.17 With Fig.16 Same configuration.

[0192] In the present embodiment, assuming that the user has already purchased a ticket, the process starts when an instruction to purchase an image is issued to the allocation unit 207 (step 1).

[0193] The distribution unit 207 requests the object camera image from the image processing device 201 (step 2), and in the image processing device 201, requests the image from the storage device 204 (step 3). Thereafter, the selection / upload unit 203 sends the image data to the distribution unit 207 (step 4). At this time, since the image data includes date information (shooting date and time), cropping position information of each seat, and image data before cropping, the amount of data increases. When a cropping instruction is issued from the distribution unit 207 to the cropping unit 202 (step 5), the cropping unit 202 performs cropping, and distributes the cropped image data to the distribution unit 207 (step 6). Thereafter, the cropped image is distributed from the distribution unit to the audience who purchased the image (step 7).

[0194] As mentioned above, in Fig.17 In the method, even if the viewer has not indicated the image purchase intention in advance, the image cropped for the crop group can be provided to the viewer. However, there are many exchanges in the system and equipment, and it may take more time to provide the image than for the viewer who wants to purchase the image in advance.

[0195] When the image is provided to the audience, the seat information is also linked and provided to the audience so that when the audience uploads the image to a social networking site (hereinafter referred to as SNS), people who see the image can recognize at which seat position a good image can be taken.

[0196] In addition, by performing image processing related to brightness according to the seat position to provide appropriate images for dark and bright parts, better images can be provided even if a wide range of viewers including a plurality of seats are included in the same viewing angle. In the case where the brightness change in the same viewing angle exceeds a predetermined amount, it is easier to obtain an original image and then process the image later.

[0197] Next, we will refer to Fig.18 A series of image capture processing in the present embodiment is described. A series of image capture processing is a processing for changing the capture frequency according to the vacancy state of the audience stand. This processing is implemented by CPU 01 deploying a program stored in ROM 02 (non-volatile memory) in RAM 03 (system memory) and then executing the program. Since it is preferable to collect more audience information before the start of the game, the processing is started at a predetermined time before the game, such as 10 minutes or 1 hour before the start of the game. Alternatively, the processing can be started immediately after the start of the game.

[0198] In step S1801, the CPU 01 acquires an image when a seat is vacant within the photographing range of each camera. As for the image when a seat is vacant within the photographing range, a pre-photographed image may be acquired, or the image may be photographed for each game.

[0199] In step S1802, the CPU 01 obtains information about viewers who intend to purchase images in advance. The viewers who intend to purchase images are Figure 7 The ticket purchase process shown is for viewers who have indicated their intention to purchase the cropped image.

[0200] In step S1803, the CPU 01 acquires information about the arrangement of the cameras and acquires information about the seats within the photographing range of the cameras. The seat type is also acquired at this time.

[0201] In step S1804, the CPU 01 calculates the shooting subject ratio of the number of viewers who are shooting subjects based on the information acquired in step S1803 and the audience information acquired in step S1802. The shooting subject ratio is obtained by dividing the number of viewers who intend to purchase images by the number of seats. In step S1804, the shooting subject ratio of each camera is calculated.

[0202] In step S1805 , the CPU 01 acquires the photographic subject ratio of camera n=1 calculated in step S1804 .

[0203] In step S1806, the CPU 01 determines whether there is a camera n photographing subject, that is, a viewer who intends to purchase images within the photographing range of camera n. If it is determined that there is a camera n photographing subject, the process proceeds to step S1807; otherwise, the process proceeds to step S1809.

[0204] In step S1807, the CPU 01 determines the shooting interval based on the calculated shooting subject ratio. For example, if the shooting subject ratio of the audience who intends to purchase the image is 50% or more, the CPU 01 sets the shooting interval to T = 5 seconds, and if it is 30% or more, the CPU 01 sets the shooting interval to T = 10 seconds. If the number of shots increases, the number of images stored in the storage device 204 increases, and there is a possibility that the speed of providing images to the audience may decrease. Therefore, by determining the number of images to be stored in the storage device 204 (i.e., the number of images to be shot) relative to the number of images that may be provided to the audience, the speed of providing images can be prevented from decreasing.

[0205] In step S1808, the CPU 01 sets the photographing interval T so that the camera n performs photographing at the set photographing interval.

[0206] In step S1809, the CPU 01 changes the shooting setting of the camera n. When there is no shooting object, for example, the camera n can be used as a camera for shooting the game scene (instead of shooting the audience), or can be used as a camera for shooting fireworks or scoreboards. Alternatively, shooting is not required. This can reduce the number of images stored in the storage device 204.

[0207] In step S1810, the CPU 01 determines whether the setting of step S1808 or S1809 has been made for all cameras. If it is determined that the shooting setting for all cameras has been made, the process proceeds to step S1811, otherwise, the shooting subject ratio of camera n=n+1 is obtained, and the process proceeds to step S1806.

[0208] In step S1811, the CPU 01 executes a series of image capturing processes (during the game). Fig.19 Describes a series of image capture processes (during a game).

[0209] Next, we will refer to Fig.19 Describes a series of image capture processes (during the game). When the process is completed Fig.18 In step S1811, Fig.19 Even if it is not during the game, the shooting interval of each camera n can be set in step S1808. Fig.19 The processing shown (step S1809). Fig.19 It is used to update the previous Fig.18 The processing of the shooting interval set in . Before the game, the information when the ticket was purchased is known, but after the game starts, it is known how many spectators are actually in the seats. This processing is realized by the CPU 01 deploying the program stored in the ROM 02 (non-volatile memory) in the RAM 03 (system memory) and then executing the program.

[0210] In step S1901, the CPU 01 acquires camera arrangement information, shooting range information, and viewer information intending to purchase images regarding camera n=1.

[0211] In step S1902, the CPU 01 acquires the image captured by camera n immediately before.

[0212] In step S1903, the CPU 01 compares the image at the time of seat vacancy acquired in step S1801 when the seat is vacant with the image acquired in step S1902, and determines the area where the audience is seated.

[0213] In step S1904, the CPU 01 calculates the ratio of the human region.

[0214] In step S1905, the CPU 01 changes the shooting interval T according to the increase in the number of people. For example, in the ticket purchase information, 30% of the seats are scheduled to be filled, but if the tickets are purchased after the start of the game or just before the start of the game and 60% of the seats are filled, the shooting interval T set in step S1808 is changed from 10 seconds to 7 seconds. The audience who purchased the tickets before and after the start of the game is likely not to have made any contact with the audience. Figure 7 Therefore, the shooting interval T set in step S1808 is changed according to the ratio of the increase or decrease of people. In step S1905, the shooting interval T may be changed according to the ratio of the area of ​​the audience calculated in step S1904, or the non-designated seats of the first place may not be required. Figure 8 processing, and can be based on Fig.19 The shooting interval T is set based on the ratio calculated in step S1904.

[0215] In step S1906, the CPU 01 determines whether the setting of step S1905 has been performed for all cameras. When it is determined that the processing of step S1905 has been performed for all cameras, the processing proceeds to step S1907, and if not, the camera arrangement information, shooting range information, and audience information intending to purchase images related to camera n=n+1 are acquired, and then the processing proceeds to step S1902.

[0216] In step S1907, the CPU 01 determines whether a predetermined time has elapsed. The predetermined time is, for example, 20 minutes or 30 minutes, and is an interval for updating the shooting interval T in step S1905. If it is determined that the predetermined time has elapsed, the process proceeds to step S1901; otherwise, the process proceeds to step S1908.

[0217] In step S1908, the CPU 01 determines whether to end Fig.19 If it is judged to be finished Fig.19 If the processing is completed, the processing ends; otherwise, the processing enters step S1907. Fig.19 The processing ends in response to the end of shooting.

[0218] As described above, according to this embodiment, since the shooting interval T is changed based on the vacant seat information, it is possible to reduce the amount of image data while providing images without missing shooting opportunities when there are many spectators. Since shooting opportunities differ according to each group and spectator, as the number of spectators photographed in one shooting increases, the shooting opportunities become more variable. Therefore, by determining the shooting interval T according to the number of spectators, the possibility of being able to shoot without missing shooting opportunities for each spectator when the number of spectators increases is increased. When the number of spectators decreases, the amount of data can be reduced.

[0219] Next, we will refer to Figures 20 to 21C right Fig.19 A description is given of various variations of the series of image capture processes (during a match) described in . Fig.19 Similar to the flowchart shown, when the process enters Fig.18 At step S1811 shown in FIG. 1 , the Fig. 20 and FIG. 21A to FIG. 21C This processing is realized by the CPU 01 deploying a program stored in the ROM 02 (non-volatile memory) in the RAM 03 (system memory) and then executing the program.

[0220] Fig. 20 This is a description of a modified example of changing the shooting interval based on face detection information.

[0221] In step S2001, the CPU 01 detects the face of the viewer from the image captured by the camera n. In the face detection in step S2001, the face of the viewer who indicates the image purchase intention in advance is detected.

[0222] In step S2002, the CPU 01 counts the number of persons whose faces are detected.

[0223] In step S2003, CPU 01 calculates the ratio of the number of viewers who indicated the image purchase intention in advance to the number of people whose faces were detected counted in step S2002. As a result, it can be known how many viewers in the shooting range of camera n are actually viewers who indicated the image purchase intention in advance.

[0224] In step S2004, the CPU 01 specifies a specific subject indicating a subject such as baseball team colors, flags, merchandise, etc. The specific subject may be specified by the viewer who has indicated a purchase intention, or may be specified by the image providing side.

[0225] In step S2005 , the CPU 01 detects a specific subject from the image acquired in step S2001 .

[0226] In step S2006, the CPU 01 changes the shooting interval T according to the ratio of the number of people whose faces are detected calculated in step S2003, or changes the shooting interval T according to the number of specific subjects detected in step S2005. Whether to change the shooting interval T according to the number of people whose faces are detected or to change the shooting interval T according to the number of specific subjects can be selected by the viewer who indicates the purchase intention, or can be set on the image providing side. For example, if the number of people whose faces are detected is 50%, it can be set at Fig.18 The shooting interval T set in step S1808 is set to twice the interval. In addition, for example, when there are many spectators wearing baseball team color uniforms, the possibility that the spectators may be excited is high; therefore, by shortening the shooting interval T, the possibility of missing a shooting opportunity can be reduced.

[0227] In step S2007, the CPU 01 changes the coordinates to be focused and the zoom amount based on the face detection result in step S2001. As a result, a more focused image can be provided to a viewer who intends to purchase the image based on the position of the detected face.

[0228] As described above, according to the embodiment, the shooting interval T is changed according to the detected face, so the number of images stored in the storage device 204 can be reduced. Fig. 20 processing.

[0229] Next, FIG. 21A to FIG. 21C Changing the shooting conditions based on the ratio of empty seats in the seats enables shooting under shooting conditions suitable for more spectators. In outdoor sports such as baseball and soccer, the brightness of the seats greatly varies depending on the amount of incident sunlight and the position of the roof. At this time, if shooting is performed with, for example, exposure that matches a bright position, there is a possibility that the shooting may not be suitable for spectators in a dark area. In this case, the processing for setting the ISO speed related to the exposure, aperture value, and shutter speed to be suitable for more spectators is FIG. 21A to FIG. 21C processing.

[0230] In step S2101, the CPU 01 acquires illumination information of predetermined N sections from the acquired image of the camera n. It is assumed that the N sections are sections consisting of seats (such as four or six adjacent seats, etc.) and include the entire area of ​​the shooting range of the camera n. The acquired image of the camera n is captured at the timing of step S2101, but may be captured before step S2101.

[0231] In step S2102, the CPU 01 sets the judgment interval n=1. The judgment interval indicates a judgment to classify the interval into one of illumination levels 1 to 4 based on the illumination information obtained in step S2102, where the illumination gradually becomes darker in the order of 1, 2, 3, and 4. The illumination level classifies the possible predetermined illumination at the seat into multiple levels. Four illumination levels as described in the present embodiment are not required, and if the illumination does not change greatly, there may be two or three illumination levels. In addition, if the shooting settings related to the exposure do not change even when the illumination changes, the illumination level may be given a wider range as one illumination level.

[0232] In step S2103 , the CPU 01 determines, based on the illuminance information of the determination interval n, whether the determination interval n is an interval of illuminance level 1. If it is determined that the interval is illuminance level 1, the process proceeds to step S2104 ; otherwise, the process proceeds to step S2105 .

[0233] In step S2104, the CPU 01 registers the determination section n in the illuminance level 1 list.

[0234] In step S2105, the CPU 01 determines based on the illuminance information of the determination interval n whether the determination interval n is an interval of illuminance level 2. If it is determined that the interval is illuminance level 2, the process proceeds to step S2106; otherwise, the process proceeds to step S2107.

[0235] In step S2106, the CPU 01 registers the determination section n in the illuminance level 2 list.

[0236] In step S2107, the CPU 01 determines based on the illuminance information of the determination interval n whether the determination interval n is an interval of illuminance level 3. If it is determined that the interval is illuminance level 3, the process proceeds to step S2108; otherwise, the process proceeds to step S2109.

[0237] In step S2108, the CPU 01 registers the determination section n in the illuminance level 3 list.

[0238] In step S2109, the CPU 01 registers the determination section n in the illuminance level 4 list.

[0239] In step S2110, the CPU 01 determines whether the determination for all determination intervals has been completed. If the determination in step S2110 is "yes", the process proceeds to step S2112; otherwise, the process proceeds to step S2111.

[0240] In step S2111 , the CPU 01 sets the determination interval n=n+1 and proceeds to step S2103 .

[0241] Go to Fig.21B In step S2112, the CPU 01 counts the number of people whose faces are detected in the intervals of illumination levels 1 and 2. That is, the number of faces of viewers in the area of ​​illumination level 1 and the number of faces of viewers in the area of ​​illumination level 2 are counted respectively.

[0242] In step S2113, the CPU 01 determines whether the number of people whose faces are detected in the interval of illumination level 1 is greater than the number of people in the interval of illumination level 2. If it is determined that the number of people whose faces are detected in the interval of illumination level 1 is greater than the number of people in the interval of illumination level 2, the process proceeds to step S2114, and if not, the process proceeds to step S2115.

[0243] In step S2114 , the CPU 01 sets the photographing setting related to the exposure conforming to the illuminance 1 as the photographing setting 1 .

[0244] In step S2115 , the CPU 01 sets the photographing setting related to the exposure conforming to the illuminance 2 as the photographing setting 1 .

[0245] In step S2116, CPU 01 compares the empty seat image in the interval of illumination levels 3 and 4 with the acquired image of camera n, and then determines the area where the audience is seated. The acquired image of camera n is captured by camera n in step S2116, but may be captured before step S2116.

[0246] In step S2117, the CPU 01 calculates the ratio of the area where people exist in the interval of illumination levels 3 and 4. Since the illumination at illumination levels 3 and 4 is low, the accuracy of human face detection may be lower than that at illumination levels 1 and 2. Therefore, instead of detecting faces, the number of people sitting is counted by comparing images. On the other hand, in the case of illumination levels 1 and 2, face detection is used to count the number of people more accurately.

[0247] In step S2118, the CPU 01 determines whether the number of people in the interval of illumination level 3 is greater than the number of people in the interval of illumination level 4. If it is determined that the number of people in the interval of illumination level 3 is greater than the number of people in the interval of illumination level 4, the process proceeds to step S2119, and if not, the process proceeds to step S2120.

[0248] Go to Fig. 21C In step S2119, the CPU 01 sets the photographing setting related to the exposure conforming to the illuminance 3 to the photographing setting 2.

[0249] In step S2120 , the CPU 01 sets the photographing setting related to the exposure conforming to the illuminance 4 as the photographing setting 2 .

[0250] Next, in steps S2121 to S2129, a process of setting the shooting interval T according to the number of people at each illuminance level is performed.

[0251] In step S2121, the CPU 01 calculates the ratio of the number of people based on the illuminance α = (the number of people in illuminance levels 1 and 2) / (the number of people in illuminance levels 3 and 4) obtained in steps S2112 and S2116. When it is determined that the value of α is 1.5 or greater, that is, when the number of people in the bright area is 1.5 times or more the number of people in the dark area within the shooting range of camera n, the process proceeds to step S2122. If 1.0 ≤ α < 1.5, the process proceeds to step S2124; if 0.7 ≤ α < 1.0, the process proceeds to step S2126; and if α < 0.7, the process proceeds to step S2128.

[0252] In step S2123, the CPU 01 sets the shooting at shooting setting 1 to T = t1, and sets the shooting at shooting setting 2 to T = t2 (> t1). For example, t1 is 5 seconds and t2 is 10 seconds.

[0253] In step S2125, the CPU 01 sets the shooting at shooting setting 1 to T = t1, and sets the shooting at shooting setting 2 to T = t3 (> t2). For example, t1 is 5 seconds and t3 is 15 seconds.

[0254] In step S2127, the CPU 01 sets the shooting at shooting setting 1 to T = t4, and sets the shooting at shooting setting 2 to T = t5 (< t4). For example, t4 is 10 seconds and t5 is 5 seconds.

[0255] In step S2129, the CPU 01 sets the shooting at shooting setting 1 to T = t6 (> t4), and sets the shooting at shooting setting 2 to T = t5. For example, t6 is 15 seconds and t5 is 5 seconds.

[0256] Therefore, when there are more people in the bright area, shooting with an exposure setting corresponding to the illuminance of the bright area is performed more frequently than shooting with an exposure setting corresponding to the illuminance of the dark area. In addition, when there are more people in the dark area, shooting with an exposure setting corresponding to the illuminance of the dark area is performed more frequently than shooting with an exposure setting corresponding to the illuminance of the bright area.

[0257] In step S2130, the CPU 01 determines whether to end FIG. 21A to FIG. 21C the process. When it is determined to end FIG. 21A to FIG. 21C the process, FIG. 21A to FIG. 21C the process ends; otherwise, the process proceeds to step S2101.

[0258] As described above, according to the described embodiment, the shooting frequency under the shooting setting that matches the illumination level is performed according to the number of spectators, so that the image that better captures the spectators, that is, the image captured under the appropriate shooting conditions, can be increased. In addition, in the area where there are fewer spectators, the shooting interval T is increased and the shooting frequency is reduced, so the amount of image data can be reduced.

[0259] Next, a method of providing an image of a spectator displayed on a monitor of a venue will be described. Sports venues and entertainment facilities such as an aquarium generally have a large screen. In addition, there is an activity in which the spectator is displayed on the screen. At this time, in order to provide the image displayed on the screen to the spectator displayed on the screen, it is necessary to specify the spectator. However, when face detection is used, it takes time in sports games with tens of thousands of spectators, etc.

[0260] In this embodiment, reference will be made to Figures 22 to 24 A method of providing an image to a viewer displayed on a screen is described.

[0261] Fig. 22 01 is a flowchart of the pre-registration process involving the screen. The process can be started in response to the start of ticket sales before the start of the game. The process is implemented by the CPU 01 deploying the program stored in the ROM 02 (non-volatile memory) in the RAM 03 (system memory) and then executing the program.

[0262] In step S2201, the CPU 01 determines whether the viewer has a cropped image purchase intention. If it is determined that there is a cropped image purchase intention, the process proceeds to step S2202; otherwise, step S2202 is skipped and the process proceeds to step S2203.

[0263] In step S2202, the CPU 01 registers the seat number in the image purchaser list.

[0264] In step S2203, the CPU 01 determines whether pre-registration for screen image acquisition has been performed. When purchasing a ticket, you can choose whether the audience wants the image to be displayed on the screen. If you choose to pre-register, the process enters step S2204; otherwise, the pre-registration process ends.

[0265] In step S2204, the CPU 01 registers the seat number in the screen list. Note that the image purchaser list and the screen list are lists created for each game.

[0266] In step S2205, the CPU 01 determines whether face registration has been performed. If pre-registration is selected in step S2203, a guide is displayed to the effect that if the audience captures their faces, images can be acquired when they appear on the screen with a higher chance. If the audience selects face registration, the face of the registrant is captured in step S2206. If face registration is not selected, the process proceeds to step S2208. Regarding capturing, when the application terminal is a smartphone, capturing can be performed using a built-in camera on the site, or a pre-captured image can be selected.

[0267] In step S2207, the CPU 01 associates the seat number and the face data.

[0268] In step S2208, the CPU 01 issues a temporary ID to the spectator, and registers the information associated in step S2207 in the screen list. The temporary ID is personal information assigned to the spectator who has registered in advance in each game, and is used to obtain an image.

[0269] As described above, according to the described embodiment, by registering the faces of viewers in advance, viewers can easily acquire images of themselves displayed on the screen.

[0270] Next, we will refer to Fig.23 Describes the spectator list created on the day of a match.

[0271] Fig.23 1 is a flowchart related to the screen registrant list creation process. This process is performed within a limited time (such as 10 minutes or 20 minutes, etc.) after the start of the game. This process is realized by the CPU 01 deploying the program stored in the ROM 02 (non-volatile memory) in the RAM 03 (system memory) and then executing the program.

[0272] In step S2301, the CPU 01 acquires the image purchase list and screen list before creation.

[0273] In step S2302, the CPU 01 determines whether a viewer holding a permanent ID has entered. The permanent ID is used to distinguish an ID from a temporary ID and is personal information assigned to a viewer in a fan club or a viewer registered in an application of a team. The viewer holding a permanent ID is a frequent viewer and has visited many times, so it is assumed that facial data already exists.

[0274] In step S2303, the CPU 01 registers the permanent ID holding viewer who has entered in the screen list and prepares face data.

[0275] In step S2304, the CPU 01 stores the facial data of the viewer who intends to purchase the cropped image. Regarding the viewer who intends to purchase the cropped image, the facial data from the cropped image is stored.

[0276] In step S2305, the CPU 01 stores the facial data of viewers who have no facial data among the viewers who have been registered to acquire the screen image. Trimming is performed to include the viewers at the positions registered in step S2204, and then the facial data is stored.

[0277] In step S2306, the CPU 01 creates an active subject list from the image purchase list and the screen list. This is a list used when the camera captures a scene of cheering spectators during a game, and is displayed on the screen. Since not all seats are displayed even if captured by multiple cameras, seats that were not captured in the camera initially during the event are not included in the active subject list. For example, seats on a floor that are not captured from the position of the camera for the event, or seats in front of the camera for the event are not included in the active subject list.

[0278] In step S2307, the CPU 01 creates an in-game display list from the image purchase list and the screen list. There are cases where the state in which the audience is photographed together with the athletes during the game can be displayed on the screen. For example, when the athletes are photographed, the audience in the seats in area 1 may also be photographed. Therefore, the in-game display list is created from the image purchase list and the screen list with seats that may be the shooting range of the camera used to shoot the athletes during the game as the object.

[0279] As described above, according to the described embodiment, since the list is created for viewers who may want to display images on the screen, the processing load at the time of face detection can be reduced, and the image can be quickly provided to the viewer. In addition, by creating different lists according to the trigger when displaying on the screen, the processing load can be further reduced.

[0280] Next, we will refer to Fig.24 Describes the image rendering process.

[0281] Fig.24 1 is a flowchart related to a process for providing a screen image. The process starts in response to the start of display on the screen by the viewer. The process is implemented by the CPU 01 deploying a program stored in the ROM 02 (non-volatile memory) in the RAM 03 (system memory) and then executing the program.

[0282] In step S2401, the CPU 01 determines whether the system is in an active state. If it is determined to be in an active state, the process proceeds to step S2402; otherwise, the process proceeds to step S2403.

[0283] In step S2402 , the CPU 01 prepares an active subject list.

[0284] In step S2403, the CPU 01 prepares to display a list of subjects during the game.

[0285] In step S2404, the CPU 01 acquires a screen image that captures the audience. This image may be acquired from a camera that captures the audience, or a camera for observing the screen may be provided.

[0286] In step S2405, the CPU 01 detects the face of the main subject in the image acquired in step S2404.

[0287] In step S2406, the CPU 01 matches the face detected in step S2405 with the face data of the ID holder in the list prepared in step S2402 or S2403, and determines whether they match. If it is determined that the ID holder and the face data match, the process proceeds to step S2407; otherwise, the process proceeds to step S2408.

[0288] In step S2407, the CPU 01 stores the image displayed on the screen and the seat number and date of the audience member whose facial data is determined to have matched in step S2406 in the storage device 204, and immediately distributes the image to the audience member. It should be noted that a moving image can be distributed here instead of a still image. In step S2408, the CPU 01 determines whether the face detected in step S2405 matches the facial data of the audience member who intends to purchase the cropped image in the list prepared in step S2402 or S2403. If the facial data of the audience member who intends to purchase the cropped image matches the detected face, the process proceeds to step S2409, and if it does not match, the process proceeds to step S2410.

[0289] In step S2409, the CPU 01 stores in the storage device 204 an image cut out around the time when the viewer whose face data has been determined in step S2408 to match the image displayed on the screen was captured, and then immediately distributes the cut out image to the viewer.

[0290] In step S2410, the CPU 01 determines whether the face detected in step S2405 matches the facial data of the person who wishes to obtain the screen image in the list prepared in step S2402 or S2403. If it is determined that the person who wishes to obtain the screen image matches the facial data, the process proceeds to step S2411. If not, the process proceeds to step S2412. Fig.24In addition, when it is determined that an activity of displaying the audience on the screen has occurred, the process enters step S2401 again and performs the process.

[0291] In step S2411, the CPU 01 stores the image with the seat number and date in the storage device 204 and distributes it to the audience. The distribution method may be a distribution method different from the method of distributing the cropped image to a person who wishes to purchase the cropped image, or may be the same method.

[0292] Furthermore, the image displayed on the screen distributed to the audience may be an image from a camera, or may be a shot of the screen itself.

[0293] According to the above-described embodiment, when an image is provided to a viewer after the face of the viewer displayed on the screen is detected, the image can be provided quickly and reliably.

[0294] Since a match is determined for a detected face by creating a list of viewers who may want the image and narrowing it down to a list of viewers within a range of possibilities for display for each activity, processing can be performed faster.

[0295] <Other embodiments>

[0296] In the above-described embodiment, the present invention has been described as the image providing system, but it may be configured as an image providing apparatus.

[0297] Notice, Figure 2 The image processing device 201 does not need to be a single device, but may be divided into a plurality of devices.

[0298] Despite Figure 7 In the description, it has been described that the image purchase intention is input in step S702 immediately after the ticket purchase in step S701, but in the case where the image purchase intention of step S702 is accepted, the ticket information such as the ticket code and number can be read before the game starts or during the game. At this time, once the shooting starts (such as after the game starts, etc.), the group information generated in step S705 can be updated. As a result, even if the number of people in the group changes on the day or during the game, by updating the group information, it is possible to perform a cropping including all members of the group. In addition, it is possible to configure so that a shooting reservation is realized before the ticket is purchased. In this case, if the ticket information is read when the user purchases the ticket and decides on the seat, shooting is performed based on the read seat.

[0299] Figures 11A to 11C The seating arrangement pattern of the groups shown need not be determined by the CPU 01 and may be determined on the user application side.

[0300] exist Fig.19In step S1905, it has been described that the shooting interval is changed according to the increase in the number of people; however, data related to the number of spectators of the game can be maintained in advance, and the shooting interval can be controlled based on the data. At this time, data on the number of spectators according to the area can be maintained, and the shooting interval can be changed according to the area.

[0301] When Fig.24 When providing images in step S2411, the images may be provided in a set with images taken before and after the time provided in step S967. In addition, in step S2411, images may be provided so that the user can know the position of the camera that photographed the audience displayed on the screen.

[0302] The present invention may also be implemented in such a process, wherein a program for implementing one or more of the above-mentioned functions of each embodiment is provided to a system or device via a network or storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention may also be implemented by a circuit (e.g., ASIC) that implements one or more functions.

[0303] The above-mentioned various embodiments are merely illustrative in the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these embodiments. In other words, the present invention can be embodied in various forms without departing from the technical idea of ​​the present invention or its main features.

[0304] Note that the above-described various controls described as being performed by the image processing apparatus 201 may be performed by one piece of hardware or a plurality of pieces of hardware (eg, a plurality of processors and circuits) that share processing to control the entire apparatus.

[0305] In addition, although the present invention has been described in detail based on the preferred embodiments of the present invention, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. In addition, each embodiment of the above-mentioned embodiment is only one embodiment of the present invention, and each embodiment of these embodiments can also be appropriately combined.

[0306] In the above-mentioned embodiment, the case where the present invention is applied to the image processing device 201 has been described as an example, but the present invention is not limited to this example, and can be applied to any electronic device or system including a plurality of electronic devices capable of controlling an image. That is, the present invention can be applied to a personal computer, a PDA, a mobile phone terminal, a portable image viewer, a printer device including a display, a digital photo frame, a music player, a game console, an e-book reader, etc. In addition, it goes without saying that it is also applicable to control in combination with AI (artificial intelligence).

[0307] The present invention is also implemented by performing the following processing. That is, the software (program) for implementing the functions of the above-mentioned embodiments is provided to the system or device through a network or various storage media, and the computer (or CPU or MPU) of these systems or devices reads and executes the program code. In this case, the program and the storage medium storing the program constitute the present invention.

[0308] The present invention is not limited to the above-described embodiments, and various changes and modifications may be made within the spirit and scope of the present invention. Therefore, in order to apprise the public of the scope of the present invention, the following claims are made.

Claims

1. An image providing system, comprising: an acquisition component for acquiring ticket information including position information related to seat positions for each user group from an external ticket purchasing agency; as well as A control component, for controlling the image providing system to: Before the image capturing unit starts a series of image capturing in which the image capturing unit captures a plurality of images, a cropping range of an image to be captured by the image capturing unit is determined for each user group based on the ticket information so that the cropping range includes a seat position for the user group, After the series of image capturing is started, cropping of cutting out a portion of the captured image is continuously applied based on the cropping range determined for each user group, and Each cropped image is uploaded to a server accessible to users who have purchased tickets through the ticket purchasing agency.

2. The image providing system according to claim 1, wherein: During the series of image capturing, if the ticket information is updated, the control section controls the image providing system to update the cropping range based on the updated ticket information.

3. The image providing system according to claim 1, wherein: The series of image capturing is started regardless of the timing at which the acquiring section acquires the ticket information.

4. The image providing system according to claim 1, wherein: The image providing system is capable of receiving images from a plurality of cameras placed at different locations, and The control section controls the image providing system to apply cropping to an image from a camera among the plurality of cameras that captures an image capturing range including seat positions of users of a user group, wherein the cropping cuts out a portion of the captured image based on the cropping range.

5. The image providing system according to any one of claims 1 to 4, wherein: Each user who has purchased a ticket through the ticket purchasing agency is enabled to obtain the cropped image from the server by inputting information on the date and time of the event at which the series of image captures were conducted or inputting the seat position.

6. A control method for an image providing system, the method comprising: acquiring ticket information including location information regarding seat locations for each user group from an external ticket purchasing agency; Before the image capturing unit starts a series of image capturing in which the image capturing unit captures a plurality of images, determining a cropping range of an image to be captured by the image capturing unit for each user group based on the ticket information so that the cropping range includes a seat position for the user group; After starting the series of image capturing, continuously applying cropping that cuts out a portion of the captured image based on the cropping range determined for each user group; and Each cropped image is uploaded to a server accessible to users who have purchased tickets through the ticket purchasing agency.

Images