Image photographing system

Abstract

The present invention automatically captures an image of a subject user located in a predetermined range and reduces time and effort of the subject user. An image photographing system comprises: a plurality of positioning sensors that perform ultra-wide-band (UWB) communication with a tag carried by a user, calculate a distance to the tag and a direction of the tag, and detect a tag ID assigned to the tag; a registration terminal that groups the tag IDs of the tags and registers the tag IDs together with a group ID; and a photographing control device that detects a position of the user carrying the tag by using calculation results of the plurality of positioning sensors, controls a photographing device to perform photographing when the user is within a predetermined photographing area, and stores a photographed image and the group ID in association with each other.

Description

Imaging system

[0001] The present invention relates to an imaging system.

[0002] Conventionally, in entertainment facilities such as theme parks and amusement parks, services have been provided to take pictures of guests inside attractions and sell printed copies of the taken pictures after the attractions end. For example, shooting has been performed using sensor detections such as object detection by an object detection sensor, detection of a change in light amount by a light amount sensor, and person detection by a person detection sensor as shooting triggers. Also, a system has been proposed in which a shooting ID is read before and after shooting, the taken picture and the shooting ID are associated and stored, and at a sales terminal, the taken picture associated with the shooting ID is browsed and purchased.

[0003] However, in the conventional system, since it is necessary to read the shooting ID before and after shooting, the operation of the user (guest) is complicated. Also, it has been unclear whether the user is within the shooting area intended by the user at the time of shooting.

[0004] Also, other users may be captured in the picture during shooting, and personal information (face image) of other users may be unintentionally acquired. When there are other users between the imaging device and the subject user, other users may also interfere with the shooting.

[0005] Japanese Unexamined Patent Application Publication No. 2021 - 150907, Japanese Unexamined Patent Application Publication No. 2023 - 708

[0006] The present invention has been made in view of the above - described conventional situation, and an object thereof is to provide an imaging system that can automatically capture a subject user located within a predetermined range and reduce the labor of the subject user.

[0007] The image capture system according to the present invention comprises: a plurality of positioning sensors that communicate with a tag held by a user via UWB (Ultra-Wide-Band), calculate the distance to the tag and the direction of the tag, and detect the tag ID assigned to the tag; a registration terminal that groups the tag IDs of the tags and registers them together with the group ID; and a shooting control device that uses the calculation results of the plurality of positioning sensors to detect the location of the user holding the tag, controls the shooting device to take a picture when the user is within a predetermined shooting area, and saves the captured image linked to the group ID.

[0008] According to the present invention, it is possible to automatically photograph subjects (users) located within a predetermined range while reducing the effort required of the subjects (users).

[0009] This figure shows an example of using the image capture system according to an embodiment of the present invention. This is a block diagram of the image capture system. This is a block diagram of the management device. This is a block diagram of the image capture control device. This is a block diagram of the sales terminal. This is a flowchart explaining the image capture method according to the same embodiment. This is a flowchart explaining the image capture method according to the same embodiment. This is a flowchart explaining the head posture estimation method. This figure shows an example of face region expansion. This figure shows an example of head posture parameters.

[0010] Hereinafter, embodiments of the present invention will be described based on the drawings.

[0011] Figure 1 shows a leisure facility with an image capture system installed, and Figure 2 shows a schematic configuration of the image capture system. The leisure facility has multiple attractions #1 to #N. For example, user U moves to the next attraction after experiencing one attraction. Automatic image capture of user U is performed at each attraction. Each attraction is equipped with a shooting device 1, a positioning sensor 2, and an image capture control device 3. The shooting device 1 is, for example, a digital camera. It is preferable that multiple positioning sensors 2 are installed at each attraction.

[0012] User U receives a tag T from a staff member at the entrance reception. Tag T and positioning sensor 2 are configured to send and receive pulse signals used in UWB (Ultra-Wide-Band) communication. Pulse signals used in UWB communication have an extremely short pulse width (e.g., 2ns) and a bandwidth of 500 MHz or more (ultra-wideband). Frequency bands that can be used for UWB communication include 3.1 GHz to 10.6 GHz, 3.4 GHz to 4.8 GHz, 7.25 GHz to 10.6 GHz, 22 GHz to 29 GHz, etc.

[0013] Each tag T is assigned a unique ID (identification information), and staff register the ID (tag ID) of the tag T distributed to user U. For example, tag T has code information such as a barcode or two-dimensional code printed on it that indicates the tag ID. Staff use a reader 5 such as a hand scanner to read the code information and operate the first registration terminal 4 to register the ID of the tag T distributed to user U. If user U is visiting as a group, such as family or friends, staff use the first registration terminal 4 to group the tag IDs, assign a group name (group ID), and register them. Even if user U is alone, the group ID is assigned to the user's tag ID and registered.

[0014] The shape of the tag T is not particularly limited; it can be any shape that user U can carry. For example, wristband-type or neck-worn tag T are easy for user U to carry and easy to collect and manage.

[0015] User U enjoys attractions #1 through #N in order while possessing tag T. At each attraction, multiple positioning sensors 2 transmit pulse signals at regular intervals (e.g., every 0.2 seconds). Tag T is a pulse-re-emitting UWB communicator. Upon receiving a pulse signal, it amplifies and detects the signal, re-oscillates it using UWB, adds a tag ID, and returns the signal. Positioning sensors 2 receive the returned signal, detect the tag ID, and calculate the distance and direction to tag T.

[0016] For example, the distance to tag T is calculated based on the time of arrival (TOA) from the transmission of the pulse signal to the reception of the return signal. Alternatively, the return signal can be received using a planar array antenna such as a trapezoidal array, the phase of the return signal can be determined, and the direction of tag T can be calculated.

[0017] The shooting control device 3 accurately detects the position of tag T (the position of user U) from the calculation results of multiple positioning sensors 2.

[0018] Each attraction has a pre-set shooting area R1 (shooting area). If user U is within shooting area R1, shooting device 1 will take a picture of user U. Shooting may also be performed if all registered group users U are within shooting area R1.

[0019] Additionally, a shooting stop area R2 may be set in advance. The shooting stop area R2 is the area between the shooting device 1 and the shooting area R1, or the area behind the shooting area R1 as seen from the shooting device 1. If a user is in this shooting stop area R2, shooting may be stopped. This prevents other users unrelated to the subject user in the shooting area R1 from appearing in the captured image. Whether or not a user is in the shooting stop area R2 is determined based on the position of tag T.

[0020] It is preferable that the shooting stop area R2 is located between the shooting device 1 and the shooting area R1. Furthermore, it is preferable that the shooting stop area R2 is located behind the shooting area R1 when viewed from the perspective of the shooting device 1. This allows for accurate prevention of other users appearing in the captured image.

[0021] The shooting stop area R2 may be positioned to surround the shooting area R1. The shooting stop area R2 may also be set in a fan shape with the shooting device 1 as the vertex, taking into consideration the field of view (shooting range) of the shooting device 1. The shooting stop area R2 may be adjacent to the shooting area R1, or there may be a gap between them.

[0022] The sizes of the shooting area R1 and the shooting stop area R2 are not particularly limited and can be set to any size. For example, the size of the shooting area R1 can be set to a radius of approximately 2m.

[0023] The shooting control device 3 links the image captured by the shooting device 1 with the subject user's location, tag ID, and group ID, and transmits it to the management device 20 (see Figure 2).

[0024] The management device 20 stores the received captured images in groups. The management device 20 analyzes the captured images and calculates a score. For example, scoring is performed based on the gaze and facial expressions of the subject user. If the location of a landmark L, such as a symbolic statue or character figurine, is registered, the distance between the landmark L and the subject user may also be used in the score calculation.

[0025] User U returns Tag T at the exit reception after completing all attractions. Staff members use a reader 7, such as a hand scanner, to read the code information printed on Tag T and confirm that all Tag Ts for the group are present. After confirming the return of all Tag Ts, a ticket with code information such as a barcode or QR code indicating the group ID is printed from the printer 8. Staff members hand the ticket to User U.

[0026] The timing of handing this ticket to user U is not limited to when tag T is returned. For example, when grouping tag IDs at the entrance reception and distributing tag T to user U, a ticket with code information indicating the group ID could be printed and given to user U.

[0027] User U moves to the sales counter with their ticket and hands it to a staff member. The staff member uses a reader 10, such as a hand scanner, to read the code information printed on the ticket and obtains a group ID. The sales terminal 9 creates picture book data using the captured image corresponding to the group ID and displays it on the display 11. For example, the picture book data is created by compositing the captured image onto a picture book template. Alternatively, captured images that receive high scores through image analysis may be used for compositing.

[0028] Multiple picture book templates may be associated with multiple attractions #1 to #N or multiple shooting areas R1, and images taken at each attraction or shooting area R1 may be combined with the corresponding picture book template. If the score of the captured image obtained through image analysis is low (below a predetermined value), a pre-prepared default image may be combined with the picture book template instead of the captured image.

[0029] The user checks the picture book displayed on the display 11 and purchases it if they like it. After payment for the picture book, the shipping information is entered into the sales terminal 9. The sales terminal 9 sends the picture book data and shipping information to the printing factory. Later, the picture book, bound using the captured image, is delivered to the user.

[0030] Thus, by using the image capture system according to this embodiment, it is possible to capture natural images of users experiencing attractions in a specific shooting area. Furthermore, since there is no need to read the shooting ID before or after shooting, the shooting operation can be simplified. In addition, by using high-scoring images, it is possible to increase the purchase rate of picture books.

[0031] Next, the configuration of the image acquisition system will be described in detail. As shown in Figure 3, the management device 20 is a computer having a storage unit 21 such as an HDD or SSD, and the CPU executes a management program to realize the functions of the area setting unit 22, the shooting condition setting unit 23, the group information receiving unit 24, the group information transmitting unit 25, the image receiving unit 26, the analysis unit 27, and the image transmitting unit 28.

[0032] The area setting unit 22 sets area information indicating the location (region) of the shooting area R1 and shooting stop area R2 for each attraction. For example, the facility manager operates the management device 20 and inputs the coordinates of the positioning sensor 2, shooting area R1, shooting stop area R2, etc., to set the area information. In the case of an attraction where a landmark L is installed, the coordinates of the landmark L are also included in the area information. The area information is stored in the storage unit 21. The area setting unit 22 transmits the area information for the corresponding attraction to the shooting control device 3 for each attraction.

[0033] The shooting condition setting unit 23 sets shooting conditions for each attraction to determine whether or not shooting is permitted. Examples of shooting conditions include: "Shooting is performed when all users registered in the group are in shooting area R1", "Shooting is performed when at least one user registered in the group is in shooting area R1", "Shooting is not performed if at least one user is in shooting cancellation area R2", "Shooting is not performed if there are other users in shooting cancellation area R2 who are from a different group than the users in shooting area R1", and "Shooting is not performed if there are users from two or more groups in shooting area R1". The facility manager can set the shooting conditions by operating the management device 20. The shooting conditions are stored in the storage unit 21. The shooting condition setting unit 23 transmits the shooting conditions to the shooting control device 3 for each attraction.

[0034] The group information receiving unit 24 receives group information from the first registration terminal 4, including the grouped tag ID and the group ID that indicates this group. The received group information is stored in the storage unit 21.

[0035] The group information transmission unit 25 transmits group information to the camera control device 3 and second registration terminal 6 of each attraction.

[0036] The image receiving unit 26 receives captured images, the tag IDs and group IDs of the subject users in the captured images, and the coordinates of the subject users (tag T) from the shooting control device 3 of each attraction. The data received by the image receiving unit 26 is stored in the storage unit 21.

[0037] The analysis unit 27 analyzes the captured image received by the image receiving unit 26 and performs scoring. For example, the analysis unit 27 calculates a score based on the subject user's gaze and facial expressions. The image analysis algorithm is not particularly limited and known algorithms can be used. In the case of an image taken at an attraction where a landmark L is installed, the analysis unit 27 may calculate the distance from the subject user to the landmark L from the subject user's coordinates and the coordinates of the landmark L, and reflect the calculated distance in the score. For example, the shorter the distance to the landmark L, the higher the score. The calculated score is stored in the storage unit 21, linked to the captured image.

[0038] Image analysis may be performed on a computer separate from the management device 20. In this case, the management device 20 transmits the captured image to an external computer and receives the analysis result.

[0039] When the image transmission unit 28 is notified of the group ID from the sales terminal 9, the image transmission unit 28 transmits the captured image associated with this group ID to the sales terminal 9. The image transmission unit may transmit only the captured images with a score of a predetermined value or more to the sales terminal 9. Alternatively, score information may be transmitted to the sales terminal 9 together with the captured image. Information on the attraction taken may also be transmitted together with the captured image.

[0040] As shown in FIG. 4, the imaging control device 3 installed in each attraction is a computer having a storage unit 31 such as an HDD or SSD, and when the CPU executes an imaging control program, the area information reception unit 32, the imaging condition reception unit 33, the group information reception unit 34, the position detection unit 35, the determination unit 36, the imaging instruction unit 37, the image acquisition unit 38, and the functions of the image transmission unit 39 are realized.

[0041] The area information reception unit 32 receives area information including coordinates of the imaging area R1, the imaging stop area R2, etc. from the management device 20 and stores it in the storage unit 31.

[0042] The imaging condition reception unit 33 receives imaging conditions from the management device 20 and stores them in the storage unit 31.

[0043] The group information reception unit 34 receives group information from the management device 20 and stores it in the storage unit 31.

[0044] The position detection unit 35 detects the position of the user within the attraction from the distance and direction to the user calculated by the plurality of positioning sensors 2. The position detection unit 35 detects and monitors the tag ID and position of all users within the attraction.

[0045] The determination unit 36 determines whether or not to perform imaging by the imaging device 1 based on the area information, the imaging conditions, and the position of the user. For example, the determination unit 36 determines to perform imaging when all users in the same group are within the imaging area R1 and no one is in the imaging stop area R2.

[0046] When the shooting instruction unit 37 determines that shooting is to be performed by the determination unit 36, it controls the imaging device 1 to perform shooting. As soon as the shooting instruction unit 37 determines that shooting is to be performed, it may instruct the imaging device 1 to shoot, or may instruct shooting after a predetermined time has elapsed. The shooting instruction unit 37 may instruct a single shooting, or may instruct continuous shooting a predetermined number of times. The shooting instruction unit 37 may instruct video shooting for a predetermined time. The captured video includes a plurality of frames.

[0047] The image acquisition unit 38 acquires a captured image from the imaging device 1, and associates and stores in the storage unit 31 the coordinates, tag ID, and group ID of the subject user within the shooting area R1.

[0048] The image transmission unit 39 transmits the captured image, the coordinates of the subject user, the tag ID, and the group ID to the management device 20.

[0049] As shown in FIG. 5, the sales terminal 9 installed at the sales corner is a computer having a storage unit 91 such as an HDD or an SSD. When the CPU executes a sales program, the functions of the group ID acquisition unit 92, the image acquisition unit 93, the picture book data generation unit 94, the display processing unit 95, the user information input unit 96, and the picture book data transmission unit 97 are realized.

[0050] The group ID acquisition unit 92 acquires the group ID from the code information printed on the ticket owned by the user using the reader 10.

[0051] The image acquisition unit 93 notifies the management device 20 of the group ID acquired by the group ID acquisition unit 92, and acquires the captured image corresponding to the group ID. For example, the image acquisition unit 93 acquires an image in which the users within the group corresponding to this group ID are shown.

[0052] The picture book data generation unit 94 generates picture book data by combining the captured images acquired by the image acquisition unit 93 with the picture book template stored in the memory unit 91. If the user has not taken any photos at any of the attractions, or if the score of the captured images is low, and there are not enough images to combine with the picture book template, the default images stored in the memory unit 91 may be used.

[0053] The display processing unit 95 displays the generated picture book data on the display 11.

[0054] The user information input unit 96 accepts user information such as the user's name and address (picture book delivery address information).

[0055] The picture book data transmission unit 97 transmits the picture book data and user information to the printing plant.

[0056] Next, the image acquisition method according to this embodiment will be explained following the flowcharts shown in Figures 6 and 7.

[0057] Before users enter the leisure facility, the area setting unit 22 of the management device 20 sets the shooting area R1 and shooting cancellation area R2 for each attraction (step S11). The set area information is transmitted to the shooting control device 3 of each attraction. In addition, the shooting conditions are also transmitted to the shooting control device 3 of each attraction along with the area information.

[0058] At the entrance reception of the leisure facility, the tag ID is read by the reader 5 and then the tag T is distributed to the user. The first registration terminal 4 groups the read tag IDs together and assigns a group ID, and registers them (step S12). The first registration terminal 4 transmits the group information, which includes the group ID and the tag IDs included in the group, to the management device 20. The management device 20 transmits the group information to the camera control device 3 and the second registration terminal 6 of each attraction.

[0059] Users who receive a tag T enjoy the attraction. At each attraction, the positioning sensor 2 and the shooting control device 3 detect the location of all users (tags T held by the users) at predetermined time intervals (step S13).

[0060] The determination unit 36 ​​of the shooting control device 3 determines whether or not to perform shooting with the shooting device 1 based on area information, shooting conditions, and the user's position. For example, the determination unit 36 ​​determines to perform shooting if the user is in the shooting area R1 (step S14_Yes) and there are no users from other groups in the shooting cancellation area R2 (step S15_Yes). If none of the users in the same group are in the shooting area R1, the determination unit 36 ​​may determine not to perform shooting.

[0061] If no users are in the shooting area (step S14_No) or if there are users from other groups in the shooting cancellation area R2 (step S15_No), it is determined that shooting will not be performed.

[0062] When it is determined that an image should be taken, the shooting instruction unit 37 controls the shooting device 1 and the shooting is performed (step S16).

[0063] The shooting control device 3 acquires the captured image from the shooting device 1 and associates the captured image with the tag ID and group ID of the subject user in the shooting area R1. The shooting control device 3 transmits the captured image, along with the coordinates, tag ID, and group ID of the subject user in the shooting area R1, to the management device 20 (step S17). The management device 20 stores the captured image, the coordinates, tag ID, and group ID (step S18).

[0064] The analysis unit 27 of the management device 20 analyzes the captured images and calculates a score (step 19).

[0065] After the user has finished experiencing all the attractions, they return the tag T at the exit reception. The reader 7 reads the tag ID of the returned tag T (step S21). The second registration terminal 6 registers the tag ID read by the reader 7 as a returned tag. The second registration terminal 6 also refers to the group information and identifies the group corresponding to the returned tag T (step S22). Once all tag T within the same group has been returned, the second registration terminal 6 controls the printer 8 and prints a ticket with code information indicating the group ID of this group (step S23).

[0066] When the user receives the ticket printed from printer 8, they move to the sales counter and have the ticket code information read by reader 10 (step S24). The ticket may also be printed during group ID registration in step S12 and given to the user before experiencing the attraction.

[0067] The sales terminal 9 transmits the group ID read by the reader 10 to the management device 20 and acquires the captured image corresponding to the group ID (step S25).

[0068] The picture book data generation unit 94 of the sales terminal 9 generates picture book data (step S26) by combining the captured image acquired in step S25 with the picture book template. The generated picture book data is displayed on the display 11 (step S27).

[0069] When user information such as the user's name and the delivery address for the picture book is entered into the sales terminal 9 (step S28), the terminal 9 transfers the picture book data and user information to the printing factory (step S29).

[0070] In the above embodiment, when grouping the tag IDs of multiple users who have come as a group, a priority may be set. For example, if a family comes, the tag ID of the tag T distributed to the child may be set to "high" priority, and the priority may be included in the group information. The shooting control device 3 may determine to take a picture if at least one of the high-priority users is in the shooting area R1, even if the other users are not in the shooting area R1.

[0071] The shooting control device 3 may decide not to take a picture if there are no users with "high" priority in the shooting area R1, even if other users are in the shooting area R1. The number of users with "high" priority can be freely set within the group.

[0072] The analysis unit 27 of the management device 20 may assign a higher score if a user with higher priority has a good expression.

[0073] In the above embodiment, an example was described in which the code information printed on the tag T is read by readers 5 and 7 at the entrance and exit reception to obtain the tag ID. However, the tag ID may also be obtained using communication equipment similar to the positioning sensor 2.

[0074] Figure 1 shows an example where one camera 1 is installed at each attraction, but multiple camera 1s may be installed, and a shooting area R1 and a shooting stop area R2 may be set for each camera 1. The shape and size of the shooting area R1 and the shooting stop area R2 can be arbitrarily set for each camera 1.

[0075] In the above embodiment, an example of selling picture books using captured images was described, but it is also possible to sell printed materials such as composite images created by combining captured images with templates other than picture books, or printed materials with the captured images themselves printed on paper. Alternatively, a kiosk terminal for selling printed materials may be installed. The reader of the kiosk terminal reads the code information of the ticket and obtains the group ID. The kiosk terminal receives the captured image corresponding to the group ID from the management device 20 and displays it on the touch panel. The user selects their favorite image from the images displayed on the touch panel. After payment is received, the kiosk terminal prints the selected image or the composite image using the selected image using its built-in printer.

[0076] In the above embodiment, the analysis unit 27 of the management device 20 may analyze the captured image, estimate the head posture (head orientation) of the subject user, and calculate a score based on the estimation result. An example of a method for estimating head posture will be explained with reference to the flowchart shown in Figure 8.

[0077] First, the analysis unit 27 performs resizing of the captured image for face detection processing (step S31). The image size of the captured image affects the processing time of the subsequent face detection process. Generally, a smaller image size shortens the processing time of face detection. However, if the image size is too small, the accuracy of face detection decreases. Therefore, the presence or absence of resizing processing and the amount of resizing are set according to the shooting conditions.

[0078] Resizing is not limited to making images smaller; images may also be enlarged. If a machine learning model for face detection is used in the subsequent face detection process, the image may be scaled to match the size of the model.

[0079] Next, the analysis unit 27 performs face detection on the resized captured image (step S32). If multiple faces are present in the captured image, each face is detected. When using a machine learning model for face detection, the face detection model is loaded and the captured image is input to detect the face region. The machine learning model for face detection is not limited and can be RetinaFace, MTCNN, BlazeFace, etc.

[0080] Face detection may be performed using image processing libraries such as DLib, OpenCV, or InsightFace, instead of using machine learning models.

[0081] The face detection process yields the coordinates of the face regions in the captured image and the face detection confidence score corresponding to each face region.

[0082] Next, the analysis unit 27 filters out areas from the face region detected in step S32 that have a low detection confidence level and are likely to be areas other than faces (step S33). Face regions with a detection confidence score below a predetermined threshold are not subjected to subsequent head pose estimation processing (step S34_No). The threshold is set according to the shooting conditions, etc. Face regions with a face detection confidence score higher than the predetermined threshold are subject to subsequent head pose estimation processing (step S34_Yes).

[0083] Next, the analysis unit 27 expands the face region (step S35). For example, as shown in Figure 9, the face region detected in step S32 only includes the vicinity of the face, so the face region is expanded to include the entire head. This is expected to improve the accuracy of the head pose estimation process.

[0084] Next, the analysis unit 27 crops the expanded face region from the original image (step S36).

[0085] Next, the analysis unit 27 performs resizing of the cropped face region image (step S37). The size of the face region image affects the processing time of the subsequent head pose estimation process. Generally, the smaller the image size, the shorter the processing time for head pose estimation. However, if the image size is too small, the accuracy of head pose estimation will decrease. Whether or not resizing is performed and the amount of resizing are set according to the shooting conditions.

[0086] Next, the analysis unit 27 performs head pose estimation on the resized face region image (step S38). For example, the analysis unit 27 performs head 3-axis estimation. When using a machine learning model for head 3-axis estimation, the head pose is estimated by loading the head 3-axis estimation model and inputting the face region image. The machine learning model for head 3-axis estimation is not particularly limited, and models such as 6DRepNet, head-pose-estimation, and DMHad can be used.

[0087] If the face detection process in step S32 detects the landmark coordinates of the face using an image processing library, this information may be used to perform head 3-axis estimation.

[0088] Through three-axis head estimation, the rotation angles of the three axes—yaw, pitch, and roll—are obtained as head posture parameters, as shown in Figure 10.

[0089] If there are any unprocessed face regions (step S39_No), the process returns to the face detection score filtering in step S33 and is repeated. Once all face regions detected from the captured image have been processed (step S39_Yes), the head pose estimation process is terminated.

[0090] The head posture estimation process described above allows us to obtain head posture parameters for each subject in the captured image.

[0091] The analysis unit 27 calculates a score for the captured image using head posture parameters. For example, a range of head posture parameters that result in a high score is pre-set for each scene (each attraction or each camera). If the head posture parameters fall within the set range, the captured image is assigned a high score; if the head posture parameters fall outside the set range, the captured image is assigned a low score. This ensures that the image with the highest score is the optimal image for each scene.

[0092] For example, in scenes where you want a higher score when the subject is facing the camera, you would set -20° ≤ Yaw ≤ 20° and -20° ≤ Pitch ≤ 20°. If the head posture parameters of the subjects in the captured image fall within this range, the score coefficient for each subject is set to 1; otherwise, the score coefficient is set to 0. As a result, the more subjects there are facing the camera, the higher the score of the captured image will be.

[0093] When the recording device 1 records a video, it may perform head pose estimation processing on each of the multiple images (frames) included in the video data, calculate a score, and extract the image (frame) with the highest score.

[0094] In the above embodiment, a configuration was described in which the tag T and the positioning sensor 2 perform UWB communication, but a sensor having a position detection accuracy comparable to that of UWB may also be used.

[0095] The present invention is not limited to the embodiments described above, and the components can be modified and implemented in practice without departing from the spirit of the invention. Furthermore, various inventions can be formed by appropriately combining the multiple components disclosed in the embodiments. For example, some components may be deleted from all the components shown in the embodiments. Moreover, components from different embodiments may be appropriately combined. For example, the following invention is provided.

[0096] [1] An image capture system comprising: a plurality of positioning sensors that communicate with a tag held by a user using UWB (Ultra-Wide-Band) communication to calculate the distance to the tag and the direction of the tag, and detect the tag ID assigned to the tag; a registration terminal that groups the tag IDs of the tags and registers them together with the group ID; and a shooting control device that uses the calculation results of the plurality of positioning sensors to detect the location of the user holding the tag, and when the user is within a predetermined shooting area, controls the shooting device to take a picture, and saves the captured image linked to the group ID. [2] The image capture system according to [1], wherein the registration terminal groups the tag IDs of the tags held by each of the plurality of users and registers them together with the group ID. [3] The image capture system according to [2], wherein the shooting control device takes a picture when at least one of the plurality of users corresponding to the grouped tag IDs is within the shooting area. [4] The image capture system according to [2] or [3], wherein the shooting control device takes a picture when all of the plurality of users corresponding to the grouped tag IDs are within the shooting area. [5] The image shooting system according to any one of [1] to [4], wherein the shooting control device determines not to take a picture if another user possessing the tag is in a predetermined shooting cancellation area, even if the user is in the shooting area. [6] The image shooting system according to [5], wherein the shooting cancellation area is located between the shooting device and the shooting area. [7] The image shooting system according to [5] or [6], wherein the shooting cancellation area is located behind the shooting area as seen from the shooting device. [8] The image shooting system according to any one of [1] to [4], wherein the shooting control device determines not to take a picture if another user belonging to a different group from the user in the shooting area is in a predetermined shooting cancellation area. [9] The image shooting system according to [2], wherein the registration terminal sets a high priority to the tag ID of the tag possessed by at least one of the multiple users, and the shooting control device takes a picture if the user corresponding to the high priority is in the shooting area.

[10] The image capture system according to any one of [1] to [9], further comprising a management device that receives the captured images, the tag ID of the tag held by a user in the shooting area, the group ID including the tag ID, and the user's location information from the shooting control device, and saves the captured images for each group, wherein the management device performs image analysis of the captured images, calculates a score, and saves the calculated score associated with the captured images.

[11] The image capture system according to

[10] , wherein the management device calculates a score for the captured images using the distance between the location of a user in the shooting area and a predetermined landmark.

[12] The image capture system according to

[10] or

[11] , wherein the management device estimates the head posture of the user in the captured images and calculates the score based on the estimation result.

[13] The image capture system according to

[12] , wherein the management device performs three-axis head estimation, determines the rotation angles of the three axes, yaw, pitch, and roll, as head posture parameters, and calculates the score using the head posture parameters.

[14] The image capture system according to

[13] , wherein the management device determines whether the head posture parameter falls within a range predetermined for each scene when calculating the score.

[15] The image capture system according to any one of

[12] to

[14] , wherein the management device detects the user's face region image from the captured image and estimates the head posture using the face region image.

[16] The image capture system according to

[15] , wherein the entire head of the user is included in the face region image.

[17] The image capture system according to any one of

[10] to

[16] , further comprising a sales terminal that acquires a captured image from the management device in which the score is greater than or equal to a predetermined value, and synthesizes the acquired captured image into a predetermined template to generate a composite image.

[18] The image capture system according to

[17] , wherein the sales terminal synthesizes the captured image acquired from the management device into a picture book template to generate picture book data.

[19] The image capture system according to

[18] , wherein the shooting area and the picture book template into which the captured image is synthesized are associated.

[20] The image capture system according to

[18] or

[19] , wherein the sales terminal synthesizes a default image onto the picture book template if there are no captured images with a score equal to or greater than a predetermined value to be synthesized onto the picture book template.

[21] The image capture system according to any one of [1] to

[20] , wherein the registration terminal obtains the tag ID of the tag using a communication device having the same configuration as the positioning sensor and registers it together with the group ID.

[0097] 1. Imaging device 2. Positioning sensor 3. Imaging control device 4. First registration terminal 6. Second registration terminal 9. Sales terminal T tag

Claims

1. An image capture system comprising: a plurality of positioning sensors that communicate with a tag held by a user via UWB (Ultra-Wide-Band) to calculate the distance to the tag and the direction of the tag, and detect the tag ID assigned to the tag; a registration terminal that groups the tag IDs of the tags and registers them together with the group ID; and a shooting control device that uses the calculation results of the plurality of positioning sensors to detect the location of the user holding the tag, controls the shooting device to take a picture when the user is within a predetermined shooting area, and saves the captured image linked to the group ID.

2. The image capture system according to claim 1, wherein the registration terminal groups the tag IDs of the tags possessed by each of the multiple users and registers them together with the group ID.

3. The image capture system according to claim 2, wherein the shooting control device performs shooting when at least one of the plurality of users corresponding to the grouped tag IDs is within the shooting area.

4. The image capture system according to claim 3, wherein the shooting control device performs shooting when all of the multiple users corresponding to the grouped tag IDs are within the shooting area.

5. The image capture system according to claim 1, wherein the shooting control device determines not to take a picture if another user possessing the tag is in a predetermined shooting cancellation area, even if the user is in the shooting area.

6. The image acquisition system according to claim 5, wherein the shooting cancellation area is located between the shooting device and the shooting area.

7. The image acquisition system according to claim 5, wherein the shooting cancellation area is located behind the shooting area as viewed from the shooting device.

8. The image capture system according to claim 1, wherein the shooting control device determines not to take a photograph if another user belonging to a different group from the user in the shooting area is in a predetermined shooting cancellation area.

9. The image capture system according to claim 2, wherein the registration terminal sets a high priority to the tag ID of the tag held by at least one of the multiple users, and the shooting control device takes a photograph when the user corresponding to the high priority is within the shooting area.

10. The image capture system according to claim 1, further comprising a management device that receives the captured images, the tag ID of the tag held by a user in the shooting area, the group ID including the tag ID, and the user's location information from the shooting control device, and saves the captured images for each group, wherein the management device performs image analysis of the captured images, calculates a score, and saves the calculated score associated with the captured images.

11. The image capture system according to claim 10, wherein the management device calculates a score for the captured image using the distance between the position of the user within the shooting area and a predetermined landmark.

12. The image acquisition system according to claim 10, wherein the management device estimates the user's head posture in the captured image and calculates the score based on the estimation result.

13. The image acquisition system according to claim 12, wherein the management device performs three-axis estimation of the head, determines the rotation angles of the three axes, yaw, pitch, and roll, as head posture parameters, and calculates the score using the head posture parameters.

14. The image acquisition system according to claim 13, wherein the management device determines whether the head posture parameters fall within a range predetermined for each scene when calculating the score.

15. The image acquisition system according to claim 12, wherein the management device detects the user's face region image from the captured image and estimates the head posture using the face region image.

16. The image acquisition system according to claim 15, wherein the facial region image includes the entire head of the user.

17. The image capture system according to claim 10, further comprising a sales terminal that acquires captured images with a score equal to or greater than a predetermined value from the management device, and synthesizes the acquired captured images onto a predetermined template to generate a composite image.

18. The image capture system according to claim 17, wherein the sales terminal generates picture book data by combining the captured image acquired from the management device with a picture book template.

19. The image capture system according to claim 18, wherein the shooting area and the picture book template for compositing the captured images are associated.

20. The image capture system according to claim 19, wherein if there are no captured images with a score equal to or greater than a predetermined value to be composited onto the picture book template, the sales terminal composites a default image onto the picture book template.

21. The image capture system according to claim 1, wherein the registration terminal obtains the tag ID of the tag using a communication device having the same configuration as the positioning sensor and registers it together with the group ID.

Images