Program, subject image extraction device, and subject image extraction system

Abstract

To provide a program for imaging a subject moving at a high speed and obtaining a clear image of the subject, a subject image extraction apparatus, and a subject image extraction system.SOLUTION: A program causes a computer to function as: person area estimation means 21 for processing each of captured images consecutively acquired by imaging means to estimate a person area which is an area in which a person is located; seat area estimation means 22 which estimates a seat area including the person area; overlap confirmation means 23 which confirms whether the seat area overlaps a set area which is a predetermined area in a captured image; area distance calculation means 24 which calculates, when the seat area overlaps the set area, an area distance which is a distance between predetermined points in the seat area and the set area; and subject image extraction means 25 which extracts, as a subject image, a captured image having the minimum area distance, from an overlap image group which is a group of consecutive captured images of which the seat area and the set area overlap each other.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to a program, a subject image extraction device, and a subject image extraction system.

Background Art

[0002] Conventionally, in amusement parks, amusement parks, race courses, etc., a subject moving at high speed has been photographed with a camera. Various techniques have been proposed as techniques for extracting a suitable image from a subject moving at high speed. Among them, a method using image analysis of camera images and detection based on movement and human characteristics has also been disclosed (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the above conventional technology has a problem that the equipment becomes complicated, such as a separate sensor being required as a shooting trigger.

[0005] Therefore, an object of the present disclosure is to provide a program, a subject image extraction device, and a subject image extraction system capable of photographing a subject moving at high speed and obtaining an image in which the subject is suitably photographed.

Means for Solving the Problems

[0006] To solve the above problems, in the present disclosure, person region estimation means for processing each captured image continuously acquired by the imaging means and estimating a person region which is a region where a person is located, seat region estimation means for estimating a seat region including the person region, A means for checking whether there is any overlap between a predetermined area in the captured image, which is a set area, and the seat area. If the seat area and the setting area overlap, a region distance calculation means calculates the region distance, which is the distance between predetermined points in the seat area and the setting area. A subject image extraction means that extracts a subject image from a group of overlapping images, which is a group of consecutive images in which the seat area and the setting area overlap, the image with the smallest area distance. It provides programs to make computers function.

[0007] Furthermore, the program disclosed herein is The aforementioned seat is a seat that accommodates multiple people, The seat area estimation means may estimate a seat area as a region that includes the areas of multiple adjacent people.

[0008] Furthermore, in the program of this disclosure, The aforementioned seat area estimation means is The method may determine the line segments connecting the centroids of the multiple adjacent person regions, and if the angle between these line segments and the horizontal direction of the captured image is less than or equal to a predetermined angle, the multiple adjacent person regions may be estimated as seating areas.

[0009] Furthermore, in the program disclosed herein, The aforementioned photograph shows a vehicle with multiple rows of seats arranged in the direction of travel. The subject image extraction means may identify the row order of the seats corresponding to each subject image based on the interval between a certain subject image and the subject image immediately preceding it, from among the subject images extracted from each group of duplicate images.

[0010] Furthermore, in the program of this disclosure, The subject image extraction means compares the region distance of each of the captured images in the duplicate image group with the region distance of the immediately preceding captured image, and if the region distance of the later captured image is greater than the region distance of the immediately preceding captured image, it extracts the immediately preceding captured image as the subject image. The person region estimation means may also use the captured image of a location that has moved backward by a predetermined interval in the order of shooting as the next processing target.

[0011] Furthermore, in this disclosure, A person region estimation means that processes each captured image obtained sequentially by the shooting means to estimate the person region, which is the area where a person is located, A seat area estimation means for estimating a seat area including the aforementioned person area, A means for checking whether there is any overlap between a predetermined area in the captured image, which is the setting area, and the seat area, If the seat area and the setting area overlap, the area distance calculation means calculates the area distance which is the distance between predetermined points in the seat area and the setting area. A subject image extraction means extracts a subject image from a group of overlapping images, which is a series of images in which the seat area and the setting area overlap, the image with the smallest area distance. The present invention provides a subject image extraction device having the following characteristics.

[0012] Furthermore, in this disclosure, A subject image extraction system comprising: a shooting means for continuously capturing images; a subject image extraction device for acquiring captured images captured by the shooting means; and a user terminal for acquiring subject images extracted by the subject image extraction device, The subject image extraction device is A person region estimation means that processes each captured image acquired by the aforementioned shooting means to estimate the person region, which is the area where a person is located. A seat area estimation means for estimating a seat area including the aforementioned person area, A means for checking whether there is any overlap between a predetermined area in the captured image, which is the setting area, and the seat area, When the seat area and the setting area overlap, area distance calculation means for calculating an area distance that is the distance between predetermined points of the seat area and the setting area; Subject image extraction means for extracting, as a subject image, a captured image in which the area distance is minimized from a group of overlapping images that are a group of consecutive captured images in which the seat area and the setting area overlap; Provided is a subject image extraction system having the above.

Effect of the Invention

[0013] According to the present disclosure, it is possible to provide a program, a subject image extraction device, and a subject image extraction system that can photograph a subject moving at high speed and obtain a suitable still image.

Brief Description of the Drawings

[0014] [Figure 1] It is a diagram showing the configuration of a subject image extraction system according to an embodiment of the present disclosure. [Figure 2] It is a functional block diagram showing details of the processing device 11. [Figure 3] It is a flowchart showing an outline of the processing of a subject image extraction system according to an embodiment of the present disclosure. [Figure 4] It is a diagram showing the relationship between the imaging device 12, the imaging object, and the imaging angle of view. [Figure 5] It is a diagram showing the relationship between the person area and the captured image. [Figure 6] It is a diagram showing the relationship between the seat area and the captured image. [Figure 7] It is a diagram showing a case where a total of four person areas, two below and two above, are estimated in one captured image. [Figure 8] It is a diagram showing a seat area estimated when the angle formed by the line segment connecting the centers of gravity of the person areas and the horizontal direction of the image is less than or equal to a predetermined angle. [Figure 9] It is a diagram showing the setting area and the seat area in the captured image. [Figure 10]This diagram shows the line segment connecting the centers of the setting area and the seat area in the captured image. [Figure 11] This diagram shows the relationship between the distance between the centers of the seat area and the setting area in each captured image. [Figure 12] This figure shows the relationship between each captured image and the distance between predetermined points in the seat area and the set area, when there are empty seats along the way. [Figure 13] This diagram shows the cropping process for the subject image. [Figure 14] This image shows an inpainting of a subject image. [Figure 15] This figure shows an example of its application. [Modes for carrying out the invention]

[0015] Preferred embodiments of this disclosure will be described in detail below with reference to the drawings. <System Configuration> Figure 1 shows the configuration of a subject image extraction system according to one embodiment of the present disclosure. In Figure 1, 10 is the subject image extraction system, 11 is the processing unit, 12 is the imaging device, 13 is the display device, 14 is the printer, 15 is the image data output unit, and 30 is the user terminal. Of these, the processing unit 11, the display device 13, the printer 14, and the image data output unit 15 constitute the subject image extraction device 40.

[0016] The processing unit 11 is a device that performs various calculations in the subject image extraction system 10, and is realized by incorporating a program to realize the functions of the subject image extraction system 10 into a computer having a CPU, main memory, non-volatile storage device, etc. The computer that realizes the processing unit 11 may have a processor such as a GPU. The processing unit 11 is connected to the shooting device. The shooting device 12 is a shooting means that takes pictures and acquires the captured images as image data, and is realized by a video shooting camera equipped with an image sensor such as a CCD. Video is obtained by continuously shooting still images. In this specification, when simply referred to as "image," it means a still image.

[0017] The display device 13 is a display means that displays information based on the results processed by the processing device 11, and is implemented using general-purpose display equipment such as a liquid crystal display or an organic EL display. The printer 14 and image data output unit 15 are image output means that output images. The printer 14 prints the image onto paper or the like. The image data output unit 15 is a data output device that outputs the image as data. The image data output unit 15 may output the image as a captured image, which is a still image extracted as a subject image, or it may output a series of captured images, including the subject image and the captured images before and after it, as a video.

[0018] The components of the subject image extraction system 10 may be partially or entirely arranged within a single housing. In this embodiment, the processing unit 11, display device 13, printer 14, and image data output unit 15 that constitute the subject image extraction device 40 are housed within a single housing and installed as an image provision terminal. In this embodiment, customers who appear in the output subject images can output the subject images in which they appear as subjects from the image provision terminal (subject image extraction device 40). For example, the image data output unit 15 can output the subject images to a user terminal 30 such as a smartphone via wireless communication.

[0019] Figure 2 is a functional block diagram showing the details of the processing unit 11. In this embodiment, the processing unit 11 is realized by incorporating a dedicated program into a general-purpose computer. As shown in Figure 2, the processing unit 11 includes a person area estimation means 21, a seat area estimation means 22, an overlap confirmation means 23, an area distance calculation means 24, a subject image extraction means 25, and an image processing means 26.

[0020] The person region estimation means 21 is a means for estimating the region where a person is located by performing image analysis on the captured image acquired by the shooting device 12. This is realized by a CPU, GPU, etc., reading and executing a program or trained model stored in a non-volatile memory device. The seat area estimation means 22 is a means for estimating a seat area that includes the area of ​​a person, and is realized by the CPU reading and executing a program stored in a non-volatile memory device.

[0021] The overlap confirmation means 23 is a means for confirming whether or not there is an overlap between a predetermined area in the captured image, which is the setting area, and the seat area, and is implemented by the CPU reading and executing a program stored in a non-volatile memory device. The area distance calculation means 24 is a means for calculating the distance between predetermined points in the seat area and the setting area when the seat area and the setting area overlap, and is implemented by the CPU reading and executing a program stored in a non-volatile memory device. The subject image extraction means 25 is a means for extracting a captured image from a group of overlapping images, which is a group of consecutive captured images in which the seat area and the setting area overlap, the image with the smallest area distance (the distance between predetermined points in the seat area and the setting area), as a suitable subject image, and is implemented by the CPU reading and executing a program stored in a non-volatile memory device.

[0022] The processing unit 11 is equipped with an input unit such as a keyboard or mouse, and accepts instructions from the user. Based on the user's instructions, various settings can be set.

[0023] <Processing operation> Next, the processing operation of the subject image extraction system according to this embodiment will be described along with the subject image extraction method according to this embodiment. Figure 3 is a flowchart showing the processing overview of the subject image extraction system according to this embodiment.

[0024] First, the subject image extraction system is activated and processing begins. Then, the shooting device 12 continuously takes pictures, and the image data obtained from the shooting is captured as the captured image (step S1). Figure 4 is a diagram showing the relationship between the shooting device 12, the subject to be photographed, and the shooting angle of view. In the example in Figure 4, the subjects to be photographed are a ride such as a roller coaster, which is an example of a ride attraction, and the people riding on the ride. In this embodiment, the vehicle to be photographed is assumed to have a structure in which the seats can accommodate multiple people. However, it is not necessary for multiple people to be accommodated in the seats. In this state, a shooting angle of view that can photograph the people riding on the ride is set, and the shooting device 12 is installed in a position that can photograph with that shooting angle of view. The shooting angle of view is fixed in a predetermined position.

[0025] The subject image extraction system according to this embodiment is particularly suitable for photographing people as subjects in situations where multiple rows of seats are arranged along the direction of travel, such as on a roller coaster. The shooting device 12 takes pictures at predetermined intervals within the shooting angle shown in Figure 4 and acquires still images. The captured images are obtained at predetermined time intervals. Therefore, when the acquired captured images are displayed in sequence, they appear as a video. For this reason, it can be said that the shooting device 12 is acquiring moving images. The acquired captured images are passed to the processing device 11 for predetermined processing. The shooting device 12 takes pictures at a predetermined rate (fps: frames (images) / second), acquires the captured images, and sends them to the processing device 11. The predetermined rate can be set arbitrarily. In this embodiment, it is set to 60fps. The acquisition of captured images by the shooting device 12 is performed continuously while the subject image extraction system is running.

[0026] When processing begins in the subject image extraction system, the processing unit 11 performs analysis and other processing on the captured image. The analysis rate of the captured image in the processing unit 11 may be the same as or different from the shooting rate of the shooting device 12. For example, if the shooting rate is 60fps as described above, the analysis rate in the processing unit 11 may be the same as the shooting rate, 60fps, or the rate may be reduced to, for example, 40fps.

[0027] In the processing unit 11, the person region estimation means 21 performs person region estimation (step S2). Specifically, the person region estimation means 21 analyzes the captured image and identifies a predetermined area as a person region. Figure 5 is a diagram showing the relationship between the person region and the captured image. As shown in Figure 5, if two people are detected, two person regions are estimated on the captured image. In the example in Figure 5, a rectangular person region is estimated so as to include the faces of the people. The method used by the person region estimation means 21 to identify the person region is not particularly limited, but a trained model obtained by machine learning or a rule-based method can be used. The shape of the person region is not particularly limited, but in this embodiment, as shown in Figure 5, the person region is rectangular. In Figure 5, the person region is shown as a solid rectangle.

[0028] Once the person area estimation means 21 estimates the person area, the seat area estimation means 22 estimates the seat area (step S3). The seat area estimation means 22 estimates the seat area based on parameters set according to the shooting angle and the type of vehicle. Specifically, it obtains a bounding rectangle that encompasses multiple adjacent person areas from the person area estimation means 21. Then, it estimates the area obtained by taking margins on the top, bottom, left, and right sides from the obtained bounding rectangle as the seat area. The distance (number of pixels) to neighboring person areas and the margin width (number of pixels) on the top, bottom, left, and right are parameters set according to the shooting angle and the type of vehicle. Such parameters can be set by inputting them to the processing unit 11 via the instruction input unit.

[0029] Figure 6 shows the relationship between the seat area and the captured image. When two person areas are close together, the seat area is set as an area that encompasses both person areas, as shown in Figure 6. The shape of the seat area is not particularly limited, but in this embodiment, the seat area is rectangular, as shown in Figure 6. In Figure 6, the person areas are shown as solid rectangles, and the seat area is shown as dashed rectangles.

[0030] Figure 7 shows a case where a total of four person regions are estimated in a single captured image: two at the bottom and two at the top. In the example in Figure 7, the two lower person regions indicate the front seats of the vehicle, and the two upper person regions indicate the rear seats of the vehicle. Therefore, it is preferable to estimate one seat region that includes the two lower person regions and one seat region that includes the two upper person regions.

[0031] In an example like Figure 7, if proximity is determined simply by the distance between the person regions, a seat region including both the lower and upper person regions will be set. As a result, the actual seat will not be set as a seat region. Therefore, in this embodiment, a line segment is found connecting the centroids of the person regions, and if the angle between this line segment and the horizontal direction of the image (horizontal direction: left and right direction in Figure 7) is less than or equal to a predetermined angle, the seat region including those two person regions is estimated. This predetermined angle can be set arbitrarily, but it is preferably 20 degrees or less, and more preferably 10 degrees or less. This is because the shooting angle is usually set so that people are photographed in a line, so the closer the predetermined angle is to 0 degrees, the better the people will be photographed. Figure 8 shows the estimated seat region when the angle between the line segment connecting the centroids of the person regions and the horizontal direction of the image is less than or equal to the predetermined angle. In Figure 8 as well, the person regions are shown as solid rectangles, and the seat regions as dashed rectangles.

[0032] Once the seat area estimation means 22 estimates the seat area, the overlap confirmation means 23 checks whether the seat area and the setting area overlap (step S4). Specifically, whether the seat area and the setting area overlap is determined by whether the seat area and the setting area contain at least one identical pixel. The setting area is set in a predetermined area of ​​the captured image. The setting area can be set at any position. However, it is preferable to set the setting area in a position suitable for the person who is the subject.

[0033] For example, by setting the setting area to the center of the captured image, it becomes possible to distinguish between captured images where the person area is located away from the center and captured images where the person area is located close to the center. The setting area may be an area directly specified by the user using the instruction input unit. Alternatively, the setting area may be an area set over a predetermined range from the center point of the captured image. Figure 9 shows the setting area and the person area in a captured image. In Figure 9, the seat area is shown as a dashed rectangle, and the setting area is shown as a dotted rectangle. In the example in Figure 9, the setting area and the person area overlap.

[0034] If the overlap confirmation means 23 determines that the seat area and the setting area overlap, the area distance calculation means 24 calculates the area distance, which is the distance between predetermined points in the seat area and the setting area (step S5). The predetermined points in the area can be set arbitrarily. However, it is preferable that the predetermined points accurately reflect the proximity of the seat area and the setting area. For this reason, the center or centroid of the area may be set as the predetermined points. In this embodiment, the predetermined point is set as the center of the area. Figure 10 shows a line segment connecting the centers of the setting area and the person area in the captured image. The length of the line segment connecting the centers shown in Figure 10 is calculated as the area distance between the setting area and the seat area. The processing in steps S2 to S5 is performed for each captured image. As a result, for each captured image in which the seat area and the setting area overlap, the distance between predetermined points in the seat area and the setting area is obtained as the area distance. Therefore, it is not necessary to calculate the area distance for captured images in which the seat area and the setting area do not overlap.

[0035] After processing steps S1 to S5 has been performed on each captured image, the subject image extraction means 25 extracts a suitable captured image as the subject image from among the analyzed captured images (step S6). Specifically, it identifies and extracts a captured image in which a person is suitably depicted from among a plurality of consecutively acquired captured images. Typically, it is preferable that the person, who is the subject, is depicted in a specific position within the shooting angle. For example, when photographing a roller coaster at a predetermined position, a preferred area in the shooting angle where the person is located is identified. In this embodiment, a predetermined area within the entire area of ​​the captured image is set as a set area based on the shooting angle of the shooting device 12. Therefore, the closer the position of the person is to the set area, the more suitable the image is considered to be.

[0036] Therefore, in this embodiment, a suitable captured image is identified based on the region distance, which is the distance between predetermined points in the seat region and the setting region. Specifically, first, a group of consecutive images in which the seat region and the setting region overlap is identified as a group of overlapping images. Therefore, if there is a captured image in which the seat region and the setting region do not overlap, the group of overlapping images is interrupted at that point. Figure 11 is a diagram showing the relationship between each captured image and the region distance, which is the distance between predetermined points in the seat region and the setting region. Of these, Figure 11(a) is a graph showing the region distance, which is the distance between predetermined points in the seat region and the setting region, for each captured image. In Figure 11(a), the horizontal axis represents the order in which consecutive captured images (frames) are taken, and the vertical axis represents the region distance, which is the distance between predetermined points in the seat region and the setting region. In addition, along the horizontal axis, the shaded area represents a captured image in which the seat region and the setting region overlap, and the unshaded area represents a captured image in which the seat region and the setting region do not overlap. That is, the shaded area represents a group of overlapping images, and the unshaded area represents a group of non-overlapping images.

[0037] In the example in Figure 11(a), duplicate and non-duplicate image sets appear alternately. This indicates that non-duplicate image sets, where the seat area is far from the designated area, are interspersed with duplicate image sets, where the seat area is close to the designated area. Depending on the attraction's configuration, some images in the non-duplicate image sets may not show the seat area. Images in the duplicate image sets often show the seat area close to the designated area and are therefore desirable images.

[0038] As shown in Figure 11(a), in the duplicate image group, the distance graph has a local minimum. Therefore, in each duplicate image group, it is possible to identify the captured image in which the distance between predetermined points in the seat area and the setting area is minimized. In the captured image in which the distance between predetermined points in the seat area and the setting area is minimized, it is considered that the seat area is captured in the most desirable position. For this reason, in this embodiment, the subject image extraction means 25 extracts the captured image in which the distance between predetermined points in the seat area and the setting area is minimized as the subject image in one duplicate image group. The extracted subject image is a captured image in which the subject is captured in a suitable manner.

[0039] In each set of overlapping images, subject images for each seat can be extracted by extracting the image in which the distance between predetermined points in the seat area and the setting area is minimized. Figure 11(b) shows the image extracted from the three sets of overlapping images shown in Figure 11(a) in which the distance between predetermined points in the seat area and the setting area is minimized. As shown in Figure 11(b), the subject images extracted from the three sets of overlapping images each depict people in the first, second, and third rows of seats in a suitable state for use as subjects. According to this embodiment, it is possible to obtain subject images for each seat when photographing a vehicle with multiple rows of seats in the direction of travel.

[0040] The example in Figure 11 illustrates the case where there are people in the first, second, and third rows. Next, we will explain the case where there are no people in the intermediate rows. Figure 12 shows the relationship between each captured image and the distance between predetermined points in the seat area and the setting area when there are empty seats in between. Compared to Figure 11, in Figure 12, the extraction interval between the second and third subject images is considerably larger than the extraction interval between the first and second subject images. The extraction interval can be converted into the number of captured images, i.e., the number of frames, between two subject images. Normally, in a high-speed attraction, each seat will reach a certain shooting point at the same interval. Therefore, if the extraction interval of subject images in which people are detected is large, it means that there were empty seats in between. In this embodiment, a standard extraction interval, which is the extraction interval (number of captured images) between adjacent seats, is set, and by comparing with this standard extraction interval, it is determined whether or not there were empty seats between subject images.

[0041] The standard extraction interval, which is the extraction interval (number of images taken) between adjacent seats, may be measured and set in advance. Alternatively, the minimum extraction interval among all extracted subject images may be used as the standard extraction interval. By comparing a certain extraction interval with the standard extraction interval, it is possible to identify how many empty seats were in between. For example, in the example in Figure 12, suppose the first and second subject images were extracted at the standard extraction interval. Then, suppose the extraction interval between the second and third subject images was approximately twice the standard extraction interval. In this case, it can be considered that there was an empty seat between the second and third subject images. If the second subject image was taken of a seat in the second row, then the third subject image is identified as having taken of a seat in the fourth row. In this way, it is possible to identify which row of seats each captured image depicts. In reality, the extraction interval is not necessarily an integer multiple of the standard extraction interval. Therefore, in this embodiment, the value obtained by dividing the extraction interval by the standard extraction interval is rounded to an integer value. Assuming that the number of columns in subsequent subject images is shifted by that integer value compared to the previous subject image, the number of columns in the subsequent subject image can be determined.

[0042] In this embodiment, in step S6, after the subject image is extracted, the image processing means 26 performs further processing on the subject image. Examples of processing on the subject image include cropping and inpainting. Cropping is the process of cutting out a portion of an image. In this embodiment, only the seat area can be cut out from the subject image to obtain a new cropped image. Figure 13 shows the cropping of the subject image. Figure 13(a) is the subject image before cropping, and Figure 13(b) is the cropped image after cropping. This cropped image can be output as data or printed and provided to the customer. In Figure 13(a), the cropping area is shown as a dashed rectangle.

[0043] Furthermore, inpainting is a process in which a portion of a subject image is repainted with other information. In this embodiment, if a person is detected outside the seating area, that portion is repainted with other information. This other information could be, for example, a picture of a character, or surrounding pixels. Figure 14 shows the inpainting of a subject image. Figure 14(a) is the subject image before inpainting, and Figure 14(b) is the processed image after inpainting. This processed image can be output as data or printed and provided to the customer. Note that the image processing means 26 is not an essential means, and it is not necessarily required to process the extracted subject image.

[0044] As described above, once the subject image is extracted, the image output means outputs the extracted subject image (step S7). Specifically, it outputs the subject image and its column order (information on which column it is). The captured image extracted as the subject image is output to an image output means such as a printer 14 or an image data output unit 15.

[0045] The output of the subject image to the image output means may be by directly outputting the subject image extracted by the subject image extraction means 25. Alternatively, the subject image extracted by the subject image extraction means 25 may be displayed on the display device 13, and after the user confirms the content, the subject image may be output to the image output means based on instructions from the instruction input unit. The image data output unit 15 may also transmit the image data to a user terminal 30, which is implemented as a smartphone, via a network.

[0046] The subject image extraction means 25 may extract multiple subject images from a single group of duplicate images. For example, it may extract the image with the smallest area distance and the images taken before and after it. In such a case, the image data output unit 15 displays the multiple subject images on the display device 13 or user terminal 30 to prompt the user to make a selection. The image data output unit 15 then transmits the subject image selected by the user to the user terminal 30. As described above, the subject images are associated with information about which row the seat is in. Therefore, if the user specifies the row number, the image data output unit 15 can also display the subject image corresponding to that row number on the display device 13 or user terminal 30.

[0047] <Application Examples> Next, an application example will be described. In the application example, the process of extracting the subject image in step S6 differs from the embodiment described above. Figure 15 shows an application example. In this system, analysis may be performed on each captured image as described above to determine the overlap between the seat area and the setting area, and the distance between predetermined points in the seat area and the setting area may be calculated. However, if analysis is performed on all captured images, the processing load on the system will increase, making high-speed processing difficult. Therefore, in the application example, measures are taken to reduce the processing load on the system. Specifically, as explained in Figure 13, instead of extracting the image with the smallest distance between the centers of the seat area and the setting area from all captured images in the overlapping image group as the subject image, the image captured immediately before the point in time when the distance between the centers of the seat area and the setting area begins to increase is extracted as the subject image. Then, the analysis of the captured images is performed from a point moved by a predetermined interval from that point. In other words, the captured image at a point moved by a predetermined interval backward in the shooting order is targeted for the next processing.

[0048] The predetermined interval can be the standard extraction interval multiplied by a predetermined ratio α. This ratio α can be set arbitrarily, but it is preferably 0.5 or greater, and more preferably 0.8 or greater. If the ratio α is too small, the number of captured images that are omitted from processing will be small, and the reduction in processing load will not be sufficient. If the ratio α is too large, there is a risk of failing to process captured images where the distance between the centers of the seat area and the setting area should be minimized. For example, if the ratio α = 0.9, 90% of captured images do not need to be processed, and the system's processing load is greatly reduced. Therefore, it will appear as if the number of captured images skipped is indicated by the rightward arrow in Figure 15.

[0049] While preferred embodiments of the present disclosure have been described above, the present disclosure is not limited to the above embodiments, and various modifications are possible. For example, in the above embodiments, a computer is used as the processing unit 11, and a dedicated program stored in the storage device is executed by the CPU to realize the person area estimation means, seat area estimation means, overlap confirmation means, area distance calculation means, subject image extraction means, and image processing means. However, each of these means may also be incorporated into hardware as an arithmetic circuit. [Explanation of symbols]

[0050] 10. Subject Image Extraction System 11. Processing Unit 12. Imaging device 13...Display device 14. Printer 15. Image data output section 21...Human region estimation means 22. Seating area estimation means 23. Method for checking for duplicates 24...Area distance calculation means 25...Subject image extraction means 26. Image processing methods 30. User terminals 40...Subject image extraction device

Claims

1. A person region estimation means that processes each captured image obtained sequentially by the shooting means to estimate the person region, which is the area where a person is located. A seat area estimation means that includes the aforementioned person area and estimates the seat area in which the seat is located. A means for checking whether there is any overlap between a predetermined area in the captured image, which is a set area, and the seat area. If the seat area and the setting area overlap, a region distance calculation means calculates the region distance, which is the distance between predetermined points in the seat area and the setting area. A subject image extraction means that extracts a subject image from a group of overlapping images, which is a group of consecutive images in which the seat area and the setting area overlap, the image with the smallest area distance. A program that makes a computer function.

2. The aforementioned seat is a seat that accommodates multiple people, The program according to claim 1, wherein the seat area estimation means estimates a region encompassing multiple adjacent person areas as a seat area.

3. The aforementioned seat area estimation means is The program according to claim 2, wherein a line segment is found connecting the centroids of the multiple adjacent person regions, and if the angle between that line segment and the horizontal direction of the captured image is less than or equal to a predetermined angle, the multiple adjacent person regions are estimated to be seat regions.

4. The aforementioned photograph shows a vehicle with multiple rows of seats arranged in the direction of travel. The program according to claim 1, wherein the subject image extraction means identifies the row order of the seats corresponding to each subject image based on the interval between a certain subject image and the subject image immediately preceding it, from among the subject images extracted from each group of duplicate images.

5. The subject image extraction means compares the region distance of each of the captured images in the duplicate image group with the region distance of the immediately preceding captured image, and if the region distance of the later captured image is greater than the region distance of the immediately preceding captured image, it extracts the immediately preceding captured image as the subject image. The program according to any one of claims 1 to 4, wherein the person region estimation means makes the captured image of a location moved backward by a predetermined interval in the order of shooting the next processing target.

6. A person region estimation means that processes each captured image obtained sequentially by the shooting means to estimate the person region, which is the area where a person is located, A seat area estimation means that estimates a seat area in which a seat is located, including the aforementioned person area, A means for checking whether there is any overlap between a predetermined area in the captured image, which is the setting area, and the seat area, When the seat area and the setting area overlap, a region distance calculation means calculates the region distance, which is the distance between predetermined points in the seat area and the setting area. A subject image extraction means extracts a subject image from a group of overlapping images, which is a series of images in which the seat area and the setting area overlap, the image with the smallest area distance. A subject image extraction device having the following features.

7. A subject image extraction system comprising: a shooting means for continuously capturing images; a subject image extraction device for acquiring the captured images captured by the shooting means; and a user terminal for acquiring the subject images extracted by the subject image extraction device, The subject image extraction device is A person region estimation means that processes each captured image acquired by the aforementioned shooting means to estimate the person region, which is the area where a person is located. A seat area estimation means that estimates a seat area in which a seat is located, including the aforementioned person area, A means for checking whether there is any overlap between a predetermined area in the captured image, which is the setting area, and the seat area, When the seat area and the setting area overlap, a region distance calculation means calculates the region distance, which is the distance between predetermined points in the seat area and the setting area. A subject image extraction means extracts a subject image from a group of overlapping images, which is a series of images in which the seat area and the setting area overlap, the image with the smallest area distance. A subject image extraction system having the following features.

Images