Information processing device

Abstract

To provide an information processing device that performs processing so that a plurality of images captured by a moving body can be used for various purposes.SOLUTION: An information processing device has: an image acquisition section that acquires image data, including a plurality of images captured by an imaging device of a moving body and movement state information indicating a movement state of the moving body at capturing each of the plurality of images; an image evaluation section that evaluates a degree of usability of each of the plurality of images on the basis of the movement state information on each of the plurality of images; and an image extraction section that extracts at least one image out of the plurality of images on the basis of evaluation on the degree of usability of each of the plurality of images obtained by the image evaluation section.SELECTED DRAWING: Figure 1A

Description

[Technical field]

[0001] The present invention relates to an information processing apparatus for processing information about an image. [Background technology]

[0002] With the recent spread of digital imaging technology, imaging devices are now capable of performing imaging operations in cooperation with various devices. For example, imaging devices can add image information such as the imaging location and imaging direction to images by cooperating with a positioning device. This allows the imaging device to generate slideshow images that display images of a specific location in chronological order. For example, Patent Document 1 discloses an image display device that extracts images taken in the same direction from an unspecified number of images that are stored and have information on the imaging direction recorded thereon, and creates a slideshow. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2012-159895 A Summary of the Invention [Problem to be solved by the invention]

[0004] For example, a moving object such as a car or a bicycle can function as an information medium that captures images of the road conditions and records the situation information by mounting an imaging device on the moving object. Also, for example, a car, due to its characteristics as a moving object, can capture images at any point where it can travel.

[0005] On the other hand, when images are extracted based on the image capturing position or image capturing direction and made into a slideshow as in Patent Document 1, a problem may occur in that images in the slideshow may contain a large amount of unnecessary information, such as an obstacle that existed near the image capturing device being captured in the slideshow image. For this reason, there are cases in which slideshow images containing information unnecessary for a specific purpose are generated.

[0006] The present invention has been made in consideration of the above-mentioned points, and one of its objectives is to provide an information processing device that processes multiple images captured by a moving object so that they can be used for various purposes. [Means for solving the problem]

[0007] The invention described in claim 1 is characterized by having an image acquisition unit that acquires image data including a plurality of images captured by an imaging device possessed by a moving body and movement state information indicating the movement state of the moving body at the time of capturing each of the plurality of images, an image evaluation unit that evaluates the usefulness of each of the plurality of images based on the movement state information for each of the plurality of images, and an image extraction unit that extracts at least one image from the plurality of images based on the evaluation of the usefulness of each of the plurality of images by the image evaluation unit. [Brief description of the drawings]

[0008] [Figure 1A] 1 is a block diagram of an information processing device according to a first embodiment. [Figure 1B] 1 is a diagram illustrating an example of the configuration of image data acquired by an information processing device from a moving object; [Diagram 2] 2 is a block diagram of an image evaluation unit in the information processing device according to the first embodiment. [Figure 3A] 5 is a diagram showing an example of evaluation criteria of an image evaluation unit in the information processing device according to the first embodiment. FIG. [Figure 3B] 11A and 11B are diagrams illustrating examples of image evaluation by an image evaluation unit. [Figure 4A] 2 is a block diagram of an image extraction unit in the information processing device according to the first embodiment. [Figure 4B] 11A and 11B are diagrams illustrating an example of an image extracted by an image extracting unit. [Diagram 5] FIG. 4 is a diagram illustrating an operation flow of the information processing device according to the first embodiment. [Figure 6] FIG. 11 is a block diagram of an information processing device according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The present invention will be described in detail below with reference to the embodiments. EXAMPLES

[0010] 1A is a block diagram showing the configuration of an information processing device 10 and moving objects 20 and 30 according to a first embodiment. In this embodiment, the information processing device 10 communicates with the moving objects 20 and 30 and acquires a plurality of images from the moving objects 20 and 30. In this embodiment, the information processing device 10 performs information processing on each of the plurality of images acquired from the moving objects 20 and 30. In this embodiment, the information processing device 10 is a server.

[0011] Each of the moving bodies 20 and 30 is, for example, an automobile, a motorcycle, a bicycle, a train, a ship, etc. equipped with an imaging device. The moving body 20 or 30 may be a pedestrian carrying a smartphone with an imaging function or a digital camera with a communication function.

[0012] Furthermore, the information processing device 10 may be connected to a moving body other than the moving bodies 20 and 30 and acquire images from the moving body. In the following, a case will be described in which each of the moving bodies 20 and 30 is a vehicle such as an automobile.

[0013] First, the vehicles 20 and 30 as the moving bodies 20 and 30 will be described. In this embodiment, the vehicles 20 and 30 are equipped with an imaging device (not shown) that images the surroundings of the vehicles 20 and 30, respectively. For example, the imaging device is a drive recorder mounted on each of the vehicles 20 and 30. The drive recorder as the imaging device, for example, constantly images the situation in front of the vehicle 20 or 30. Also, a large number of images are recorded as video on a recording medium connected to the imaging device.

[0014] Furthermore, the vehicles 20 and 30 are each equipped with a positioning device (not shown) that measures the positions of the vehicles 20 and 30. For example, the positioning device has a GPS (Global Positioning System) receiver and calculates latitude and longitude that indicate the positions of the vehicles 20 and 30. Furthermore, the positioning device calculates (estimates) the moving directions of the vehicles 20 and 30 based on, for example, changes in the positions of the vehicles 20 and 30.

[0015] Furthermore, the vehicles 20 and 30 are each equipped with a detection device (not shown) that detects the running state (moving state) of the vehicles 20 and 30. For example, the detection device includes a speed sensor and an acceleration sensor that detect the speed and acceleration of the vehicle 20 or 30. The detection device also includes a distance sensor that detects the inter-vehicle distance (inter-moving body distance) between the vehicle 20 or 30 and another vehicle (moving body) when the other vehicle exists in front of the vehicle 20 or 30. For example, the detection device has a millimeter wave radar.

[0016] Next, the information processing device 10 will be described. The information processing device 10 has an image acquisition unit 11 that acquires, from the vehicles 20 and 30, a plurality of images captured by the vehicles 20 and 30 as image data DT. The image data DT may be, for example, a collection of a plurality of image data each including one image, or may be one image data including a plurality of images.

[0017] 1B, the configuration of the image data DT acquired by the image acquisition unit 11 will be described. In this embodiment, the image data DT is a data set consisting of multiple image data DT1, DT2, ... each including one image IM, IM2, .... The image acquisition unit 11 acquires each of the image data DT1, DT2, ... and acquires an image group IM including multiple images IM1, IM2, ....

[0018] 1B, the image data DT records the point, direction (orientation), height, and date and time at which each of the images IM1, IM2, ... was captured as the image capturing point P, image capturing direction D, image capturing height H, and image capturing date and time T. The image data DT also records the speed of the vehicle 20 or 30 at the time when each of the images IM1, IM2, ... was captured as the vehicle speed (moving speed) V, and the distance to the vehicle in front as the inter-vehicle distance (distance between moving objects) S.

[0019] In this embodiment, the imaging devices mounted on the vehicles 20 and 30 generate image data DT in which, for example, the imaging point P, the imaging direction D, the imaging height H, and the imaging date and time T are assigned to the images IM1, IM2, ... as image information for each of the images IM1, IM2, .... For example, information on the imaging point P and the imaging direction D can be acquired from the positioning devices of the vehicles 20 and 30. The imaging height H can be acquired based on the initial setting (installation position setting) at the time of installation of the imaging device. In addition, the imaging date and time T can be acquired or corrected, for example, by receiving Internet time based on the built-in clock of the imaging device, or by using a received standard radio wave or GPS signal.

[0020] Furthermore, the imaging devices mounted on the vehicles 20 and 30 generate image data DT in which, for example, the vehicle speed V and the vehicle distance S are assigned to the images IM1, IM2, ... as running state information (movement state information) of the vehicles 20 and 30 at the time of capturing each of the images IM1, IM2, .... For example, the vehicle speed V and the vehicle distance S can be acquired from the detection devices of the vehicles 20 and 30.

[0021] The imaging devices of the vehicles 20 and 30 generate image data DT to which the above-mentioned image information and driving state information are added, and transmit the image data DT to the information processing device 10. The image acquisition unit 11 receives this image data DT from the vehicles 20 and 30, thereby acquiring each of the images IM1, IM2, ... and various information related thereto.

[0022] 1B, the image acquisition unit 11 can recognize that the image IM1 is an image captured at the time and date T1, at the point P1, in the direction (bearing) of the direction D1, and that the height of the imaging device from the ground is H1. The image acquisition unit 11 can also recognize that when the image IM1 was captured, a vehicle (e.g., the vehicle 20) was traveling at a speed V1, and that a vehicle ahead was located at a distance S1 away.

[0023] The information processing device 10 has an image evaluation unit (hereinafter simply referred to as the evaluation unit) 12 that evaluates the usefulness of each of the images IM1, IM2, ... based on driving state information (movement state information) of the vehicle 20 or 30 at the time when each of the images IM1, IM2, ... is captured.

[0024] In this embodiment, the evaluation unit 12 detects an imaged obstacle (hereinafter simply referred to as an obstacle) included in each of the images IM1, IM2, .... In this embodiment, the evaluation unit 12 calculates an occupancy rate of the obstacle in the image. Also, in this embodiment, the evaluation unit 12 evaluates the usefulness of each of the images IM1, IM2, ... based on the detected obstacle.

[0025] The information processing device 10 has an image extraction unit (hereinafter simply referred to as the extraction unit) 13 that extracts at least one image from each of the images IM1, IM2, ... based on the evaluation of the usefulness of each of the images IM1, IM2, ... by the evaluation unit 12. The extraction unit 13 selects, for example, an image having a predetermined usefulness or higher from each of the images IM1, IM2, ....

[0026] The information processing device 10 has an album creation unit 14 that creates an album in which the images extracted by the extraction unit 13 are rearranged in chronological order. For example, the album creation unit 14 rearranges, in chronological order, images taken at the same imaging point P and imaging direction D from at least one image extracted by the extraction unit 13. Then, the album creation unit 14 generates image data for the album including the rearranged images.

[0027] The information processing device 10 also has a storage unit 15 that stores processing information and processing results of the image acquisition unit 11, the evaluation unit 12, the extraction unit 13, and the album creation unit 14. The information processing device 10 also has a processing control unit 16 that controls the processing operations of the image acquisition unit 11, the evaluation unit 12, the extraction unit 13, the album creation unit 14, and the storage unit 15.

[0028] 2 is a block diagram showing a detailed configuration of the evaluation unit 12. In this embodiment, the evaluation unit 12 has an obstacle information acquisition unit 12A that acquires information about obstacles included in each of the images IM1, IM2, .... The obstacle information acquisition unit 12A acquires, for example, information about the shapes of obstacles that prevent the imaging of a subject to be imaged (for example, a building or a site) when viewed from various directions. For example, the obstacles are other moving objects on the road, such as vehicles and pedestrians, and the obstacle information acquisition unit 12A acquires information indicating these shapes as obstacle information.

[0029] Depending on the purpose of the image, vehicles may be the subject. For example, when images IM1, IM2, ... are used to investigate congestion or traffic volume, vehicles and the number of vehicles are surveyed objects and subjects. In this case, vehicles may be excluded from obstacles.

[0030] The evaluation unit 12 has an image analysis unit (hereinafter simply referred to as analysis unit) 12B that analyzes each of the images IM1, IM2, . . . based on the obstacle information acquired by the obstacle information acquisition unit 12A.

[0031] In this embodiment, analysis unit 12B has an obstacle detection unit (hereinafter referred to as a detection unit) 12B1 that detects obstacles and their areas included in each of images IM1, IM2, .... Analysis unit 12B also has an occupancy rate calculation unit (hereinafter simply referred to as a calculation unit) 12B2 that calculates an obstacle occupancy rate, which is the ratio of obstacles detected by detection unit 12B1 to each of images IM1, IM2, ....

[0032] The evaluation unit 12 has a map information acquisition unit 12C that acquires map information. For example, the evaluation unit 12 refers to the map information and estimates buildings, roads, scenery, and the like that may be included in an image captured at each imaging point P. In addition, the evaluation unit 12 recognizes road information such as the legal speed limit of the road at each imaging point P based on the map information.

[0033] Moreover, the evaluation unit 12 has a usefulness determination unit 12D that determines the usefulness of each of the images IM1, IM2, ... based on the analysis result of the analysis unit 12B. In this embodiment, the usefulness determination unit 12D determines the usefulness of each of the images IM1, IM2, ... based on the occupancy rate of obstacles in each of the images IM1, IM2, ... calculated by the calculation unit 12B2 of the analysis unit 12B.

[0034] Further, the usefulness determination unit 12D of the evaluation unit 12 determines the usefulness of each of the images IM1, IM2, ... based on the driving state information (movement state information) of the vehicle 20 or 30 in each of the images IM1, IM2, .... Specifically, in this embodiment, the usefulness determination unit 12D determines the usefulness of each of the images IM1, IM2, ... based on the analysis result (e.g., occupancy rate) of the analysis unit 12B and the speed and following distance of the vehicle 20 or 30 at the time of imaging.

[0035] Fig. 3A is a diagram showing an example of evaluation criteria of usefulness by the evaluation unit 12. Specifically, Fig. 3A is a diagram showing an example of evaluation criteria of usefulness by the usefulness determination unit 12D of the evaluation unit 12. As shown in Fig. 3A, for example, when no obstacle is detected by the analysis unit 12B, the usefulness determination unit 12D determines that the image has a usefulness A, which is the highest usefulness SC.

[0036] In this embodiment, when an obstacle is detected by the detection unit 12B1 of the analysis unit 12B and the calculation unit 12B2 calculates that the occupancy rate of the obstacle is equal to or less than a predetermined rate (for example, 5% or less), the usefulness determination unit 12D determines that the image has a usefulness of B. For example, even if an obstacle is detected, if the occupancy rate is 5% or less, the usefulness SC as a whole is often high.

[0037] Furthermore, when the vehicle speed V at the time of capturing an image is equal to or lower than a predetermined speed (for example, equal to or lower than 10 km / h), the usefulness determining unit 12D determines that the image has a usefulness of B. For example, an image captured when the vehicle speed V is equal to or lower than 10 km / h often has high image quality and a relatively high usefulness SC.

[0038] Furthermore, when the vehicle speed V at the time of capturing the image is within the legal speed range, the usefulness determining unit 12D determines that the image has a usefulness of B. For example, when the vehicle 20 or 30 is traveling at a speed close to the legal speed, the vehicle distance is often sufficiently secured and obstacles are rarely captured large. Therefore, an image captured by the vehicle 20 or 30 traveling at the legal speed often has a relatively high usefulness SC.

[0039] Furthermore, the usefulness determining unit 12D determines that an image with an obstacle occupancy rate of 6 to 50% is an image with a usefulness rate of C, and that an image with an obstacle occupancy rate of 51% or more is an image with a usefulness rate of D, which is the lowest usefulness rate. For example, when the obstacle occupancy rate is 50% or less, the image is useful depending on the purpose, but the usefulness rate SC is relatively low in many cases. Also, when the obstacle occupancy rate exceeds 51%, the image is often not useful.

[0040] Also, as shown in FIG. 3A, the usefulness determining unit 12D determines that the image has a usefulness of D when the inter-vehicle distance S from the vehicle ahead at the time of capturing the image is equal to or less than a predetermined distance (e.g., 5 m or less). This is to prevent erroneous detection of an obstacle when the inter-vehicle distance S is short. For example, consider a case where an obstacle is present very close to the imaging device, such as when the vehicle is stopped at an intersection with a vehicle ahead. An image captured in this situation is often mostly hidden by the obstacle, and has a low usefulness SC. However, there are cases where the shape of the obstacle cannot be recognized in the image. The usefulness determining unit 12D can appropriately determine the usefulness SC of an image captured in such a situation.

[0041] Thus, in this embodiment, the evaluation unit 12 evaluates each of the images IM1, IM2, ... based on driving state information of the vehicle 20 or 30 at the time when each of the images IM1, IM2, ... was captured (e.g., vehicle speed V and vehicle distance S), and also based on obstacles and their occupancy rate.

[0042] The evaluation unit 12 may use each of the image information and the driving condition information included in the image data DT as a parameter, and may score the usefulness level of each of the images IM1, IM2, ... for each parameter. The evaluation unit 12 may also evaluate the usefulness SC of each of the images IM1, IM2, ... based on the total score of the usefulness level for each of these parameters.

[0043] 3B is a diagram showing an example of evaluation of each of the images IM1, IM2, ... by the evaluation unit 12. As shown in FIG. 3B, in this embodiment, the analysis unit 12B of the evaluation unit 12 detects an obstacle OB for each of the images IM1, IM2, ... by the detection unit 12B1, and calculates the occupancy rate RT of the obstacle OB by the calculation unit 12B2. Then, the usefulness determination unit 12D determines the usefulness SC of each of the images IM1, IM2, ... based on the obstacle OB detected by the analysis unit 12B. In addition, the usefulness determination unit 12D determines the usefulness SC taking into account the traveling state of the vehicle.

[0044] For example, if a vehicle is detected as an obstacle OB in the area of ​​image IM1 and its occupancy rate RT is calculated to be 5%, the usefulness determining unit 12D determines that image IM1 is an image with usefulness B. Also, for example, if no obstacle OB is detected in the area of ​​image IM3, the usefulness determining unit 12D determines that image IM3 is an image with usefulness A. On the other hand, for image IM4, although no obstacle OB was detected, the vehicle distance S at the time of image capture was 2 m, so the image is determined to have usefulness D.

[0045] 4A is a block diagram showing a detailed configuration of the extraction unit 13. In this embodiment, the extraction unit 13 has an extraction condition acquisition unit 13A that acquires extraction conditions, which are conditions for acquiring conditions for extracting an image. The extraction condition acquisition unit 13A acquires, for example, an allowable range of usefulness SC when extracting an image.

[0046] The extraction condition acquisition unit 13A also acquires, as the extraction conditions, designation of image information (e.g., imaging point P) of each of the images IM1, IM2, .... The extraction condition acquisition unit 13A also acquires, as the extraction conditions, designation of locations on the map, such as intersection names, road names, and building names.

[0047] The extraction unit 13 has a map information acquisition unit 13B that acquires map information as reference information when extracting an image. For example, when the extraction condition acquisition unit 13A acquires a designation of an imaging point P or a designation of a specific place name on a road, the extraction condition acquisition unit 13A compares the designated imaging point P or place name with the map information and determines the acquired extraction condition (imaging point P and imaging direction D).

[0048] The extraction unit 13 has an image selection unit (hereinafter simply referred to as a selection unit) 13C that selects an image to be extracted from each of the images IM1, IM2, ... based on the extraction conditions acquired by the extraction condition acquisition unit 13A. The extraction unit 13 also has an extraction data generation unit 13D that generates image data including the image selected by the selection unit 13C as extraction data.

[0049] Fig. 4B is a diagram showing an example of image extraction by the extraction unit 13. Fig. 4B shows an example of image extraction conditions and an example of extracted images EIM, which are extracted images. For example, when the extraction conditions obtained are that the usefulness SC is usefulness A or B, the imaging point P is point P1, and the imaging direction D is direction D1, images IM1, IM3, and IM4 are extracted (selected) as extracted images EIM.

[0050] Similarly, as shown in FIG. 4B, the extraction unit 13 selects and extracts at least one image from each of the images IM1, IM2, ... based on image information of each of the images IM1, IM2, ..., such as the imaging height H, as an extracted image EIM.

[0051] The extraction unit 13 may adjust the conditions of the usefulness SC when extracting, based on the number of extracted images. For example, when there is only one extraction image EIM, the image selection unit 13C may expand the allowable range of the usefulness SC and select an image again. Furthermore, when the extraction condition acquisition unit 13A does not acquire the extraction conditions, the extraction unit 13 may extract only images with usefulness A as the extraction images EIM.

[0052] In this way, the extraction unit 13 extracts at least one image from each of the images IM1, IM2, . . . based on at least the evaluation result by the evaluation unit 12 of the usefulness of each of the images IM1, IM2, .

[0053] The album creation unit 14 creates an album in which the extracted images EIM extracted by the extraction unit 13 are rearranged in chronological order. For example, when the extraction unit 13 extracts images from a plurality of imaging points P or imaging directions D, the album creation unit 14 may rearrange the images for each imaging point P or each imaging direction D in chronological order.

[0054] 5 is a diagram showing an operation flow by the information processing device 10. First, in this embodiment, the image acquisition unit 11 of the information processing device 10 acquires image data DT (step S11). Next, the evaluation unit 12 evaluates the usefulness of images IM1, IM2, ... included in the image data DT (step S12).

[0055] Next, the extraction unit 13 acquires extraction conditions for images (step S13). Then, the extraction unit 13 extracts images based on the acquired extraction conditions (step S14). Next, the album creation unit 14 rearranges the extraction images EIM extracted by the extraction unit 13 (step S15). The album creation unit 14 also creates an album for displaying the rearranged extraction images EIM (step S16). In this manner, the information processing device 10 performs processing operations on each of the acquired images IM1, IM2, ...

[0056] The above-mentioned operation flow is merely an example. For example, after the image acquisition by the image acquisition unit 11 (step S11), the evaluation of the image by the evaluation unit 12 and the extraction of the image by the extraction unit 13 (steps S12 to S14) may be performed simultaneously. For example, the extraction unit 13 may extract an image after acquiring extraction conditions other than the usefulness SC, and may evaluate the extracted image. In this case, the extraction unit 13 may re-extract the image after evaluation.

[0057] Also, the rearrangement of the extracted images EIM by the album creating section 14 (step S15) may be performed simultaneously with the extraction by the extracting section 13 when the extracting section 13 extracts the images (step S14). That is, the rearrangement may be performed in chronological order when the extracting section 13 extracts the images.

[0058] 3A, the evaluation unit 12 performs image analysis on each of the images IM1, IM2, ..., and evaluates the usefulness of each of the images IM1, IM2, ..., taking into consideration the traveling state of the vehicle 20 or 30 at the time of capturing the images. However, the evaluation content and evaluation criteria of the evaluation unit 12 are not limited to this.

[0059] The evaluation unit 12 may evaluate the usefulness of each of the images IM1, IM2, ... based only on the traveling state of the vehicle 20 or 30 at the time when each of the images IM1, IM2, ... was captured. For example, the evaluation unit 12 may determine a criterion for the usefulness SC based only on the speed V or the inter-vehicle distance S of the vehicle 20 or 30 at the time when the image was captured.

[0060] For example, when the speed V is equal to or greater than a predetermined value, the possibility of including an obstacle OB is low, so this may be used as a criterion to determine a high usefulness SC, that is, a usefulness level A. Also, when the speed V increases by a predetermined value or more from a stopped state, it is expected that the vehicle is starting to move and the distance to the vehicle ahead will be large, so this may be used as a criterion to determine a relatively high usefulness B.

[0061] Also, in this embodiment, the evaluation unit 12 has been described as determining (rating) the usefulness SC of the images IM1, IM2, ... in four stages by the usefulness determination unit 12D. However, the evaluation content of each of the images IM1, IM2, ... by the evaluation unit 12 is not limited to this. The evaluation unit 12 only needs to evaluate at least the usefulness SC of each of the images IM1, IM2, .... For example, the evaluation unit 12 may only determine whether each of the images IM1, IM2, ... has usefulness (whether the usefulness SC is high or low) based on the evaluation criteria as described above.

[0062] In the present embodiment, the evaluation unit 12 includes a calculation unit 12B2 that calculates the occupancy rate RT of the obstacle OB by image analysis, and a usefulness determination unit 12D that determines the usefulness SC based on the calculation unit 12B2. However, the evaluation unit 12 may detect the obstacle OB included in each of the multiple images IM1, IM2, ..., and evaluate the usefulness SC based on the detected obstacle OB. For example, the evaluation unit 12 can detect the obstacle OB and determine the usefulness SC by acquiring the vehicle distance S, without performing image analysis or calculating the occupancy rate RT.

[0063] 1B is merely an example. The image data DT only needs to include a plurality of images IM1, IM2, ... and driving state information indicating the driving state of the vehicle 20 or 30 at the time of capturing the images. For example, the image data DT does not need to include the capturing height H and the vehicle speed V.

[0064] As described above, the information processing device 10 has an image acquisition unit 11 that acquires image data DT including multiple images IM1, IM2, ... captured by an imaging device possessed by the moving body 20 or 30 and movement state information indicating the movement state of the moving body 20 or 30 at the time of capturing each of the multiple images IM1, IM2, ..., an image evaluation unit 12 that evaluates the usefulness SC of each of the multiple images IM1, IM2, ... based on the movement state information for each of the multiple images IM1, IM2, ..., and an image extraction unit 13 that extracts at least one image EIM from the multiple images IM1, IM2, ... based on the evaluation of the usefulness SC of each of the multiple images IM1, IM2, ... by the image evaluation unit 12.

[0065] Therefore, the information processing device 10 performs an appropriate evaluation on each of the images captured by the moving body 20 or 30, and extracts a desired useful image based on the evaluation. Therefore, it is possible to provide an information processing device 10 that performs processing so that a plurality of images captured by the moving body 20 or 30 can be used for various purposes.

[0066] Moreover, the image evaluation unit 12 detects an obstacle OB included in each of the multiple images IM1, IM2, ..., and evaluates the usefulness SC based on the detected obstacle OB. Therefore, the usefulness SC of the image can be appropriately evaluated based on the detection of the obstacle OB in addition to the moving state information.

[0067] The image evaluation unit 12 also includes an occupancy calculation unit 12B2 that calculates an occupancy RT, which is the ratio of the obstacle OB in each of the multiple images IM1, IM2, ..., and a usefulness determination unit 12D that determines a usefulness SC of each of the multiple images IM1, IM2, ..., based on the occupancy RT of the obstacle OB. Therefore, the usefulness of each of the multiple images IM1, IM2, ..., that is, the useful area that does not include the obstacle OB, becomes clear. Also, only useful images can be accurately extracted.

[0068] The image data DT also includes, as movement state information of the moving object 20 or 30, a moving speed V of the moving object 20 or 30 at the time of capturing each of the multiple images IM1, IM2, .... The image evaluation unit 12 then evaluates the usefulness SC of each of the multiple images IM1, IM2, .... based on the moving speed V. By taking into account the moving speed V of the moving object 20 or 30 at the time of capturing the image when evaluating the usefulness SC of the image, it is possible to appropriately evaluate the usefulness SC of the image.

[0069] The image data DT also includes, as movement state information of the moving body 20 or 30, an inter-moving body distance S between the moving body 20 or 30 and another moving body present in the imaging direction D of each of the multiple images IM1, IM2, ... at the time of imaging each of the multiple images IM1, IM2, .... The image evaluation unit 12 also evaluates the usefulness SC of each of the multiple images IM1, IM2, ... based on the inter-moving body distance S. By considering the inter-moving body distance S of the moving body 20 or 30 at the time of imaging when evaluating the usefulness SC of the image, it is possible to appropriately evaluate the usefulness SC of the image.

[0070] The image evaluation unit 12 also has a map information acquisition unit 12C that acquires map information, and a usefulness determination unit 12D determines the usefulness SC of each of the multiple images IM1, IM2, ... based on the map information. Therefore, it is possible to appropriately evaluate images that may have a high usefulness SC, such as an intersection.

[0071] The image data DT also includes the imaging point P and imaging direction D of each of the multiple images IM1, IM2, ..., and the image extraction unit 13 extracts at least one image EIM from the multiple images IM1, IM2, ... based on the imaging point P and the imaging direction D. Therefore, it is possible to extract only images taken at the same point and in the same direction that can be used for the same purpose, for example.

[0072] The image data DT also includes the imaging height H of each of the multiple images IM1, IM2, ..., and the image extraction unit 13 extracts at least one image EIM from the multiple images IM1, IM2, ... based on the imaging height H. Therefore, for example, it is possible to extract only images captured from the same height. Therefore, it is possible to use the image data DT to observe the time series change of the subject from a fixed point, and for example, it is possible to clearly visualize the time series change of the subject.

[0073] The image data DT also includes the imaging date and time T of each of the multiple images IM1, IM2, ..., and the image extraction unit 13 extracts at least one image EIM from the multiple images IM1, IM2, ... based on the imaging date and time T. Therefore, it is possible to extract only images within a predetermined time range, for example.

[0074] The information processing device 10 also includes an album creating unit 14 that creates an album in which at least one image EIM extracted by the image extracting unit 13 is rearranged in chronological order. Therefore, images useful for a desired purpose can be automatically rearranged in chronological order.

[0075] In this embodiment, the information processing device 10 functions as a server that acquires and processes images from a plurality of moving objects 20 and 30, but the information processing device 10 is not limited to being a server. The information processing device 10 may be provided as a terminal device that acquires and processes images from only one moving object, for example. EXAMPLES

[0076] 6 is a block diagram of an information processing device 10A according to Example 2. In this example, the information processing device 10A is mounted on a moving body (vehicle) 20 and functions as an information processing terminal that performs various processes on images captured by the moving body 20.

[0077] The information processing device 10A acquires image data DT (for example, see FIG. 1B) including images IM1, IM2, ... from an imaging device 21 mounted on a vehicle serving as a moving body 20. In this embodiment, the moving body 20 has a positioning device 22 and a detection device 23. For example, the information processing device 10A may acquire image information (for example, an imaging point P and an imaging direction D, etc.) from the positioning device 22, and acquire movement state information (for example, a movement speed V and a distance S between moving bodies, etc.) from the detection device 23.

[0078] Moreover, the information processing device 10A has the same configuration as the information processing device 10, except that it has a transmission unit 17. The information processing device 10A has the transmission unit 17 that transmits at least one image EIM extracted by the image extraction unit 13 or an album created by the album creation unit 14 to the outside.

[0079] Moreover, the information processing device 10A performs the same processes as the information processing device 10 (for example, execution of the flow shown in FIG. 5) except for the process of the transmission unit 17. For example, after step S16 shown in FIG. 5, the information processing device 10A transmits the album (images for the album) to an external server by the transmission unit 17.

[0080] By including the transmission unit 17, when the information processing device 10A is mounted on a moving body 20, for example, it is possible to distribute only useful images out of images captured by the moving body 20 to the outside. For example, the information processing device 10A can provide useful images held by the moving body 20 to other moving bodies or an external server. Therefore, it is possible to provide an information processing device 10A that performs processing so that a plurality of images captured by the moving body 20 can be used for various purposes. [Explanation of symbols]

[0081] 10, 10A Information processing device 11 Image acquisition section 12 Image evaluation section 13 Image Extraction Section

Claims

[Claim 1] an image acquisition unit that acquires image data including a plurality of images captured by an imaging device of a moving object and movement state information indicating a movement state of the moving object at the time of capturing each of the plurality of images; an image evaluation unit that evaluates a usefulness of each of the plurality of images based on the movement state information for each of the plurality of images; an image extraction unit that extracts at least one image from among the plurality of images based on the evaluation of the usefulness of each of the plurality of images by the image evaluation unit.

Images