Imaging device, imaging method, imaging program, and recording medium

Abstract

To provide an imaging device, an imaging method, an imaging program, and a recording medium that enable the imaging device that captures an image according to an imaging instruction from a user, can acquire with a high probability an image of a scene that a user desires to capture.SOLUTION: An imaging device includes a storage unit that stores successive images recently captured by an imaging unit that moves with a moving object, a reception unit that receives an imaging instruction by a user operation, and an extraction unit that extracts a predetermined number of images captured by the imaging unit at each of one or more timings based on a time point when the moving object has been located at the imaging point when the moving object is positioned at a preset imaging point within a period from a time point when the reception unit receives the imaging instruction to a time point before the first time.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an imaging apparatus, an imaging method, an imaging program, and a recording medium. [Background technology]

[0002] 2. Description of the Related Art There is known a technique for extracting a desired image, such as a best still image, from a plurality of images captured in advance.

[0003] For example, Patent Document 1 discloses an imaging device that, when converting video data into a still image, creates a still image by arranging the image obtained at the shooting timing (shutter operation timing) as the main frame, and arranging a predetermined number of frame images obtained before the shooting timing as other frame images. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2007-318773 Summary of the Invention [Problem to be solved by the invention]

[0005] In imaging devices such as those described above, the timing of instructions from the user often does not match the timing at which the desired image is actually captured, and one of the issues is that even if an image obtained at the time of the user's instructions is converted into a still image, the image that the user wants to capture may not be extracted.

[0006] The present invention has been made in consideration of the above-mentioned points, and one of its objects is to provide an imaging device, an imaging method, an imaging program, and a recording medium that enable an image capturing the scenery that the user wishes to capture to be obtained with a high probability in an imaging device that captures images in accordance with imaging instructions from a user. [Means for solving the problem]

[0007] The invention described in claim 1 comprises a memory unit that stores consecutive images recently captured by an imaging unit moving together with a moving object, a reception unit that receives an imaging instruction by user operation, and an extraction unit that extracts from the memory unit as captured images a predetermined number of images captured by the imaging unit at each of one or more timings based on the time when the moving object was located at a predetermined imaging point within a period from the time when the reception unit received the imaging instruction to a time going back a first hour.

[0008] The invention described in claim 10 is an imaging method executed by an imaging device, comprising: a storage step of storing in a memory unit consecutive images recently captured by an imaging unit moving together with a moving object; a reception step of receiving an imaging instruction by a user operation; and an extraction step of extracting from the memory unit as captured images a predetermined number of images captured by the imaging unit at each of one or more timings based on the time when the moving object was located at the imaging point, if the moving object is located at a predetermined imaging point within a period from the time when the imaging instruction was received in the reception step to a time going back one hour.

[0009] The invention described in claim 11 is an imaging program executed by an imaging device equipped with a computer, which causes the computer to execute a storage step of storing in a memory unit consecutive images recently captured by an imaging unit moving together with a moving object, a reception step of receiving an imaging instruction by user operation, and an extraction step of extracting from the memory unit as captured images a predetermined number of images captured by the imaging unit at each of one or more timings based on the time when the moving object was located at the imaging point, if the moving object is located at a predetermined imaging point within a period from the time when the imaging instruction was received in the reception step to a time going back one hour. [Brief description of the drawings]

[0010] [Figure 1] 1 is a diagram showing a front seat portion of a vehicle in which an imaging device according to an embodiment is mounted; [Diagram 2] 1 is a block diagram showing a configuration of an imaging device according to an embodiment. [Diagram 3] FIG. 4 is a diagram showing an example of information included in an electronic map according to an embodiment; [Figure 4] 5A and 5B are diagrams illustrating an example of imaging timing of the imaging device according to the embodiment. [Diagram 5] 5 is a flowchart showing an example of a routine executed by the imaging device according to the embodiment. [Figure 6] 10 is a flowchart showing an example of a subroutine executed by the imaging device according to the embodiment. [Figure 7] 10 is a flowchart showing another example of a subroutine executed by the imaging device according to the embodiment. [Figure 8] 10 is a flowchart showing another example of the routine executed by the imaging device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In the following description and accompanying drawings, substantially the same or equivalent parts are designated by the same reference numerals. EXAMPLES

[0012] A configuration of an imaging device 10 according to an embodiment will be described with reference to Fig. 1 to Fig. 3. Fig. 1 is a diagram showing a front seat portion of a vehicle M as an example of a moving body in which an imaging device 10 according to an embodiment is mounted.

[0013] 1, the imaging device 10 is disposed in the center console of a vehicle M. The imaging device 10 is a device that acquires an image of an external landscape from the vehicle M, and extracts an image as a captured image from the acquired image in response to an imaging instruction from a user.

[0014] The front camera 11 as an imaging unit is provided on the back of the rearview mirror RM, that is, on the surface facing the windshield FG. The imaging direction of the front camera 11 is the traveling direction of the vehicle M. When the traveling direction of the vehicle M is the forward direction, the front camera 11 is disposed so as to be able to capture images of a range including the front, right front, and left front of the vehicle M. Therefore, the front camera 11 can capture images within the range of the field of view of a user sitting in the front seat of the vehicle M when looking forward. The front camera 11 is connected to the imaging device 10 so as to be able to communicate with the imaging device 10, and can transmit a signal of the captured image to the imaging device 10.

[0015] The front camera 11 may be provided anywhere on the vehicle M as long as it can capture an image in front of the vehicle M. For example, the front camera 11 may be disposed on the dashboard DB, or may be provided on the outer surface (exterior surface) of the vehicle M, such as on the hood or the front bumper.

[0016] The side camera 13 as an imaging unit is attached above the left front door. The side camera 13 captures images to the side of the vehicle M, i.e., in the horizontal direction perpendicular to the traveling direction of the vehicle M. When the traveling direction of the vehicle M is the forward direction, the side camera 13 is disposed so as to be able to capture images in a range including the left side, the left front, and the left rear. Therefore, the side camera 13 can capture images within the range of the field of view when a user riding on the left side of the front seat or the left side of the rear seat of the vehicle M is looking to the left. The side camera 13 can transmit a signal of the captured image to the imaging device 10.

[0017] The side camera 13 may be provided at any location of the vehicle M as long as it is capable of capturing an image of the side of the vehicle M. For example, the side camera 13 may be provided on an exterior surface of the vehicle M, for example, above a door.

[0018] Note that cameras (not shown) similar to the above-described front camera 11 and side camera 13 may be attached to the vehicle M so as to be capable of capturing images of a range including the right side of the vehicle M. In addition, cameras similar to the above-described front camera 11 and side camera 13 may be attached to the vehicle M as rear cameras capable of capturing images of the rear of the vehicle M.

[0019] The microphone 15 is a voice input device arranged to acquire voice within the vehicle M. The microphone 15 is communicably connected to the imaging device 10 and is capable of converting the acquired voice into an electrical signal and transmitting it to the imaging device 10. The imaging device 10 receives a signal from the microphone 15 and performs voice recognition on the voice based on the user's speech, and when it recognizes a specific word, it recognizes the voice as an imaging instruction from the user.

[0020] The touch panel display 17 includes a touch panel and a display. The touch panel display 17 is communicably connected to the imaging device 10. The display of the touch panel display 17 displays an image supplied from the imaging device 10. The touch panel of the touch panel display 17 transmits a signal indicating an input operation by touching the touch panel to the imaging device 10.

[0021] For example, touch panel display 17 displays on the display a button indicating an image capturing instruction to imaging device 10. For example, when the button indicating the image capturing instruction is pressed, imaging device 10 accepts the image capturing instruction from the user via the touch panel of touch panel display 17.

[0022] The GNSS receiver 19 is a device that receives signals (GNSS signals) from GNSS (Global Navigation Satellite System) satellites. The GNSS receiver 19 is arranged, for example, on a dashboard DB. The GNSS receiver 19 may be arranged anywhere as long as it can receive GNSS signals.

[0023] The GNSS receiver 19 is communicatively connected to the imaging device 10 and is capable of transmitting the received GNSS signal to the imaging device 10. The imaging device 10 acquires and stores current position information of the vehicle M using the GNSS signal.

[0024] The imaging device 10 acquires and stores moving images captured by the front camera 11 and the side camera 13. In other words, in this embodiment, the imaging device 10 has a function similar to that of a so-called drive recorder.

[0025] When the imaging device 10 receives an input operation indicating an imaging instruction from a user by voice operation or touch operation on the touch panel via the microphone 15 or the touch panel display 17, the imaging device 10 captures a still image as a captured image using the stored moving image. Specifically, the imaging device 10 extracts a predetermined number of frame images from a large number of frame images constituting the stored moving image to generate a still image. The imaging device 10 transmits the extracted still image to a server (not shown), i.e., uploads it.

[0026] 2 is a block diagram showing the configuration of the imaging device 10. As shown in FIG. 2, the imaging device 10 is configured by connecting each unit via a system bus 21.

[0027] The input unit 23 is an interface that communicatively connects the imaging device 10 with devices provided in the vehicle M1.

[0028] The input unit 23 communicably connects the imaging device 10 to the front camera 11 and the side camera 13. The input unit 23 also communicably connects the imaging device 10 to the microphone 15.

[0029] The input unit 23 connects the imaging device 10 and the touch panel of the touch panel display 17 so as to be able to communicate with each other.

[0030] The input unit 23 is an interface that communicatively connects the imaging device 10 to the GNSS receiver 19 and a sensor 24 provided in the vehicle M. The sensor 24 is a sensor group including an acceleration sensor and a gyro sensor. For example, the acceleration sensor may be built into the imaging device 10.

[0031] The imaging device 10 creates a driving history of the vehicle M by acquiring and storing information regarding the driving of the vehicle M, including the current position of the vehicle M, from the GNSS receiver 19 and the sensor signals of the sensor 24.

[0032] The large-capacity storage device 25 is a storage device configured, for example, by a hard disk device, an SSD (solid state drive), a flash memory, etc. The large-capacity storage device 25 stores various programs executed by the imaging device 10. Note that the various programs may be acquired, for example, from another server device or the like via a network, or may be recorded on a recording medium and read via various drive devices. For example, the large-capacity storage device 25 stores an imaging program executed by the imaging device 10 when extracting an image from a moving image to create a still image.

[0033] The large-capacity storage device 25 stores information necessary for executing various programs. For example, the large-capacity storage device 25 stores information related to the driving history of the vehicle M. Specifically, the driving history includes, for example, information in which the position of the vehicle M at each time at a predetermined interval is associated with the time.

[0034] The mass storage device 25 stores a database in which data necessary for executing various programs is stored.

[0035] The large-capacity storage device 25 includes a map information database (hereinafter referred to as map information DB) 25A. Map information (hereinafter also referred to as electronic map) of an area in which the vehicle M travels is stored in the map information DB 25A.

[0036] 3 is a diagram showing an example of information stored together with an electronic map in the map information DB 25A. The map information DB 25A stores set points, which are points on an electronic map for which imaging is recommended when a moving object is located, or points that are set in advance as points where a user is expected to wish to image, in association with the electronic map. In the following description, the set points are also referred to as "imaging points."

[0037] As shown in Fig. 3, each piece of location information indicating a set point is associated with a set point ID for identifying the set point. Note that the location information indicating a set point may be information indicating a road link or a lane link, or may be information indicating an area.

[0038] As described above, the set point is a point where imaging is recommended when the moving body is located, or a point where the user is expected to want to image. The set point is, for example, a scenic point. For example, the set point is a point where a spectacular feature such as a mountain, the sea, or a building can be seen from the moving body when the moving body is located at the set point. The imaging device 10 identifies the image to be extracted based on the timing when the vehicle M was located at the set point, thereby making it possible to increase the probability of acquiring an image showing the scenery that the user wants to image.

[0039] 2 again, the large-capacity storage device 25 has a moving image storage unit 25B. The moving image storage unit 25B stores moving images captured by the front camera 11 and the side camera 13. The moving image storage unit 25B holds, for example, a certain period of moving images. Also, the moving image storage unit 25B stores, for example, a predetermined amount of moving images.

[0040] Furthermore, the large-capacity storage device 25 has an image temporary storage section 25C. In the image temporary storage section 25C, for example, some images included in the moving image stored in the moving image storage section 25B are extracted and temporarily stored. For example, the image temporary storage section 25C is a memory section in which new images are stored from time to time, like a ring buffer. For example, a plurality of frame images from a moving image of several tens of seconds can be stored in the image temporary storage section 25C. For example, the image temporary storage section 25C stores frame images extracted from the moving image at predetermined time intervals. In the image temporary storage section 25C, old frame images are rewritten with new frame images, so that images are updated from time to time.

[0041] For example, when the moving images stored in the moving image storage unit 25B are captured at a frame rate of about 15 to 30 fps (for example, 27.5 fps), the temporary image storage unit 25C may store, for example, 10 frame images, one per second, for about 10 seconds. For example, about six images per second may be stored at equal time intervals for several tens of seconds.

[0042] The video storage section 25B and the temporary image storage section 25C of the large-capacity storage device 25 store consecutive images captured recently by an imaging section moving together with the moving object. Note that consecutive images refer to images arranged in chronological order.

[0043] The control unit 27 is configured with a CPU (Central Processing Unit) 27A, a ROM (Read Only Memory) 27B, a RAM (Random Access Memory) 27C, etc., and functions as a computer. The CPU 27A reads out and executes various programs stored in the ROM 27B and the large-capacity storage device 25 to realize various functions.

[0044] The control unit 27 reads out and executes the imaging program stored in the mass storage device 25, thereby extracting a captured image in response to an imaging instruction from the user.

[0045] Control unit 27 stores the moving images captured by front camera 11 and side camera 13 in moving image storage unit 25B of mass storage device 25 via input unit 23. Control unit 27 also extracts some images included in the moving images stored in moving image storage unit 25B and temporarily stores them in temporary image storage unit 25C.

[0046] In other words, the control unit 27 executes a storage step of storing consecutive images recently captured by an imaging unit moving together with the moving object in the moving image storage unit 25B or the temporary image storage unit 25C as a storage unit.

[0047] The control unit 27 acquires voice inside the vehicle M from the microphone 15 via the input unit 23 and performs voice recognition. When the control unit 27 recognizes a specific voice based on the speech of a user inside the vehicle M, it accepts an image capture instruction from the user. For example, when the control unit 27 recognizes a predetermined wake word and voice command, it accepts an image capture instruction from the user. In other words, the control unit 27 accepts an image capture instruction by a voice operation of the user.

[0048] Control unit 27 recognizes an input operation using touch panel display 17 via input unit 23. When control unit 27 recognizes a specific contact operation performed on the touch panel of touch panel display 17, it accepts an image capture instruction from the user. For example, when control unit 27 recognizes that an image capture instruction button displayed on the display of touch panel display 17 has been pressed, it accepts an image capture instruction from the user.

[0049] In this way, the control unit 27 accepts an image capturing instruction from a user by recognizing a user operation such as a voice operation by the user's speech or a touch operation on a touch panel by the user. The control unit 27 functions as a reception unit that executes a reception step of accepting an image capturing instruction by a user operation.

[0050] The control unit 27 acquires the current position of the vehicle M based on the data acquired from the GNSS receiver 19 and the sensor 24 via the input unit 23. The control unit 27 also refers to the electronic map stored in the map information DB 25A to identify a position on the electronic map that corresponds to the current position of the vehicle M, and functions as a position identification unit.

[0051] In addition, the control unit 27 generates a driving history including information associating the current position of the vehicle M with the current time, and stores the history in the large-capacity storage device 25.

[0052] When the control unit 27 receives an image capture instruction from the user, it extracts a predetermined number of images from among the images stored in the large-capacity storage device 25 as captured images.

[0053] More specifically, when vehicle M is located at a preset imaging point within a period going back a predetermined length of time (first time) from the time when the imaging instruction is received, control unit 27 functions as an extraction unit that executes an extraction step of extracting as captured images from temporary image storage unit 25C as a memory unit a predetermined number of images captured by front camera 11 or side camera 13 as an imaging unit at each of one or more timings based on the time when vehicle M was located at the imaging point.

[0054] The control unit 27 extracts a predetermined number of images from the images stored in the image temporary storage unit 25C. The image temporary storage unit 25C stores images obtained by thinning out a large number of frame images constituting a moving image, so that the extraction of captured images is efficiently executed. The control unit 27 may extract a predetermined number of images from the moving image stored in the moving image storage unit 25B.

[0055] The output unit 29 is connected to the display of the touch panel display 17 provided in the vehicle M. The output unit 29 is an interface for supplying various information to be displayed on the display in accordance with commands from the control unit 27.

[0056] The output unit 29 outputs an image to be displayed on the display of the touch panel display 17 in response to an instruction from the control unit 27. For example, the output unit 29 supplies, to the display, an image for presenting to the user an image capture instruction button for receiving an image capture instruction.

[0057] In addition, the output unit 29 may be communicatively connected to a speaker (not shown) provided in the vehicle M, and may notify the user by voice, for example, that an imaging instruction has been accepted or that imaging has been completed.

[0058] The transmitting / receiving unit 31 is a network adapter such as a network interface card (NIC) connected to a wireless device (not shown). The transmitting / receiving unit 31 transmits and receives data between the imaging device 10 and the outside in accordance with an instruction from the control unit 27.

[0059] For example, the transmitting / receiving unit 31 functions as a transmitting unit that transmits to a server an image extracted as a captured image by the imaging device 10. Also, for example, the transmitting / receiving unit 31 may receive map information for updating the map information DB.

[0060] The extraction of the captured image by the control unit 27 will be described with reference to Fig. 4. Fig. 4 is a diagram showing a schematic view of a vehicle M traveling along a road L.

[0061] However, from when the user issues an image capture instruction operation until the control unit 27 of the imaging device 10 accepts the image capture instruction operated by the user, a processing time is required for the control unit 27 to recognize the image capture instruction. Furthermore, an arbitrary time is also required from the time when the user wants to capture an image to when the image capture instruction is actually issued. In particular, when the vehicle M is traveling at high speed or traveling around a curve, due to the delay of these times, the scenery captured by the front camera 11 or the side camera 13 at the time when the control unit 27 of the imaging device 10 recognizes the image capture instruction may be significantly different from the scenery that the user wanted to capture.

[0062] In Fig. 4, vehicle M at time (t0) when control unit 27 of imaging device 10 receives an imaging instruction from a user is indicated as M(t0), and the point where vehicle M was located at the time when the imaging instruction was received is indicated as P1. Also in Fig. 4, vehicle M at time (t1) a predetermined length of time T1 (first time) prior to the time when the imaging instruction was received is indicated as M(t1), and the point where vehicle M was located at the time prior to the time T1 is indicated as P2. Also in Fig. 4, a set point S1 on road L is indicated. For example, set point S1 is a point where imaging is recommended because feature A is visible from a moving body located at set point S1.

[0063] When the control unit 27 receives an image capturing instruction from the user, it refers to the travel history and map information of the vehicle M from a time point (t1) that is a time T1 (first time) prior to the time point when the image capturing instruction was received to the time point (t0) when the image capturing instruction was received. As described above, in the travel history, the points where the vehicle M was located are associated with the times. Also, as described above, the map information includes position information of set points that are points that are set in advance (see FIG. 3).

[0064] The control unit 27 determines whether or not the vehicle M was located at a preset point in the travel history. The control unit 27 determines whether or not the set point is included in a route (journey) traveled by the vehicle M during a period from time (t1) to time (t0).

[0065] When the control unit 27 determines that the vehicle M was located at the set location, it extracts a predetermined number of images from the temporary image storage unit 25C as captured images at one or more times based on the time when the vehicle M was located at the set location.

[0066] For example, in the example of Fig. 4, the route (journey) traveled by the vehicle M from time point (t1) to time point (t0) includes the set point S1. In Fig. 4, the vehicle M at the time point when it was located at the set point S1 is shown as M(t3). In this case, for example, the control unit 27 extracts an image captured by the front camera 11 or the side camera 13 at the time point (t3) when the vehicle M was located at the set point S1.

[0067] For example, a set point may be set for each camera in the map information. For example, there may be a set point for the front camera 11 and a set point for the side camera 13. For example, a point in front of the vehicle M where a feature that would be desired to be photographed may be a set point for the front camera 11. For example, a point to the side of the vehicle M where a feature that would be desired to be photographed may be a set point for the side camera 13. For example, in the example of FIG. 4, the set point S1 may be set as a set point for the front camera 11, and an image captured by the front camera 11 may be preferentially extracted.

[0068] For example, a set point may be associated with information indicating the position of a feature visible from the set point. For example, the imaging device 10 may preferentially extract an image captured by either the front camera 11 or the side camera 13 depending on the positional relationship between the set point and the feature. For example, in the example of Fig. 4, the relative position of feature A with respect to the set point S1 is associated with the set point S1, and the image captured by the front camera 11 may be preferentially extracted based on the relative position.

[0069] As shown in the example of Figure 4, if the vehicle M was located at the set point S1 before the point (t0) when the control unit 27 received an instruction from the user, it is highly likely that the user wanted to capture an image including feature A, which is a feature visible from the set point S1.

[0070] For example, the time when the imaging instruction is received (t0) may include a delay from the time when the vehicle M was located at the set point S1 (t3) to the time it takes for the vehicle M to recognize the imaging instruction, as well as the time from when the user decides that he or she wants to image feature A to when the user actually issues the imaging instruction.

[0071] In this embodiment, the control unit 27 extracts images at one or more timings based on the time (t3) when the vehicle M was located at the set point, going back a predetermined time T1 from the time (t0) when the image capturing instruction was received. This makes it possible to more reliably extract the image desired by the user, for example, even when the vehicle M is traveling at high speed or traveling around a curve.

[0072] The control unit 27 extracts, for example, one image captured at the time (t3) when the vehicle M was located at the set point. The control unit 27 may extract, for example, multiple images captured at multiple timings centered around the one image captured at the time (t3) when the vehicle M was located at the set point. For example, the multiple timings may be equal time intervals.

[0073] The control unit 27 may extract, for example, a predetermined number of images captured by the imaging unit within a predetermined time period based on the point in time when the position on the electronic map identified as the position of the vehicle M was located at an imaging point previously set on the electronic map, as captured images from the temporary image storage unit 25C.

[0074] For example, in the example shown in Figure 4, the control unit 27 may extract as captured images from the temporary image storage unit 25C a predetermined number of images captured by the front camera 11 or the side camera 13 within a predetermined period including the time (t3) when the vehicle M was located at the set point S1.

[0075] For example, when a position on an electronic map identified as the position of vehicle M is located within a predetermined distance based on an imaging point previously set on the electronic map, the control unit 27 may extract a predetermined number of images captured by the imaging unit as captured images from the temporary image storage unit 25C.

[0076] The predetermined distance based on the imaging point may not be a straight-line distance, but may be a predetermined section of the route traveled by the vehicle M, that is, a predetermined distance on the route.

[0077] For example, in the example shown in Figure 4, the control unit 27 may extract as captured images from the temporary image storage unit 25C a predetermined number of images captured by the front camera 11 or the side camera 13 when the vehicle M was located within a predetermined distance based on the set point S1.

[0078] In the above description, an example has been described in which the control unit 27 determines whether or not the set point is included in the route (journey) traveled by the vehicle M during the period from the time point (t1) going back one hour from the time point when the image capture instruction was received to the time point (t0) when the image capture instruction was received, but this is not limiting. For example, it may be determined whether or not the set point is included within a predetermined period based on the time point (t1) going back one hour. For example, it may be determined whether or not the set point is included within a predetermined period before the time point (t1) going back one hour, or within a predetermined period before and after the time point (t1).

[0079] With reference to FIGS. 5 to 8, information processing executed by the control unit 27 when imaging is performed by the imaging device 10 in this embodiment will be described.

[0080] 5 is a flowchart showing an imaging routine RT1 which is an example of a routine executed by the control unit 27. The control unit 27 starts the imaging routine RT1, for example, when the imaging device 10 is powered on. In addition, the control unit 27 starts storing moving images acquired by the front camera 11 and the side camera 13 in the moving image storage unit 25B, for example, when the imaging device 10 is powered on. In addition, the control unit 27 starts storing moving images in the moving image storage unit 25B, for example, and starts temporarily storing images included in the moving images stored in the moving image storage unit 25B in the image temporary storage unit 25C.

[0081] When the control unit 27 starts the imaging routine RT1, it determines whether or not an imaging instruction has been received from the user (step S101). In step S101, for example, the control unit 27 performs voice recognition on the voice input from the microphone 15, and when it recognizes, for example, a wake word followed by a predetermined voice command indicating an imaging instruction, it determines that an imaging instruction from the user has been received.

[0082] In step S101, for example, control unit 27 may determine that an image capture instruction has been accepted when it recognizes that an image capture instruction button displayed on the display of touch panel display 17 has been pressed by a touch operation on the touch panel.

[0083] In step S101, the control unit 27 functions as a reception unit that receives an image capture instruction by a user operation.

[0084] When the control unit 27 determines in step S101 that an image capture instruction has not been accepted (step S101: NO), the control unit 27 repeats step S101 and determines again whether or not an image capture instruction has been accepted.

[0085] When the control unit 27 determines in step S101 that an image capturing instruction has been received (step S101: YES), the control unit 27 refers to the driving history going back a predetermined length of time (first time) from the time when the image capturing instruction was received (step S102). In step S102, for example, the control unit 27 refers to the driving history within the period from the time when the image capturing instruction was received to the time going back the first time.

[0086] Also, for example, in step S102, a travel history for a predetermined time width before or around the point going back the first time may be referenced. Also, for example, in step S102, a travel history within a predetermined distance of a point where the vehicle M was located at the point going back the first time may be referenced.

[0087] After executing step S102, the control unit 27 executes an image extraction subroutine and extracts a captured image from the images stored in the image temporary storage unit 25C (step S103). In step S103, for example, when the vehicle M is located at a preset image capture point (set point) in the travel history referred to in step S102, the control unit 27 extracts, as a captured image, an image captured at a timing based on the time when the vehicle M was located at the set point. In step S103, for example, an image captured at the time when the vehicle M was located at the set point is extracted.

[0088] In step S103, for example, a plurality of images captured before and after the image captured at the time when the vehicle M was located at the set point may be extracted, centering on the image captured at the time when the vehicle M was located at the set point. Also, in step S103, a plurality of images including images captured when the vehicle M was located at points before and after the set point may be extracted, centering on the image captured at the time when the vehicle M was located at the set point.

[0089] After executing step S103, the control unit 27 transmits the image extracted in step S103 to the outside (step S104). In step S104, for example, the control unit 27 transmits the extracted image to a preset server. In step S104, for example, the control unit 27 may transmit the extracted image to a terminal device owned by the user.

[0090] After executing step S104, the control unit 27 ends the imaging routine RT1 and starts a new imaging routine RT1.

[0091] FIG. 6 is a flowchart showing an image extraction subroutine RT2, which is an example of an image extraction subroutine executed by the control unit 27 as an extraction unit in step S103 of the imaging routine RT1.

[0092] When starting the image extraction subroutine RT2, the control unit 27 determines whether or not the vehicle M was located at a preset imaging point (set point) in the travel history referred to in step S102 of the imaging routine RT1 (step S201).

[0093] For example, in step S102 of the imaging routine RT1, when the driving history for the period from the time when the imaging instruction was received to the time going back one hour (hereinafter also referred to as the first period) is referenced, the control unit 27 determines in step S201 that the vehicle M was located at the set point if the set point is included in the route traveled by the vehicle M during the first period.

[0094] When the control unit 27 determines in step S201 that the vehicle M was located at the set point (step S201: YES), it identifies an image captured at the time when the vehicle M was located at the set point (step S202). In step S202, for example, the control unit 27 refers to the image temporary storage unit 25C of the large-capacity storage device 25, and identifies an image captured at the time closest to the time when the vehicle M was located at the set point (hereinafter also referred to as a specific time).

[0095] After executing step S202, the control unit 27 extracts a predetermined number of images captured within a predetermined time span based on the specific time point (step S203). In step S203, for example, one image captured at the specific time point is extracted. Also, in step S203, for example, a plurality of images including one image captured at the specific time point may be extracted from among the images captured within a predetermined time span based on the specific time point.

[0096] In step S203, for example, a predetermined number of images captured within a period up to a predetermined time before the specific time point may be extracted. Also, in step S203, for example, a predetermined number of images captured within a predetermined period before and after the specific time point may be extracted.

[0097] In step S203, for example, the captured image is extracted from the images stored in the image temporary storage unit 25C. This allows efficient extraction. Note that in step S203, the captured image may be extracted from the images stored in the video storage unit 25B.

[0098] When the control unit 27 determines in step S201 that the vehicle M was not located at the set location (step S201: NO), it extracts one image or multiple images captured by the imaging unit at a timing that is a second time, different from the first time, prior to the time when the imaging instruction was received (step S204).

[0099] For example, the second time is set to a length of time that is preset by statistically taking into consideration the time it takes for control unit 27 to perform voice recognition or recognize a touch operation on a touch panel to receive an image capture instruction, as well as the time from when the user desires to capture an image to when the user issues an image capture instruction.

[0100] After executing step S203 or step S204, the control unit 27 ends the image extraction subroutine RT2.

[0101] According to the image extraction subroutine RT2, in step S203, by extracting images captured within a predetermined time period based on a specific point in time, it is possible to increase the probability of extracting an image that the user desires to capture.

[0102] Furthermore, step S204 of the image extraction subroutine RT2 can increase the probability of extracting the image the user desires, even if the image the user desires to capture is an image captured at a location unrelated to the set location.

[0103] Although one or more images are extracted by the image extraction subroutine RT2, when only one image is extracted, the capacity required to store the extracted image can be reduced. Also, when only one image is extracted, it becomes easier for the user to handle the captured image.

[0104] 7 is a flowchart showing an image extraction subroutine RT3, which is another example of an image extraction subroutine executed by the control unit 27 as an extraction unit in step S103 of the imaging routine RT1. The image extraction subroutine RT3 proceeds in the same manner as the image extraction subroutine RT2 (see FIG. 6) except for step S303, and therefore a part of the description will be omitted.

[0105] When the control unit 27 starts the image extraction subroutine RT3, if it determines that the vehicle M was located at a preset imaging point (set point) in the driving history referenced in step S102 of the imaging routine RT1 (step S301: YES), it identifies an image that was captured at the time when the vehicle M was located at the set point (step S302).

[0106] After executing step S302, the control unit 27 extracts a predetermined number of images captured by the imaging unit as captured images from the temporary image storage unit 25C when the vehicle M is located within a predetermined distance based on the set point (step S303).

[0107] In step S303, the "within a predetermined distance from the set point" does not mean a straight-line distance, but a predetermined section of the route traveled by the vehicle M, and may be a predetermined distance on the route.

[0108] In step S303, for example, one image captured when the vehicle M was located at the set point is extracted. In step S303, for example, a predetermined number of images captured when the vehicle M was located within a predetermined distance before and after the set point, with the set point as the center, may be extracted.

[0109] When the control unit 27 determines in step S301 that the vehicle M was not located at the set location (step S301: NO), it extracts one image or multiple images captured by the imaging unit at a timing that is a second time, different from the first time, prior to the time when the imaging instruction was received (step S304).

[0110] After executing step S303 or step S304, the control unit 27 ends the image extraction subroutine RT3.

[0111] In this way, according to the image extraction subroutine RT3, it is possible to extract an image captured when the vehicle M was located within a predetermined distance from the set point. Even with this type of image extraction method, it is possible to increase the probability of extracting an image that the user desires.

[0112] Fig. 8 is a flowchart showing an imaging routine RT4 which is another example of a routine executed by the control unit 27. The imaging routine RT4 differs from the imaging routine RT1 described in Fig. 6 in that it includes steps S402 and S403, but proceeds in the same manner in other respects, so some of the description will be omitted.

[0113] When starting the imaging routine RT4, the control unit 27 determines whether or not an imaging instruction has been received from the user (step S401).

[0114] In step S401, the control unit 27 accepts an image capture instruction based on the user's speech or a touch operation by the user. For example, the control unit 27 performs voice recognition on the voice uttered by the user input from the microphone 15, and when the control unit 27 recognizes a predetermined voice command indicating an image capture instruction following a wake word, for example, the control unit 27 determines that an image capture instruction from the user has been accepted. For example, the control unit 27 determines that an image capture instruction has been accepted when the control unit 27 recognizes an image capture instruction by a touch operation on the touch panel of the touch panel display 17.

[0115] When the control unit 27 determines in step S401 that an imaging instruction has been received (step S401: YES), it determines whether the received imaging instruction is an imaging instruction based on a user's utterance (step S402). In step S402, for example, when the control unit 27 has executed voice recognition in step S401, it determines that the imaging instruction is based on a user's utterance.

[0116] When the control unit 27 determines in step S402 that the imaging instruction is made by speech (step S402: YES), it sets a first time, which is the time for identifying the image to be extracted, going back from the time when the imaging instruction was received, to be longer than when the imaging instruction is received by the user's touch operation (step S403).

[0117] In step S403, for example, the first time period is set to be longer by the amount of time required for recognizing an image capture instruction from the user by voice recognition exceeding the time required for recognizing a touch operation on the touch panel.

[0118] When the control unit 27 determines in step S402 that the image capture instruction is not based on the user's utterance (step S402: NO), or after executing step S403, the control unit 27 refers to the travel history going back a first time (step S404). After that, the control unit 27 executes an image extraction subroutine to extract an image as a captured image (step S405).

[0119] In step S405, a subroutine such as the image extraction subroutine RT2 shown in FIG. 6 or the image extraction subroutine RT3 shown in FIG. 7 is executed.

[0120] After executing step S405, the control unit 27 transmits the extracted image to an external device such as a server (step S406), ends the imaging routine RT4, and repeats the imaging routine RT4 anew.

[0121] According to the imaging routine RT4, the first time can be appropriately set according to the operation method when the user issues an imaging instruction. Therefore, even if the time required for voice recognition is longer than the time required for touch operation recognition, for example, a captured image can be extracted based on the time when the vehicle M was located at the set point. Therefore, the probability of extracting an image showing the scenery that the user wants to capture can be increased.

[0122] In addition, in step S405 of the imaging routine RT4, for example, if the vehicle M is not located at the set location even when referring to the driving history going back the first time (step S201: NO or step S301: NO), and an imaging instruction based on the user's speech has been accepted (step S402: YES), the length of the second time in the subroutine (step S204 or step S304) may also be set to be longer than if an instruction based on the user's touch operation had been accepted.

[0123] As a result, even if the vehicle M is not located at the set location, the second time period can be appropriately set according to the operation method used by the user to issue an image capture instruction. Therefore, even if the time required for voice recognition is longer than the time required for touch operation recognition, for example, the probability of extracting an image showing the scenery the user wishes to capture can be increased.

[0124] As described above in detail, the imaging device 10 of this embodiment includes a memory unit that stores consecutive images captured recently by an imaging unit that moves together with a moving object, a reception unit that receives an imaging instruction through a user operation, and an extraction unit that, when the moving object is located at a preset imaging point within a period from the time when the reception unit receives the imaging instruction to a time going back one hour, extracts from the memory unit a predetermined number of images captured by the imaging unit at one or more timings based on the time when the moving object was located at the imaging point as captured images.

[0125] Therefore, according to the imaging device 10, when a moving object is located at a preset imaging point, it is possible to extract an image captured at a timing based on the time when the moving object was located at the imaging point.

[0126] An image captured at a timing based on the time when the moving object was located at the imaging point is highly likely to be an image showing the scenery that the user wishes to capture.

[0127] Therefore, according to the imaging device 10, it is possible to provide an imaging device, an imaging method, an imaging program, and a recording medium that enable an image capturing the scenery that the user wishes to capture to be obtained with a high probability in an imaging device that captures images in accordance with imaging instructions from a user.

[0128] The first time period can be set to any length of time, for example, a sufficiently long time that takes into account the time required to accept an image capture instruction from the user, as well as the difference between the time when the image the user wishes to capture is captured and the time when the user issues the image capture instruction.

[0129] In the above embodiment, the imaging device 10 is described as being capable of receiving an imaging instruction from a user via the touch panel display 17, but the present invention is not limited to this. For example, the imaging device 10 does not need to be connected to the touch panel display 17. For example, the vehicle M does not need to be provided with the touch panel display 17. For example, the imaging device 10 may receive an imaging instruction only by voice recognition. Also, for example, the imaging device 10 may be configured to be capable of receiving an imaging instruction by button operation using a push button that is not a touch panel display, in addition to voice recognition.

[0130] In the above embodiment, the imaging device 10 extracts captured images from images stored in the image temporary storage unit 25C of the large-capacity storage device 25, but the present invention is not limited to this. For example, the imaging device 10 may extract captured images from videos that are sequentially stored in the video storage unit 25B and held for a certain period of time.

[0131] In the above embodiment, an example of generating a still image from an image extracted as a captured image has been described, but the present invention is not limited to this. For example, the imaging device 10 may generate a video made up of a plurality of frame images extracted as captured images.

[0132] In the above embodiment, the imaging device 10 transmits the extracted captured image to the server, but the present invention is not limited to this. The imaging device 10 may transmit the extracted captured image to another external device such as a terminal owned by the user, or may store the captured image in a storage unit. The imaging device 10 may also display the extracted captured image on a display provided on a moving object.

[0133] The configurations and routines in the above-described embodiments are merely examples and can be appropriately selected and modified depending on the application, etc. [Explanation of symbols]

[0134] 10. Imaging device 11 Front camera 13 Side camera 15. Mike 17 Touch panel display 19 GNSS Receivers 21 System Bus 23 Input section 24 Sensors 25 Mass storage 25A Map information DB 25B Video storage section 25C Temporary Image Storage Unit 27 Control Unit 29 Output section 31 Transmitter / receiver

Claims

1. a storage unit that stores successive images captured recently by an imaging unit that moves together with the moving object; a reception unit that receives an image capture instruction by a user operation; an extracting unit that extracts, when the moving object is located at a preset imaging point within a period from the time when the accepting unit accepts an imaging instruction to a time point going back a first time, from the storage unit as captured images, a predetermined number of images captured by the imaging unit at one or more timings based on the time when the moving object was located at the imaging point; An imaging device comprising:

2. a position specifying unit that specifies a position on an electronic map corresponding to a current position of the moving object, 2. The imaging device according to claim 1, wherein the extraction unit extracts, as captured images, from the storage unit, a predetermined number of images captured by the imaging unit within a predetermined time width based on a point in time when the position identified by the position identification unit was located at the imaging point previously set in the electronic map.

3. a position specifying unit that specifies a position on an electronic map corresponding to a current position of the moving object, 2. The imaging device according to claim 1, wherein the extraction unit extracts, as captured images, from the storage unit, a predetermined number of images captured by the imaging unit when the position identified by the position identification unit is located within a predetermined distance based on the imaging point previously set in the electronic map.

4. the reception unit receives the image capture instruction based on a speech and a touch operation of the user; 4. The imaging device according to claim 1, wherein the extraction unit sets the first time longer when the reception unit receives the imaging instruction based on the utterance than when the reception unit receives the imaging instruction based on the contact operation.

5. 5. The imaging device according to claim 1, wherein the extraction unit extracts, from the memory unit, as captured images, a predetermined number of images captured by the imaging unit at a timing that is a second time prior to the time when the imaging instruction was received, when the moving body was not located at a predetermined imaging point within a period from the time when the reception unit received the imaging instruction to the time that is the first time prior.

6. the reception unit receives the image capture instruction based on a speech and a touch operation of the user; 6. The imaging device according to claim 5, wherein, when the moving body is not located at a predetermined imaging point within a period from the time when the reception unit receives an imaging instruction to the time going back the first time, the extraction unit sets the second time longer when the reception unit receives the imaging instruction based on the utterance than when the reception unit receives the imaging instruction based on the contact operation.

7. 7. The imaging device according to claim 1, wherein the extraction section extracts one image captured by the imaging section at one timing based on a time point when the moving object was located at the imaging point.

8. The imaging device according to claim 1 , wherein the predetermined imaging points include a point that is set as a scenic spot.

9. The imaging device according to claim 1 , further comprising a transmission unit configured to transmit the image extracted by the extraction unit to an external device.

10. An imaging method performed by an imaging device, comprising: a storage step of storing in a storage unit consecutive images captured recently by an imaging unit moving together with the moving object; a receiving step of receiving an image capturing instruction by a user operation; an extraction step of extracting, when the moving object is located at a preset imaging point within a period from the time when the imaging instruction is received in the receiving step to a time going back a first time, from the storage unit as captured images, a predetermined number of images captured by the imaging unit at one or more timings based on the time when the moving object was located at the imaging point; An imaging method comprising:

11. An imaging program executed by an imaging apparatus including a computer, the computer comprising: a storage step of storing in a storage unit consecutive images captured recently by an imaging unit moving together with the moving object; a receiving step of receiving an image capturing instruction by a user operation; an extraction step of extracting, when the moving object is located at a preset imaging point within a period from the time when the imaging instruction is received in the receiving step to a time going back a first time, from the storage unit as captured images, a predetermined number of images captured by the imaging unit at one or more timings based on the time when the moving object was located at the imaging point; An imaging program that executes the above.

12. A computer-readable recording medium storing the imaging program according to claim 11.

Images