Information processing device

Abstract

A detection target is set according to various imaging circumstances, and the fact that the detection target has been successfully imaged is confirmed in real time.　An information processing device according to the present invention collects, in an information collection unit, basic information including travel information of a mobile body and peripheral information of the mobile body. An imaging unit continuously performs imaging. Either a case in which the basic information satisfies a prescribed condition in an evaluation target image setting unit or a case in which an instruction operation is received serves as a trigger, and images from during a prescribed period of time from among the images imaged by the imaging unit are set as evaluation target images. A detection target setting unit sets the detection target on the basis of the basic information and the trigger related to the evaluation target images. The output unit issues a notification that the detection target set from among the evaluation target images exists.

Description

[Technical field]

[0001] The present invention relates to an information processing device. [Background technology]

[0002] A drive recorder including a camera mounted on a vehicle and a recording means for recording images captured by the camera is known (see, for example, Patent Document 1). Patent document 1 describes a drive recorder that includes a determination means for determining whether or not a specified object is captured in an image, and a transmission means for transmitting the image of the specified object to a server device when the determination means determines that the specified object has been captured. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2021-002163 A Summary of the Invention [Problem to be solved by the invention]

[0004] In a drive recorder such as that described in Patent Document 1, the user must set in advance in detail a detection target for determining whether or not the user has actually captured what the user wishes to record in the video, such as cracks or ice on the road surface. For this reason, when trying to set a detection target according to various changing shooting conditions, every situation must be assumed, and it may be difficult to set the detection target in advance. Therefore, it is difficult to set a detection target according to various shooting conditions, and it is difficult to confirm in real time that a detection target according to the shooting conditions has been captured.

[0005] One example of a problem to be solved by the present invention is to set a detection target according to various shooting conditions and to confirm in real time that the detection target has been photographed. [Means for solving the problem]

[0006] In order to solve the above problem, the present invention includes an information collecting unit that collects basic information including travel information of a moving object and information about the surroundings of the moving object, an image capturing unit that continuously captures images, an evaluation target image setting unit that sets images of a predetermined time period from among images captured by the image capturing unit as evaluation target images when the basic information satisfies a predetermined condition or when an instruction operation is received, as a trigger, and a setting unit that sets images of a predetermined time period from among images captured by the image capturing unit as evaluation target images based on the trigger and the basic information related to the evaluation target images. Among the evaluation target images, the image included in the evaluation target image The apparatus is characterized by comprising a detection target setting unit which sets a detection target, and an output unit which notifies the presence of the detection target set in the evaluation target image.

[0007] The invention described in claim 2 includes an information collecting unit that collects basic information including travel information of a moving object and information about the surroundings of the moving object, an image capturing unit that continuously captures images, an evaluation target image setting unit that sets images of a predetermined time period from among images captured by the image capturing unit as evaluation target images when the basic information satisfies a predetermined condition or when an instruction operation is received, as a trigger, and a setting unit that sets images of a predetermined time period from among images captured by the image capturing unit as evaluation target images based on the trigger and the basic information related to the evaluation target images. Among the evaluation target images, the image included in the evaluation target image The apparatus is characterized by comprising a detection target setting unit which sets a detection target, and an output unit which notifies the user that the set detection target is not present in the evaluation target image.

[0008] The invention described in claim 3 includes an information collecting unit that collects basic information including travel information of a moving object and information about the surroundings of the moving object, an image capturing unit that continuously captures images, an evaluation target image setting unit that sets images of a predetermined time period from among images captured by the image capturing unit as evaluation target images when the basic information satisfies a predetermined condition or when an instruction operation is received, as a trigger, and a setting unit that sets images of a predetermined time period from among images captured by the image capturing unit as evaluation target images based on the trigger and the basic information related to the evaluation target images. Among the evaluation target images, the image included in the evaluation target image The apparatus is characterized by comprising a detection target setting section which sets a detection target, and an output section which notifies the presence or absence of the detection target set in the evaluation target image.

[0009] The invention described in claim 11 is an information processing method executed by a computer, comprising: an information collecting step of collecting basic information including travel information of a moving object and information about the surroundings of the moving object; an imaging step of continuously imaging; an evaluation target image setting step of setting images of a predetermined time period from among images captured in the imaging step as evaluation target images, triggered by either the basic information satisfying a predetermined condition or an instruction operation being received; and a step of setting a predetermined number of images as evaluation target images based on the trigger and the basic information related to the evaluation target images. Among the evaluation target images, the image included in the evaluation target image The method is characterized by comprising a detection target setting step of setting a detection target, and an output step of notifying that the set detection target is present in the evaluation target image.

[0010] The invention as recited in claim 12 is characterized in that the information processing method as recited in claim 11 is executed by a computer as an information processing program.

[0011] The invention described in claim 13 is characterized in that the information processing program described in claim 12 is stored in a computer-readable storage medium. [Brief description of the drawings]

[0012] [Figure 1] 1 is a schematic configuration diagram of an information processing device according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a configuration diagram of a terminal device shown in FIG. [Diagram 3] FIG. 2 is a configuration diagram of a server device shown in FIG. [Figure 4] FIG. 4 is a diagram showing types of classification performed by a scene identification unit shown in FIG. 3. [Diagram 5] 2 is a flowchart of an operation of the information processing device shown in FIG. 1. [Figure 6] 6 is a flowchart showing the detailed operation of step S500 shown in FIG. 5. [Figure 7] FIG. 11 is a schematic configuration diagram of an information processing device according to a second embodiment of the present invention. [Figure 8] FIG. 11 is a schematic configuration diagram of an information processing device according to a third embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an information processing device according to an embodiment of the present invention will be described. In the information processing device according to an embodiment of the present invention, the information collecting unit collects basic information including travel information of a moving object and information about the surroundings of the moving object. Then, the photographing unit continuously photographs. Then, the evaluation target image setting unit sets images for a predetermined time from among the images photographed by the photographing unit as evaluation target images, triggered by either the basic information satisfying a predetermined condition or the instruction operation being received. Then, the detection target setting unit sets a detection target based on the trigger and basic information related to the evaluation target image. Then, the output unit notifies the user that the detection target set from the evaluation target image was present. In this way, the detection target is set based on the trigger and basic information related to the evaluation target image, so that the user of the information processing device can reduce the effort of setting the detection target in detail in advance. Then, since the output unit notifies the driver that the detection target was present, the driver can confirm in real time that the image of the detection target has been acquired. Therefore, the detection target can be set according to various shooting situations, such as when the basic information satisfies a predetermined condition or when an instruction operation is received, and it is possible to confirm in real time that the detection target has been photographed.

[0014] In addition, in the information processing device according to an embodiment of the present invention, the information collecting unit collects basic information including travel information of the moving object and information about the surroundings of the moving object. Then, the photographing unit continuously photographs. Then, the evaluation target image setting unit sets images for a predetermined time from among the images photographed by the photographing unit as evaluation target images, triggered by either the basic information satisfying a predetermined condition or the instruction operation being received. Then, the detection target setting unit sets the detection target based on the trigger and basic information related to the evaluation target image. Then, the output unit notifies the user that the detection target set from the evaluation target image was not present. In this way, the detection target is set based on the trigger and basic information related to the evaluation target image, so that the user of the information processing device can reduce the effort required to set the detection target in detail in advance. Then, since the output unit notifies the driver that there was no detection target, the driver can confirm in real time that the image of the detection target could not be acquired. Therefore, the detection target can be set according to various shooting situations, such as when the basic information satisfies a predetermined condition or when an instruction operation is received, and it is possible to confirm in real time that the detection target could not be photographed. This also enables quick action to be taken when an image of the detection object cannot be captured.

[0015] In addition, in the information processing device according to an embodiment of the present invention, the information collecting unit collects basic information including the traveling information of the moving body and the surrounding information of the moving body. Then, the photographing unit continuously photographs. Then, the evaluation target image setting unit sets images for a predetermined time from among the images photographed by the photographing unit as evaluation target images, triggered by either the basic information satisfying a predetermined condition or the instruction operation being received. Then, the detection target setting unit sets the detection target based on the trigger and basic information related to the evaluation target image. Then, the output unit notifies the presence or absence of the detection target set from the evaluation target image. In this way, the detection target is set based on the trigger and basic information related to the evaluation target image, so that the user of the information processing device can reduce the effort of setting the detection target in detail in advance. Then, since the presence or absence of the detection target is notified by the output unit, the driver can check in real time whether or not the image of the detection target has been acquired. Therefore, the detection target can be set according to various shooting situations, such as when the basic information satisfies a predetermined condition or when the instruction operation is received, and whether or not the detection target has been photographed can be checked in real time.

[0016] The output unit may be configured as an audio output device that outputs audio. In this way, the presence or absence of the detection object, or the presence or absence of the detection object is output as audio. For example, in the case where the information processing device has an image display function, the driver can check in real time whether or not the image of the detection object has been acquired without interrupting driving or diverting his / her attention from driving to check the image. In addition, since the output unit can be configured as an audio output device, the present invention can also be applied to a terminal that does not have an image display function.

[0017] The information collecting unit, the photographing unit, the evaluation target image setting unit, and the output unit may be provided in a terminal device that moves together with the moving object. The detection target setting unit may be provided in a server device that can communicate with the terminal device. The evaluation target images may be composed of upload target images that can be uploaded to the server device, and the upload target images may be transmitted from the terminal device to the server device, and the detection target information related to the detection target may be transmitted from the server device to the terminal device. In this way, the detection target can be set on the server device side, thereby reducing the load on the terminal device side.

[0018] The photographing unit, the evaluation target image setting unit, and the output unit may be provided in a terminal device that moves together with the moving object. The information collecting unit and the detection target setting unit may be provided in a server device that can communicate with the terminal device. The evaluation target images may be composed of upload target images that can be uploaded to the server device, and the upload target images may be transmitted from the terminal device to the server device, and the detection target information relating to the detection target object may be transmitted from the server device to the terminal device. In this way, the basic information can be collected on the server device side, thereby reducing the load on the terminal device side.

[0019] The information collecting unit, the photographing unit, the evaluation target image setting unit, the detection target setting unit, and the output unit may be provided in a terminal device that moves with the moving body. In this case, the evaluation target image may be configured as a recorded image that cannot be overwritten. In this way, the information processing device can be configured in the terminal device without going through a server device or the like.

[0020] Furthermore, the instruction operation is performed by the voice of the operator who performs the instruction operation, and the voice does not need to include the detection target. In this way, the instruction operation that triggers the setting of the evaluation target image can be performed by the operator's voice. And since the detection target does not need to be included in this voice, for example, the operator only needs to simply say "take a picture," and the effort required to set the detection target in detail can be reduced.

[0021] The travel information may include at least one of the following: acceleration information of the moving object, position information of the moving object, information on the moving direction of the moving object, and parking or stopping information of the moving object. The surrounding information may include at least one of the following: time information, weather information, and traffic congestion information. The detection object may include at least one of the following: a moving object other than the moving object, a license plate, a person, an animal other than a human, a traffic light, a road sign, a road marking, and a predetermined landmark. In this way, various shooting situations can be used as triggers for setting an evaluation target image, and various detection objects according to the shooting situations can be set from the evaluation target image. For this reason, for example, when a moving object detects sudden braking, the acceleration information can be used as a trigger to set an evaluation target image, and a moving object other than the moving object (in this case, a moving object that may have caused the sudden braking) or the license plate of the moving object can be set as a detection target from the evaluation target image.

[0022] Furthermore, if the detection object output is not the desired object desired by the operator performing the instruction operation and the operator performs a re-instruction operation, the output unit may notify the operator that the desired object was found in the evaluation target image, that the desired object was not found, or whether the desired object was found or not. In this manner, the operator who has received the notification from the output unit can be notified again about the presence or absence of the desired object, so that even if the detection object set by the detection target setting unit is different from the operator's desire, the operator can reliably confirm the presence or absence of the desired object.

[0023] Moreover, an information processing method according to an embodiment of the present invention is an information processing method executed by a computer. In the information processing method, in an information collecting step, basic information including travel information of a moving object and information about the surroundings of the moving object is collected. Then, in a photographing step, photographing is performed continuously. Then, in an evaluation target image setting step, images for a predetermined time from among the images photographed in the photographing step are set as evaluation target images, triggered by either a case where the basic information satisfies a predetermined condition or a case where an instruction operation is received. Then, in a detection target setting step, a detection target is set based on the trigger and basic information related to the evaluation target image. Then, in an output step, a notification is given that the detection target set was found from among the evaluation target images. In this way, a detection target is set according to various photographing situations, such as a case where the basic information satisfies a predetermined condition or a case where an instruction operation is received, and it is possible to confirm in real time that the detection target has been photographed.

[0024] The above-mentioned information processing method may be executed by a computer as an information processing program. In this way, it is possible to set a detection object according to various shooting situations, such as when the basic information satisfies a predetermined condition or when an instruction operation is received, and to confirm in real time that the detection object has been shot.

[0025] The above-mentioned information processing program may be stored in a computer-readable storage medium, which allows the information processing program to be distributed as a standalone program in addition to being incorporated into a device, and allows for easy version upgrades, etc. EXAMPLES

[0026] 1 is a schematic configuration diagram of an information processing device 100 according to a first example of the present embodiment. The information processing device 100 includes a terminal device 200 and a server device 300. The terminal device 200 is, for example, a device (e.g., a drive recorder) that moves with a vehicle (mobile body) or a device (e.g., a smartphone) that is carried by a person. The server device 300 is, for example, a device owned by an administrator who manages information obtained from the terminal device 200, and is provided so as to be able to communicate with the terminal device 200.

[0027] As shown in FIG. 2, the terminal device 200 includes an image capturing section 210, a control section 220, a storage section 230, an input section 240, an output section 250, and a communication section 260.

[0028] The photographing unit 210 is composed of, for example, an outside-vehicle photographing camera (not limited to a front camera, but may also include a side or rear camera), an inside-vehicle photographing camera, etc. The control unit 220 is composed of, for example, a CPU (Central Processing Unit), etc. The storage unit 230 is composed of a hard disk, memory, etc. The input unit 240 is composed of, for example, devices such as a microphone, input buttons, etc. The output unit 250 is composed of, for example, devices such as a speaker (an audio output device that outputs audio). The communication unit 260 is composed of a communication device for transmitting and receiving information to and from other devices.

[0029] The photographing unit 210 is provided integrally with the terminal device 200 or separately, and obtains image data 400 (images) of the inside and outside of the vehicle by continuously photographing the vehicle, and temporarily stores the image data 400 in the storage unit 230 in a ring buffer format.

[0030] The control unit 220 is a part that performs processing for operating the information processing device 100, and includes an information collection unit 221 and an evaluation target image setting unit 222.

[0031] The information collecting unit 221 performs a process of collecting basic information 500. The basic information 500 is information necessary for estimating the situation when the photographing unit 210 photographed, and includes traveling information 600 of the moving body and surrounding information 700 of the moving body. The traveling information 600 includes, for example, at least one of vehicle acceleration information, vehicle position information, vehicle traveling direction information, and vehicle parking or stop information. The surrounding information 700 includes, for example, at least one of time information such as date and time, weather information, and traffic congestion status information. The surrounding information 700 may also include aggressive driving information on vehicles surrounding the vehicle.

[0032] The information collection unit 221 acquires basic information 500 in real time from, for example, a GPS (Global Positioning System) receiver, various sensors such as a gyro sensor and an acceleration sensor, or an external device that transmits weather information or traffic information, and stores it in the memory unit 230.

[0033] The evaluation target image setting unit 222 is triggered when the basic information 500 satisfies a predetermined condition or when the input unit 240 receives a first instruction operation s1 (instruction operation), and sets image data for a predetermined time period including the time of detection of the trigger from the image data 400 captured by the imaging unit 210 as an upload target image 800 (evaluation target image). The evaluation target image setting unit 222 includes a trigger detection unit 2221 and an extraction unit 2222.

[0034] The trigger detection unit 2221 monitors various sensors such as the gyro sensor and acceleration sensor described above, the storage unit 230, or the storage unit 320 of the server device 300 to detect that the basic information 500 satisfies a predetermined condition. An example of the predetermined condition is that "vehicle acceleration information has changed suddenly beyond a predetermined threshold value." This condition is satisfied, for example, when the vehicle suddenly accelerates, suddenly decelerates, or suddenly stops, or when the vehicle comes into contact with another vehicle, or when a parked vehicle is tampered with and an impact is applied to the vehicle, etc.

[0035] This predetermined condition is merely one example, and can be flexibly changed according to the user's wishes. For example, the predetermined condition may be "the vehicle has reached the vicinity of a specific location such as a famous tourist spot or a location where traffic accidents frequently occur," or "the weather has suddenly changed," "the aggressive driving behavior of another vehicle has been detected," or the like.

[0036] The trigger detection unit 2221 detects that the basic information 500 satisfies the above-mentioned preset conditions. The trigger detection unit 2221 also detects that the input unit 240 has received a first instruction operation s1 (instruction operation). The first instruction operation s1 refers to an operation performed by a user (operator) of the information processing device 100, and in this embodiment, refers to an operation performed mainly by the user's voice. This first instruction operation s1 may be a simple operation that does not specify in advance in detail the object to be detected, such as "take a picture," "start shooting," or "leave the video." In other words, the voice does not need to include the detection object 810 set by the detection object setting unit 311 described later. Furthermore, the first instruction operation s1 is not limited to an input by voice, and may be, for example, an operation by an input button.

[0037] The detected object 810 refers to an object that a user of the information processing device 100 is expected to wish to photograph, and examples of such objects include moving objects other than a vehicle equipped with the terminal device 200 in this embodiment, vehicle license plates, people, non-human animals, traffic lights, road signs, road markings, and predetermined landmarks such as famous tourist spots and photography spots.

[0038] As described above, the shooting unit 210 continues shooting without receiving the first instruction operation s1, and therefore the user's commands "shoot", "start shooting", "keep the footage", etc. here indicate the user's intention to prevent the image data 400 related to the first instruction operation s1 from being overwritten by new image data 400 by performing an overwrite prevention process, an upload process, etc.

[0039] When detecting either that "the basic information 500 satisfies a predetermined condition" or that "the input unit 240 has received the first instruction operation s1", the extraction unit 2222 sets image data for a predetermined time period including the time of detection of the trigger among the image data 400 temporarily stored in the storage unit 230 as an upload target image 800 that can be uploaded to the server device 300, and extracts it from the image data 400. The upload target image 800 is an image that constitutes the evaluation target image in the above-mentioned embodiment. Note that the time period of the upload target image 800 may be dynamically changed according to the type of trigger detected by the trigger detection unit 2221, or may be constant regardless of the type of trigger.

[0040] In this way, when the evaluation target image setting section 222 detects the above-mentioned trigger, it sets image data 400 for a predetermined time period including the time of detection of the trigger from among the image data 400 captured by the imaging section 210 as an upload target image 800 (evaluation target image). Then, the evaluation target image setting section 222 adds information about the trigger detected by the trigger detection section 2221 and basic information 500 corresponding to the time of the upload target image 800 to the extracted upload target image 800, and performs processing to transmit the information to the server device 300 via the communication section 260.

[0041] The input unit 240 receives the input of the first instruction operation s1 by collecting sound. The input unit 240 also receives the input of a second instruction operation s2 (a second instruction operation) in addition to the first instruction operation s1.

[0042] The second instruction operation s2 is an operation performed after the first instruction operation s1, and is an operation performed when, after the presence or absence of the detected object 810 is output by voice from the output unit 250 as described below, the detected object 810 is not the desired object 820 desired by the user. This second instruction operation s2 is performed by the user asking, for example, "Was 〜(the desired object 820) not shown?" to prompt the information processing device 100 to re-determine whether or not the desired object 820 was shown in the image 800 to be uploaded.

[0043] In this embodiment, the input unit 240 is configured to accept voice input, but, for example, input buttons corresponding to the first instruction operation s1 and the second instruction operation s2 may be provided on the terminal device 200, and the input of the first instruction operation s1 and the second instruction operation s2 may be accepted by the input buttons.

[0044] The output unit 250 outputs a sound indicating that the detection target 810 set in the upload target image 800 was present, that the detection target 810 was not present, or whether the detection target 810 is present or not. Specifically, if the content of the detection target information 900 described below received by the communication unit 260 from the server device 300 is "detection target 810 present", the output unit 250 outputs a sound such as "~ (detection target 810) was photographed" based on the result. If the content of the detection target information 900 is "detection target 810 not present", the output unit 250 outputs a sound such as "detection target 810 could not be photographed (or could not be detected)" based on the result. If the content of the detection target information 900 is "one detection target 810 present, other detection targets 810 not present", the output unit 250 outputs a sound such as "~ (one detection target 810) was photographed, but ~ (other detection targets 810) was not photographed".

[0045] Furthermore, when the detected object 810 output as a voice is not the desired object 820 desired by the user and the input unit 240 has accepted the second instruction operation s2 (when a re-instruction operation has been performed), the output unit 250 outputs a voice indicating that the desired object 820 was present in the upload target image 800, or that the desired object 820 was not present, or whether or not the desired object 820 exists. Specifically, if the content of the desired object information 1000 described below received by the communication unit 260 from the server device 300 is "desired object 820 exists", the output unit 250 outputs a voice such as "~(desired object 820) was photographed" based on the result. Furthermore, if the content of the desired object information 1000 is "desired object 820 not present", the output unit 250 outputs a voice such as "desired object 820 could not be photographed (or could not be detected)" based on the result. In addition, if the content of the desired object information 1000 is "There is one desired object 820, but no other desired object 820," the output unit 250 outputs a voice such as "We were able to photograph ~ (the one desired object 820), but we were unable to photograph ~ (the other desired object 820)."

[0046] The communication unit 260 transmits the upload target image 800 extracted by the extraction unit 2222 to the server device 300, and receives detection target information 900 from the server device 300. When the input unit 240 accepts the second instruction operation s2, the communication unit 260 transmits information on the second instruction operation s2 to the server device 300, and receives desired target information 1000 from the server device 300.

[0047] As shown in FIG. 3, the server device 300 includes a control unit 310, a storage unit 320, and a communication unit 330.

[0048] The control unit 310 is composed of, for example, a CPU (Central Processing Unit) etc. The storage unit 320 is composed of a hard disk and memory. The communication unit 330 is composed of a communication device for transmitting and receiving information to and from other devices.

[0049] The control unit 310 is a part that performs processing for operating the information processing device 100, and includes a detection target setting unit 311, a determination unit 312, and a re-determination unit 313. The detection target setting unit 311 is a part that sets a detection target 810 based on the above-mentioned trigger and basic information 500 related to the image to be uploaded 800, and includes an object identification unit 3111, a scene identification unit 3112, and a decision unit 3113.

[0050] The photographed object identification unit 3111 analyzes the image 800 to be uploaded, and identifies the photographed object in the image 800 to be uploaded. A known technique can be used for this identification, and a detailed description thereof will be omitted. For example, an image database 1100 of images taken by a vehicle may be stored in the storage unit 320, and the photographed object may be identified using a technique such as pattern matching.

[0051] The scene identification unit 3112 identifies the circumstances under which the image 800 to be uploaded was taken, and classifies the images by category, by referring to the trigger information attached to the image 800 to be uploaded and the basic information 500. As shown in FIG. 4, the categories include, for example, the following four categories:

[0052] The first category A is a category of manual photography at a photography point. This first category A indicates that the image related to the image to be uploaded 800 was mainly taken near famous tourist spots or photography spots. For example, the first category A is classified when the trigger related to the image to be uploaded 800 is the first instruction operation s1 by the user, and the position information of the vehicle related to the image to be uploaded 800 indicates the vicinity of the position information of any POI in the POI (Point Of Interest) database 1200 recorded in the storage unit 320.

[0053] The second category B is a category of manual photography in an emergency. This second category B indicates that the image related to the upload target image 800 is manually photographed by the user in an emergency, such as when the user mainly witnesses an accident involving another vehicle. This second category B is classified as a case where, for example, the trigger related to the upload target image 800 is the first instruction operation s1 of the user, the position information of the vehicle related to the upload target image 800 does not indicate the vicinity of the position information indicated by any POI in the POI database 1200 recorded in the storage unit 320, and the acceleration information of the vehicle related to the upload target image 800 has not changed beyond a predetermined threshold.

[0054] The third category C is a category of automatic photography while driving. This third category C indicates that the upload target image 800 was automatically photographed while driving and in an emergency, such as when the vehicle involved in an accident while driving. This third category C is classified as a case where, for example, the trigger related to the upload target image 800 is that the vehicle acceleration information has changed suddenly beyond a predetermined threshold, and the content of the basic information 500 indicates that the vehicle was driving.

[0055] The fourth category D is a category of automatic photography during parking. This fourth category D indicates that the upload target image 800 is automatically photographed during parking and in an emergency, such as when the vehicle is tampered with while parked and impacted. This fourth category D is classified as a case where, for example, the trigger related to the upload target image 800 is a sudden change in the vehicle acceleration information exceeding a predetermined threshold, and the content of the basic information 500 indicates that the vehicle was parked or stopped.

[0056] The determination unit 3113 determines the detection target object 810 from within the image 800 to be uploaded, with reference to the category classified by the scene identification unit 3112 .

[0057] Specifically, if the category of the image 800 to be uploaded is the first category A, the object that the user is expected to want to photograph is determined to be an object present at a famous tourist spot or photography spot, and one of the POIs in the POI database 1200 recorded in the memory unit 320 is determined to be the detected object 810.

[0058] In addition, if the category of the image 800 to be uploaded is the second category B, it is determined that the object that the user is expected to want to photograph is an object related to an accident other than the user's own vehicle, and vehicles other than the user's own vehicle, vehicle license plates, people, animals other than humans, traffic lights, road signs, road markings, etc. are determined to be detected objects 810.

[0059] In addition, if the category of the image 800 to be uploaded is the third category C, the objects that the user is expected to want to photograph are determined to be objects related to accidents while the vehicle is traveling, and vehicles other than the vehicle itself, vehicle license plates, people, animals other than people, traffic lights, road signs, road markings, etc. are determined to be detected objects 810.

[0060] Furthermore, when the category of the image 800 to be uploaded is the fourth category D, the object that the user is expected to wish to photograph is determined to be an object present in the vicinity of the vehicle, and people or non-human animals present in the vicinity of the vehicle, vehicles other than the vehicle, vehicle license plates, etc. are determined to be detected objects 810. At this time, other vehicles that are moving at the time of vibration detection or other vehicles with door opening / closing operations are set as detected objects 810, and other vehicles that continue to be parked in the parking lot and have no door opening / closing operations may be excluded as detected objects 810.

[0061] The determination unit 312 determines the presence or absence of the detection target object 810 determined by the determination unit 3113 from within the upload target image 800, and transmits the result as detection target information 900 from the server device 300 to the terminal device 200 via the communication unit 330. The detection target information 900 may be information simply about the presence or absence of the detection target object 810, such as "detection target object 810 present", "detection target object 810 absent", "one detection target object 810 present, other detection target object 810 absent", or the like, or may be information to which any of the above-mentioned first category A to fourth category D has been assigned. Also, it may include the number of detection targets 810.

[0062] When the communication unit 330 receives information on the above-mentioned second instruction operation s2 from the terminal device 200, the re-determination unit 313 determines whether or not there is a desired object 820 related to the second instruction operation s2 from the upload target image 800, and transmits the result as desired object information 1000 from the server device 300 to the terminal device 200 via the communication unit 330. The desired object information 1000 may be information simply about the presence or absence of the desired object 820, such as "desired object 820 present", "desired object 820 not present", "one desired object 820 present, other desired object 820 not present", or may be information to which any of the above-mentioned first category A to fourth category D has been assigned. In addition, it may include the number of desired objects 820.

[0063] The storage unit 320 stores the images 800 to be uploaded transmitted from the terminal device 200. The storage unit 320 also stores the image database 1100 and the POI database 1200 described above.

[0064] The communication unit 330 receives the upload target image 800 from the communication unit 260 of the terminal device 200, and transmits the detection target information 900 generated by the determination unit 312 to the terminal device 200. The communication unit 330 also receives information of the second instruction operation s2 from the communication unit 260 of the terminal device 200, and transmits the desired target information 1000 generated by the re-determination unit 313 to the terminal device 200.

[0065] Next, the operation (information processing method) of the information processing device 100 having the above-mentioned configuration will be described with reference to the flowcharts of Figures 5 and 6. The flowcharts of Figures 5 and 6 can be configured as an information processing program by being executed by a computer having a CPU or the like. This information processing program may be stored in a computer-readable storage medium.

[0066] First, in the terminal device 200, the photographing unit 210 starts continuous photographing, and stores the obtained image data 400 in the storage unit 230. In addition, the information collecting unit 221 starts acquiring basic information 500 from various sensors such as a GPS (Global Positioning System) receiver, a gyro sensor, an acceleration sensor, or an external device that transmits weather information or traffic information, and stores the acquired basic information 500 in the storage unit 230 every hour (step S100). This step S100 corresponds to the information collecting step and the photographing step in the above-mentioned embodiment.

[0067] Next, the evaluation target image setting unit 222 monitors the storage unit 230 and the input unit 240 by the trigger detection unit 2221, and determines whether or not a trigger is detected (step S200). As described above, the trigger is either "the basic information 500 satisfies a predetermined condition" or "the input unit 240 accepts the first instruction operation s1 (instruction operation)."

[0068] The predetermined condition in this case may be set to various conditions according to the user's desire, such as "vehicle acceleration information has changed abruptly beyond a predetermined threshold."

[0069] The first instruction operation s1 is performed by the user's vocalization, and may be a simple command such as "Take a picture," "Start filming," or "Leave the footage," which does not specify in detail the object to be detected, and the vocalization does not have to include the object to be detected 810.

[0070] If the result of the determination in step S200 is YES, the process proceeds to step S300, and if the result is NO, the process returns to step S100.

[0071] In step S300, the extraction unit 2222 of the evaluation target image setting unit 222 sets image data for a predetermined time period including the time of trigger detection from the image data 400 temporarily stored in the storage unit 230 as the upload target image 800, and extracts it from the image data 400. Note that at this time, the time period of the upload target image 800 may be dynamically changed according to the type of trigger detected by the trigger detection unit 2221, or may be set to be constant regardless of the type of trigger.

[0072] Then, the extracted image 800 to be uploaded is assigned with the trigger information detected by the trigger detection unit 2221 and the basic information 500 corresponding to the time of the image 800 to be uploaded, and transmitted to the server device 300 via the communication unit 260 (step S400). The above steps S200 to S400 correspond to the evaluation target image setting step in the above embodiment. When step S400 is completed, the process proceeds to step S500.

[0073] In step S500, the server device 300 sets a detection object 810 and determines whether or not the detection object 810 is present in the image to be uploaded 800. In this step S500, as shown in detail in Fig. 6, first, the photographed object identifying unit 3111 analyzes the image to be uploaded 800 using a known method such as pattern matching, and identifies the photographed object present in the image to be uploaded 800 (step S501).

[0074] Next, the scene identification unit 3112 identifies the situation in which the image to be uploaded 800 was taken, and classifies it into categories, with reference to the trigger information and basic information 500 attached to the image to be uploaded 800 (step S502). In this step S502, the image to be uploaded 800 is classified into a first category A, a second category B, a third category C, or a fourth category D according to the contents of the trigger and basic information 500.

[0075] For example, if the trigger for the image 800 to be uploaded is the user's first instruction operation s1, and the location information of the vehicle related to the image 800 to be uploaded indicates the vicinity of the location information of any POI in the POI (Point Of Interest) database 1200 recorded in the memory unit 320, the image 800 to be uploaded is classified into the first category A.

[0076] Also, for example, if the trigger related to the image 800 to be uploaded is the user's first instruction operation s1, the location information of the vehicle related to the image 800 to be uploaded does not indicate the vicinity of the location information indicated by any POI in the POI database 1200 recorded in the memory unit 320, and the acceleration information of the vehicle related to the image 800 to be uploaded has not changed beyond a predetermined threshold, the image 800 to be uploaded is classified into the second category B.

[0077] In addition, for example, if the trigger for the image 800 to be uploaded is a sudden change in the vehicle acceleration information exceeding a predetermined threshold and the contents of the basic information 500 indicate that the vehicle was moving, the image 800 to be uploaded is classified into the third category C.

[0078] Furthermore, for example, if the trigger for the image 800 to be uploaded is a sudden change in the vehicle's acceleration information exceeding a predetermined threshold, and the contents of the basic information 500 indicate that the vehicle was parked or stopped, the image 800 to be uploaded is classified as the fourth category D.

[0079] When the image 800 to be uploaded is classified into each category in step S502, the determination unit 3113 determines the detection object 810 according to the category (step S503).

[0080] For example, if the category of the image 800 to be uploaded is the first category A, the determination unit 3113 determines that the object that the user is expected to want to photograph is an object that exists at a famous tourist spot or photography spot, and determines one of the POIs in the POI database 1200 recorded in the memory unit 320 as the detected object 810.

[0081] In addition, if the category of the image 800 to be uploaded is the second category B, the determination unit 3113 determines that the object that the user is expected to want to photograph is an object related to an accident other than the user's own vehicle, and determines that the detected object 810 is a vehicle other than the user's own vehicle, a vehicle license plate, a person, an animal other than a person, a traffic light, a road sign, a road marking, etc.

[0082] In addition, if the category of the image 800 to be uploaded is the third category C, the determination unit 3113 determines that the object that the user is expected to want to photograph is an object related to an accident while the vehicle is traveling, and determines that the detected object 810 is a vehicle other than the vehicle, a vehicle license plate, a person, an animal other than a person, a traffic light, a road sign, a road marking, etc.

[0083] Furthermore, when the category of the image 800 to be uploaded is the fourth category D, the determination unit 3113 determines that the object that the user is expected to wish to photograph is an object existing in the vicinity of the vehicle, and determines a person or a non-human animal existing in the vicinity of the vehicle, a vehicle other than the vehicle, a vehicle license plate, etc., as the detection object 810. The above steps S501 to S503 correspond to the detection object setting process in the above-mentioned embodiment.

[0084] When the detection object 810 is determined in step S503, the determination unit 312 determines whether the detection object 810 is present in the upload target image 800 (step S504), and transmits the result as detection object information 900 from the server device 300 to the terminal device 200 via the communication unit 330 (step S505). The contents of the detection object information 900 may be, for example, "detection object 810 present", "detection object 810 absent", "one detection object 810 present, other detection object 810 absent", or the like, simply information on the presence or absence of the detection object 810, ...

[0085] Step S600 corresponds to the output step in the above embodiment. In step S600, in the terminal device 200, the communication unit 260 receives the detection target information 900, and based on the contents of the detection target information 900, the output unit 250 outputs audio.

[0086] For example, if the content of the detection target information 900 is "detection target 810 present", the output unit 250 outputs a sound such as "~ (detection target 810) was photographed" based on the result. Also, if the content of the detection target information 900 is "detection target 810 not present", the output unit 250 outputs a sound such as "detection target 810 could not be photographed" based on the result. Also, if the content of the detection target information 900 is "one detection target 810 present, other detection targets 810 not present", the output unit 250 outputs a sound such as "~ (one detection target 810) was photographed, but ~ (other detection targets 810) could not be photographed". When the sound output is completed, the process proceeds to step S700.

[0087] In step S700, the second instruction operation s2 is accepted. In this step S700, if the input unit 240 accepts the second instruction operation s2 (in the case of YES), the communication unit 260 transmits information on the second instruction operation s2 to the server device 300, and the process proceeds to step S800. If the input unit 240 does not accept the second instruction operation s2 (in the case of NO), step S700 is ended, and the operation of the information processing device 100 ends. The fact that the second instruction operation s2 has not been accepted may be determined based on a condition such as a preset acceptance time having elapsed without any input to the input unit 240, or a condition such as a voice output from the output unit 250 of "Are there any other instructions?" after the acceptance time has elapsed to the input unit 240, and a predetermined time has elapsed without any input to the input unit 240 after confirmation from the user.

[0088] In step S800, the server device 300 that has received the second instruction operation s2 uses the re-determination unit 313 to determine whether or not there is a desired object 820 related to the second instruction operation s2 in the upload target image 800, and transmits the result to the terminal device 200 as desired object information 1000. The desired object information 1000 may be information on the presence or absence of a desired object 820, such as, for example, "desired object 820 present", "desired object 820 not present", "one desired object 820 present, other desired object 820 not present", or the like, or may be information that simply indicates the presence or absence of a desired object 820, or the presence or absence of a desired object 820, or may be information that is assigned any of the above-mentioned first category A to fourth category D, or may include the number of desired objects 820.

[0089] When the terminal device 200 receives the desired target information 1000, the output unit 250 outputs voice based on the contents of the desired target information 1000 (step S900).

[0090] For example, if the content of the desired object information 1000 is "desired object 820 is present", the output unit 250 outputs a voice such as "We were able to photograph ~ (desired object 820)" based on the result. Also, if the content of the desired object information 1000 is "desired object 820 is not present", the output unit 250 outputs a voice such as "We were unable to photograph the desired object 820" based on the result. Also, if the content of the desired object information 1000 is "one desired object 820 is present, but no other desired object 820", the output unit 250 outputs a voice such as "We were able to photograph ~ (one desired object 820), but were unable to photograph ~ (other desired object 820)". This ends the operation of the information processing device 100.

[0091] According to the first embodiment, the detection object 810 is set based on the trigger related to the upload target image 800 and the basic information 500, so that the user of the information processing device 100 can reduce the effort required for setting the detection object 810 in detail in advance. Then, the presence or absence of the detection object 810 is notified by the output unit 250, so that the driver can confirm in real time whether the detection object 810 has been acquired, whether the detection object 810 has not been acquired, or whether the detection object 810 has been acquired. Therefore, the detection object 810 is set according to various shooting situations, such as when the basic information 500 satisfies a predetermined condition or when the first instruction operation s1 is received, and it is possible to confirm in real time whether the detection object 810 has been photographed, whether it has not been photographed, or whether it has been photographed at all. In addition, the detection object 810 is selectively set, so that the processing cost of the information processing device 100 is reduced. Furthermore, by selectively notifying the user of the detection object 810 through audio output, it is possible to appropriately select and notify the user of required information without causing the user to feel bothered.

[0092] Furthermore, since it is possible to confirm in real time that the detection object 810 could not be photographed, it is possible to quickly respond to the fact that the detection object 810 could not be photographed.

[0093] Furthermore, the output unit 250 may be configured with a speaker (audio output device), and in this case, the presence or absence of the detection object 810, or the absence or presence of the detection object 810 is output as an audio signal, so that, for example, in a case where the information processing device 100 has an image display function, the driver can check in real time whether or not an image of the detection object 810 has been acquired without having to interrupt driving or divert attention from driving to check the image. Furthermore, since the output unit 250 can be configured with a speaker, the present invention can also be applied to a terminal that does not have an image display function.

[0094] Furthermore, since the detection object 810 can be set on the server device 300 side, the load on the terminal device 200 side can be reduced.

[0095] In addition, the first instruction operation s1 that triggers the setting of the image 800 to be uploaded can be performed by the user's voice, and the voice does not have to include the detection object 810. For example, the user only needs to say a simple utterance such as "Take a picture," thereby reducing the effort required to set the detection object 810 in detail.

[0096] In addition, various shooting situations can be used as triggers for setting the image to be uploaded 800, and various detected objects 810 can be set from the image to be uploaded 800 according to the shooting situation. For example, when a vehicle detects sudden braking, the image to be uploaded 800 can be set using acceleration information as a trigger, and from the image to be uploaded 800, a vehicle other than the vehicle (in this case, a vehicle that may have caused the sudden braking) or the license plate of that vehicle can be set as the detected object 810.

[0097] Furthermore, if the content of the notification by the output unit does not include information on the desired object 820, the user can be notified again about the presence or absence of the desired object 820. Therefore, even if the setting of the detection object 810 by the detection object setting unit 311 differs from the user's desire, the user can reliably check whether the desired object 820 is present. EXAMPLES

[0098] Next, an information processing device 100 according to a second example of the present embodiment will be described with reference to Fig. 7. Note that the same parts as those in the first example described above are given the same reference numerals and descriptions thereof will be omitted or simplified.

[0099] 7, in the information processing device 100 according to the present embodiment, an information collecting unit 314 is provided in a control unit 310 of a server device 300. This information collecting unit 314 corresponds to the information collecting unit 221 in the first embodiment. That is, the second embodiment differs from the first embodiment in that the information collecting unit 314 is implemented in the server device 300, and the basic information 500 is stored in a storage unit 320 on the server device 300 side.

[0100] The operation (information processing method) of the information processing device 100 having this configuration will be described. Although the device where the processing is performed differs from that of the first embodiment, the operation flow (steps) of the information processing device 100 of this embodiment is similar to that of the first embodiment, and therefore the operation will be described with reference to FIG.

[0101] First, in the terminal device 200, the photographing unit 210 starts continuous photographing, and the obtained image data 400 is stored in the storage unit 230. In addition, the information collecting unit 314 of the server device 300 accesses the terminal device 200, starts acquiring basic information 500 from various sensors such as a GPS (Global Positioning System) receiver, a gyro sensor, and an acceleration sensor, or an external device that transmits weather information or traffic information, and stores the acquired basic information 500 in the storage unit 320 every hour (step S100). This step S100 corresponds to the information collecting step and the photographing step in the above-mentioned embodiment.

[0102] Next, the evaluation target image setting unit 222 monitors the memory unit 320 of the server device 300 and the input unit 240 of the terminal device 200 by the trigger detection unit 2221, and judges whether or not a trigger is detected (step S200). The contents of the trigger are as described above. Then, if the result of the judgment in step S200 is YES, the process proceeds to step S300, and if the result is NO, the process returns to step S100.

[0103] In step S300, the extraction unit 2222 of the evaluation target image setting unit 222 sets image data for a predetermined period of time including the time when the trigger is detected from the image data 400 temporarily stored in the memory unit 230 as the upload target image 800 and extracts it from the image data 400.

[0104] Then, the extracted image 800 to be uploaded is assigned with the trigger information detected by the trigger detection unit 2221 and the basic information 500 corresponding to the time of the image 800 to be uploaded, and transmitted to the server device 300 via the communication unit 260 (step S400). The above steps S200 to S400 correspond to the evaluation target image setting step in the above embodiment. When step S400 is completed, the process proceeds to step S500.

[0105] In step S500, a detection target object 810 is set, and it is determined whether or not the detection target object 810 is present in the image to be uploaded 800. In step S500, first, the photographed object identification unit 3111 analyzes the image to be uploaded 800 using a known method such as pattern matching, and identifies the photographed object in the image to be uploaded 800. Next, the scene identification unit 3112 refers to the trigger information and basic information 500 assigned to the image to be uploaded 800 to identify the situation in which the image to be uploaded 800 was photographed, and classifies the image by category. These categories are the same as the first category A, second category B, third category C, and fourth category D in the first embodiment.

[0106] Once the upload target image 800 has been classified into each category, the determination unit 3113 determines the detection target object 810 according to the category. Next, once the detection target object 810 has been determined, the judgment unit 312 determines whether the detection target object 810 is present in the upload target image 800, and transmits the result as detection target information 900 from the server device 300 to the terminal device 200 via the communication unit 330. The contents of the detection target information 900 are the same as those in the first embodiment. This ends step S500. Then, the process proceeds to step S600.

[0107] Step S600 corresponds to the output step in the above embodiment. In step S600, in the terminal device 200, the communication unit 260 receives the detection target information 900, and the output unit 250 outputs audio based on the contents of the detection target information 900. The contents of the audio output are the same as those in the first embodiment. When the audio output is completed, the process proceeds to step S700.

[0108] In step S700, the second instruction operation s2 is accepted. In step S700, if the input unit 240 accepts the second instruction operation s2 (if YES), the process proceeds to step S800. If the input unit 240 does not accept the second instruction operation s2 (if NO), the process ends step S700, and the operation of the information processing device 100 ends.

[0109] In step S800, the re-determination unit 313 detects that the input unit 240 has received the second instruction operation s2, determines whether or not the desired object 820 related to the second instruction operation s2 is present in the image to be uploaded 800, and transmits the result to the terminal device 200 as desired object information 1000. The contents of the desired object information 1000 are the same as those in the first embodiment.

[0110] When the terminal device 200 receives the desired target information 1000, the output unit 250 outputs voice based on the contents of the desired target information 1000 (step S900). The contents of the voice output are the same as those in the first embodiment. This completes the operation of the information processing device 100.

[0111] According to the second embodiment, the basic information 500 can be collected on the server device 300 side, so that the load on the terminal device 200 side can be reduced. EXAMPLES

[0112] Next, an information processing device 100 according to a third example of the present embodiment will be described with reference to Fig. 8. Note that the same parts as those in the first example described above are given the same reference numerals and descriptions thereof will be omitted or simplified.

[0113] The information processing device 100 according to this embodiment omits the above-mentioned server device 300, and the configuration of the control unit 310 of the server device 300 is implemented in the control unit 220 of the terminal device 200. Also, the configuration of the storage unit 320 of the server device 300 is implemented in the storage unit 230 of the terminal device 200. That is, the information processing device 100 is configured by the terminal device 200 without the server device 300. The control unit 220 includes a detection target setting unit 223, a determination unit 224, and a re-determination unit 225 in addition to the configuration of the first embodiment. The detection target setting unit 223 corresponds to the detection target setting unit 311 in the server device 300. Although not shown, the detection target setting unit 223 includes a shooting object identification unit 3111, a scene identification unit 3112, and a decision unit 3113, as in the first embodiment. The determination unit 224 corresponds to the determination unit 312 in the server device 300. Redetermination unit 225 corresponds to redetermination unit 313 in server device 300. The configurations and operations of detection target setting unit 223, determination unit 224, and redetermination unit 225 are substantially the same as those in the first embodiment, and therefore detailed description thereof will be omitted.

[0114] In addition to the configuration of the first embodiment, the storage unit 230 stores an image database 1100 and a POI database 1200. The image database 1100 and the POI database 1200 are substantially the same as those stored in the storage unit 320 of the server device 300 in the first embodiment.

[0115] In this manner, the information collecting section 221, the photographing section 210, the evaluation target image setting section 222, the detection target setting section 223, and the output section 250 are provided in the terminal device 200 that moves together with the moving object.

[0116] In the first embodiment, when the trigger detection unit 2221 of the evaluation target image setting unit 222 detects a trigger, the extraction unit 2222 sets image data for a predetermined time period including the time of the trigger detection among the image data 400 temporarily stored in the storage unit 230 as the upload target image 800 as the evaluation target image, and extracts it from the image data 400. However, in this embodiment, since the server device 300 is not used, when the trigger detection unit 2221 detects a trigger, the extraction unit 2222 sets image data for a predetermined time period among the image data 400 temporarily stored in the storage unit 230 as the non-overwritable recorded image 1300, and extracts it from the image data 400. In this embodiment, this recorded image 1300 constitutes the evaluation target image in the above-mentioned embodiment.

[0117] Next, an operation (information processing method) of the information processing device 100 having the above-mentioned configuration will be described. Although the device where the processing is performed differs from that of the first embodiment, the operation flow (steps) of the information processing device 100 of this embodiment is similar to that of the first embodiment, and therefore the operation will be described with reference to FIG. 5.

[0118] Steps S100 to S200 are the same as those in the first embodiment, and therefore the description thereof will be omitted.

[0119] In step S300, the extraction unit 2222 of the evaluation target image setting unit 222 sets image data for a predetermined period of time including the time of trigger detection from the image data 400 temporarily stored in the memory unit 230 as a non-overwritable recorded image 1300 and extracts it from the image data 400.

[0120] Then, the extracted recorded image 1300 is assigned with information on the trigger detected by the trigger detection unit 2221 and basic information 500 corresponding to the time of the recorded image 1300 (step S400). The above steps S200 to S400 correspond to the evaluation target image setting process in the above embodiment. When step S400 is completed, the process proceeds to step S500.

[0121] In step S500, a detection object 810 is set, and it is determined whether or not the detection object 810 is present in the recorded image 1300. In step S500, first, the photographed object identification unit 3111 analyzes the recorded image 1300 using a known method such as pattern matching, and identifies the photographed object in the recorded image 1300. Next, the scene identification unit 3112 refers to the trigger information and basic information 500 added to the recorded image 1300 to identify the situation in which the recorded image 1300 was photographed, and classifies the situation into categories. These categories are the same as the first category A, second category B, third category C, and fourth category D in the first embodiment.

[0122] When the recorded image 1300 is classified into each category, the determination unit 3113 determines the detection target 810 according to the category. Next, when the detection target 810 is determined, the judgment unit 224 judges the presence or absence of the detection target 810 in the recorded image 1300, and stores the result as the detection target information 900 in the storage unit 230. The contents of the detection target information 900 are the same as those in the first embodiment. With this, step S500 ends. Then, the process proceeds to step S600.

[0123] Step S600 corresponds to the output step in the above embodiment. In step S600, the output unit 250 outputs audio based on the contents of the detection target information 900 stored in the storage unit 230. The contents of the audio output are the same as those in the first embodiment. When the audio output is completed, the process proceeds to step S700.

[0124] In step S700, the second instruction operation s2 is accepted. In step S700, if the input unit 240 accepts the second instruction operation s2 (if YES), the process proceeds to step S800. If the input unit 240 does not accept the second instruction operation s2 (if NO), the process ends step S700, and the operation of the information processing device 100 ends.

[0125] In step S800, the re-determination unit 225 detects that the second instruction operation s2 has been received by the input unit 240, determines whether or not the desired object 820 related to the second instruction operation s2 exists in the recorded image 1300, and stores the result in the storage unit 230 as the desired object information 1000. The contents of the desired object information 1000 are the same as those in the first embodiment.

[0126] When the desired target information 1000 is stored in the storage unit 230, the output unit 250 outputs voice based on the contents of the desired target information 1000 (step S900). The contents of the voice output are the same as those in the first embodiment. This completes the operation of the information processing device 100.

[0127] According to the third embodiment, the information processing device 100 can be configured by the terminal device 200 without going through the server device 300 or the like.

[0128] The present invention is not limited to the above-mentioned embodiment. In other words, a person skilled in the art can implement the present invention by modifying it in various ways according to the conventionally known knowledge without departing from the gist of the present invention. As long as the display device of the present invention is still provided even after such modification, it is of course included in the scope of the present invention.

[0129] In the above-mentioned first, second and third embodiments, the output unit 250 is configured with an audio output device such as a speaker, but the output unit 250 is not limited to a device that outputs audio. The output unit 250 may be a device that outputs text to an external device other than the terminal device 200.

[0130] For example, this external device may be a smartphone of a user who wishes to monitor his or her own vehicle while parked, and upon detection of a trigger, the contents of the detection target information 900 and the desired target information 1000 that were output as audio by the output unit 250 described above may be configured to be displayed as a notification message on the smartphone as the external device.

[0131] In addition, an external device other than the terminal device 200 may be a server device of an insurance company with which the vehicle corresponding to the terminal device 200 has a contract, or a server device of an administrator (e.g., a delivery vehicle administrator) who manages the vehicle corresponding to the terminal device 200, and these server devices may be configured as output units to display the contents of the detection target information 900 and the desired target information 1000 as notification messages.

[0132] Furthermore, when the determination unit 3113 determines the detection object 810, the detection object 810 may be further limited using position information at the time of image capture. Specifically, when it is determined that there is no pedestrian crossing or traffic light in the vicinity of the vehicle position based on the vehicle position information at the time of trigger detection and the information of the POI database 1200 recorded in the storage unit 320, the pedestrian crossing or traffic light may be excluded as the detection object 810. This makes it possible to avoid an annoying situation in which a notification that "there is no such thing" is given even though it is known in advance that there is no such thing, and to avoid unnecessary notification by the output unit 250.

[0133] On the other hand, when it is determined that a crosswalk or a traffic light exists near the vehicle position based on the vehicle position information at the time of trigger detection and the information of the POI database 1200 recorded in the storage unit 320, the crosswalk or the traffic light may be necessarily determined as the detected object 810. This allows the user to reliably recognize that "there was something that should have been captured but was not captured well."

[0134] Furthermore, when the information processing device 100 is used to monitor a parked vehicle, the following configuration may be provided. That is, the evaluation target image setting unit 222 is configured so that "the extraction unit 2222 extracts images from immediately before to immediately after the detection of the trigger as evaluation target images," and if the license plate of the other vehicle that caused the trigger cannot be detected, "an image that goes back even further than immediately before the detection of the trigger is reset as the evaluation target image." Then, the presence or absence of the license plate is determined for the evaluation target image that has been reset. This increases the possibility of acquiring information on the license plate of the other vehicle that was not captured due to the camera's angle of view, etc.

[0135] In addition, if the above-mentioned "detection of aggressive driving behavior of another vehicle" is a specified condition detected by the trigger detection unit 2221, when determining a vehicle and its license plate as the detection object 810, a specific vehicle and its license plate that is performing behavior that is thought to be aggressive driving, such as cutting in close to the side of the road or swerving, may be determined to be the detection object 810. [Explanation of symbols]

[0136] s1 1st instruction operation (instruction operation) 100 Information processing device 210 Photography Department 221 Information Gathering Department 222 Evaluation target image setting unit 311 Detection target setting section 500 Basic Information 600 Driving Information 700 Nearby Information 800 Images to be uploaded (Images to be evaluated) 810 Detection target 250 Output section

Claims

1. An information collecting unit that collects basic information including travel information of a moving object and information about the surroundings of the moving object; A photographing unit that continuously photographs; an evaluation target image setting unit that sets a predetermined amount of images from among the images captured by the image capturing unit as evaluation target images, using a trigger in either the case where the basic information satisfies a predetermined condition or the case where an instruction operation is received; a detection target setting unit that sets a detection target included in the evaluation target image from the evaluation target image based on the trigger and the basic information related to the evaluation target image; an output unit that notifies the user that the set detection object is present in the evaluation target image.

2. An information collecting unit that collects basic information including travel information of a moving object and information about the surroundings of the moving object; A photographing unit that continuously photographs; an evaluation target image setting unit that sets a predetermined amount of images from among the images captured by the image capturing unit as evaluation target images, using a trigger in either the case where the basic information satisfies a predetermined condition or the case where an instruction operation is received; a detection target setting unit that sets a detection target included in the evaluation target image from the evaluation target image based on the trigger and the basic information related to the evaluation target image; and an output unit that notifies the user that the set detection object is not present in the evaluation target image.

3. An information collecting unit that collects basic information including travel information of a moving object and information about the surroundings of the moving object; A photographing unit that continuously photographs; an evaluation target image setting unit that sets a predetermined amount of images from among the images captured by the image capturing unit as evaluation target images, using a trigger in either the case where the basic information satisfies a predetermined condition or the case where an instruction operation is received; a detection target setting unit that sets a detection target included in the evaluation target image from the evaluation target image based on the trigger and the basic information related to the evaluation target image; and an output unit that notifies the presence or absence of the detection object set in the evaluation target image.

4. 4. The information processing apparatus according to claim 1, wherein the output unit is configured as an audio output device that outputs audio.

5. the information collecting unit, the photographing unit, the evaluation target image setting unit, and the output unit are provided in a terminal device that moves together with the moving object, the detection target setting unit is provided in a server device capable of communicating with the terminal device, the evaluation target image is configured with an upload target image that can be uploaded to the server device, the upload target image is transmitted from the terminal device to the server device, 4. The information processing apparatus according to claim 1, wherein the detection target information relating to the detection target is transmitted from the server device to the terminal device.

6. the photographing unit, the evaluation target image setting unit, and the output unit are provided in a terminal device that moves together with the moving object, the information collecting unit and the detection target setting unit are provided in a server device capable of communicating with the terminal device, the evaluation target image is configured with an upload target image that can be uploaded to the server device, the upload target image is transmitted from the terminal device to the server device, 4. The information processing apparatus according to claim 1, wherein the detection target information relating to the detection target is transmitted from the server device to the terminal device.

7. the information collecting unit, the photographing unit, the evaluation target image setting unit, the detection target setting unit, and the output unit are provided in a terminal device that moves together with the moving object, 4. The information processing apparatus according to claim 1, wherein the evaluation target image is a recorded image that cannot be overwritten.

8. the instruction operation is performed by a voice of an operator who performs the instruction operation, 4. The information processing device according to claim 1, wherein the utterance does not include the detection target.

9. The travel information includes at least one of information on acceleration of the mobile object, information on the position of the mobile object, information on the traveling direction of the mobile object, and information on parking or stopping of the mobile object. The surrounding information includes at least one of time information, weather information, and traffic congestion information, The information processing device according to any one of claims 1 to 3, characterized in that the detection object includes at least one of a moving body other than the moving body, a license plate, a person, an animal other than a human, a traffic light, a road sign, a road marking, and a predetermined landmark.

10. The information processing device according to any one of claims 1 to 3, characterized in that when the detected object that is output is not the desired object desired by the operator performing the instruction operation and the operator performs a re-instruction operation, the output unit notifies the operator that the desired object was found in the evaluation target image, that the desired object was not found, or whether the desired object was found or not.

11. 1. A computer-implemented information processing method, comprising: An information collecting step of collecting basic information including travel information of the moving object and information about the surroundings of the moving object; A shooting process for continuously shooting; an evaluation target image setting step of setting a predetermined period of images from among the images taken in the photographing step as evaluation target images, when the basic information satisfies a predetermined condition or when an instruction operation is received; a detection target setting step of setting a detection target included in the evaluation target image from the evaluation target image based on the trigger and the basic information related to the evaluation target image; and an output step of notifying that the set detection object is present in the evaluation target image.

12. An information processing program for causing a computer to execute the information processing method according to claim 11.

13. 13. A computer-readable storage medium storing the information processing program according to claim 12.

Images