Information Processing Apparatus, Information Processing Method, and Program

Abstract

To reduce a processing load caused by an information processing device.SOLUTION: An information processing device according to an embodiment of the present invention includes: a stereo camera that acquires a plurality of time series images; a recording unit that records the time series images acquired by the stereo camera; a distance acquisition unit that based on a parallax concerning a corresponding point between the plurality of time series images, acquires distance information on the target; a relative acceleration acquisition unit that based on a distance of the target, acquires relative acceleration information; and a trigger determination unit that based on the distance information on the target and the relative acceleration information, provides a trigger for starting recording by the recording unit.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Conventionally, an information processing apparatus such as a drive recorder including a camera and a recording unit that records an image acquired by the camera has been known. The drive recorder is installed in a moving body such as an automobile and records peripheral images of the moving body. The images recorded in the recording unit of the drive recorder can be reproduced and displayed to assist in analyzing the cause of a traffic accident or the like.

[0003] As such an information processing apparatus, a configuration is disclosed in which relative acceleration with respect to an object is calculated from time-series images acquired by a stereo camera, and a recording start trigger for recording the time-series images is given to a recording unit according to the calculated relative acceleration (see, for example, Patent Document 1).

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the configuration of Patent Document 1, since the relative acceleration of all objects included in the time-series images is calculated, the processing load on the information processing apparatus increases.

[0005] An object of the present invention is to reduce the processing load on an information processing apparatus.

Means for Solving the Problems

[0006] An information processing apparatus according to an aspect of the present invention includes a stereo camera that acquires a plurality of time-series images each including a road surface and an object on the road surface, Provide a trigger for recording the time-series images acquired by the stereo camera a processing unit, and the processing unit acquires distance information of an object that is equal to or higher than a predetermined height threshold from the time-series images acquired by the stereo camera, Estimate the road surface shape based on a map showing the relationship between the parallax of corresponding points among the plurality of time-series images and the number of occurrences for each parallax and based on the relative acceleration information of the object, the Determine that an object at a position lower than a predetermined height threshold that is higher than the height of the road surface is not an object to record the time-series images, and that is at a position higher than the height of the road surface and based on the relative acceleration information of the object, the Based on the difference between the previously acquired distance and the newly acquired distance indicated by the distance information of the acquired time-series images, and the elapsed time from the previous time to the current time, acquire the relative velocity information of the object, and based on the difference between the previously acquired relative velocity and the newly acquired relative velocity indicated by the relative velocity information of the acquired time-series images, and the elapsed time from the previous time to the current time, acquire the relative acceleration information of the object ​Acquired Time-series image To record Give a trigger to start recording.

Advantages of the Invention

[0007] According to the present invention, the processing load by the information processing apparatus can be reduced.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments for carrying out the invention will be described with reference to the drawings. In each drawing, the same reference numerals are assigned to the same components, and redundant descriptions are omitted as appropriate.

[0010] Hereinafter, embodiments will be described taking a drive recorder as an example of an information processing apparatus.

[0011] A drive recorder is installed in a moving body such as an automobile for the purpose of recording the situation at the time of an accident or the like, and records images taken of the outside or inside of the automobile by a camera provided on the front glass or dashboard of the automobile.

[0012] <Configuration Example> (Overall Configuration Example of Drive Recorder 100) FIG. 1 is a block diagram showing an example of the overall configuration of a drive recorder 100 according to an embodiment. As shown in FIG. 1, the drive recorder 100 includes a stereo camera 1 and a processing unit 2. The drive recorder 100 is mounted on an automobile and records an image taken by the stereo camera 1 of the traveling direction side in which the automobile travels.

[0013] The stereo camera 1 includes a right camera 11 and a left camera 12. Each of the right camera 11 and the left camera 12 includes a lens that substantially forms an image of a subject on the traveling direction side of the automobile, and an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) that images the image formed by the lens.

[0014] Each of the right camera 11 and the left camera 12 outputs a plurality of time-series images showing a series obtained by photographing the temporal change of the situation on the traveling direction side of the automobile to the processing unit 2. These time-series images are still images or moving images. A moving image can also be referred to as a video.

[0015] The right camera 11 and the left camera 12 acquire a pair of left and right time-series images by photographing a subject existing on the traveling direction side of the automobile at substantially the same timing. The right camera 11 and the left camera 12 are arranged on the automobile so as to be arranged along an array direction substantially orthogonal to the direction in which the automobile travels straight. With this arrangement, a parallax along the array direction occurs between the time-series image acquired by the right camera 11 and the time-series image acquired by the left camera 12.

[0016] Based on a pair of time-series images acquired by the right camera 11 and the left camera 12, the processing unit 2 obtains information about an object existing on the vehicle traveling direction side through calculation and records the acquisition result in the recording unit.

[0017] (Hardware configuration example of the processing unit 2) FIG. 2 is a block diagram showing an example of the hardware configuration of the processing unit 2. The processing unit 2 is constructed by a computer and includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, an HDD (Hard Disk Drive) 204, an external device connection I / F (Interface) 205, and a memory card 206. These are electrically connected to each other via a system bus B.

[0018] The CPU 201 executes various arithmetic processes and control processes. The ROM 202 stores programs used for driving the CPU 201 such as an IPL (Initial Program Loader). The RAM 203 is used as a work area for the CPU 201. The HDD 204 stores various data such as programs. The HDD 204 may be an SSD (Solid State Drive).

[0019] The external device connection I / F 205 is an interface for connecting various external devices. The external device in this case is, for example, a device such as a stereo camera 1.

[0020] The memory card 206 is a thin card-shaped auxiliary storage device. The memory card 206 incorporates a flash memory or an ultra-small hard disk and has an interface for transmitting and receiving data or signals to and from other components.

[0021] (Function configuration example of the processing unit 2) FIG. 3 is a block diagram showing an example of the functional configuration of the processing unit 2. As shown in FIG. 3, the processing unit 2 includes a parameter calculation unit 21, a recording control unit 22, and a recording unit 23. The parameter calculation unit 21 includes a correction processing unit 211, a parallax calculation unit 212, a data holding unit 213, a recognition processing unit 214, a distance acquisition unit 215, and a relative acceleration acquisition unit 216. The recording control unit 22 includes a trigger determination unit 221 and a marking unit 222.

[0022] Among these, the functions of the correction processing unit 211, the parallax calculation unit 212, the recognition processing unit 214, the distance acquisition unit 215, the relative acceleration acquisition unit 216, the trigger determination unit 221, and the marking unit 222 are realized by the CPU 201 executing a predetermined program stored in the HDD 204 or the like. The function of the data holding unit 213 is realized by the HDD 204 or the RAM 203 or the like. The function of the recording unit 23 is realized by the memory card 206 or the like.

[0023] The parameter calculation unit 21 calculates parameters such as distance information and relative acceleration information of the object based on a pair of time-series images acquired by the right camera 11 and the left camera 12, and outputs the calculation result to the recording control unit 22.

[0024] The correction processing unit 211 corrects various errors such as lens distortion such as distortion in each of a pair of time-series images acquired by the right camera 11 and the left camera 12, the mounting position of the camera, or the mounting angle of the camera.

[0025] The parallax calculation unit 212 obtains, by calculation, the parallax regarding corresponding points between two time-series images based on the pair of time-series images corrected by the correction processing unit 211. The parallax calculation unit 212 designates one of the pair of time-series images as a reference image and the other as a reference image. Hereinafter, the time-series image acquired by the right camera 11 is designated as the reference image, and the time-series image acquired by the left camera 12 is designated as the reference image.

[0026] The disparity calculation unit 212 uses, as a method for deriving disparity, the SAD (Sum of Absolute Difference) method, the SSD (Sum of Squared intensity Difference) method, etc. for the pixel p(i, j) of the reference image to search for the corresponding pixel q(i + d, j) of the reference image. The d used during this search becomes the disparity.

[0027] The data storage unit 213 stores the disparity information acquired by the disparity calculation unit 212 and the attribute information of the height, depth, width, distance, and time of the object to be described later.

[0028] The recognition processing unit 214 generates a V map based on the disparity information stored in the data storage unit 213, and executes object recognition processing based on the generated V map. Here, the V map is map information generated by counting the number of appearances for each disparity with the horizontal axis being the disparity and the vertical axis being the number of appearances. The V map is an example of a map showing the relationship between the disparity and the number of appearances for each disparity.

[0029] The recognition processing unit 214 estimates the road surface shape using the generated V map, and generates a road surface height map showing the road surface height with the depth of the image as the horizontal axis and the height of the road surface as the vertical axis. Further, the recognition processing unit 214 generates a depth map showing the depth of the image with the width of the image as the horizontal axis and the depth of the image as the vertical axis.

[0030] The recognition processing unit 214 determines whether the object is an object to be marked based on the height and depth of the object located at a position higher than the height of the road surface indicated by the road surface height map. Specifically, the recognition processing unit 214 determines that the object is an object when the height of the object from the road surface is equal to or higher than the height threshold H [cm] and the depth is equal to or higher than the depth threshold L [cm]. Here, "marking" means adding a predetermined mark to the reference image. There is no particular limitation on the shape of the mark, and various shapes of marks such as points, straight lines, curves, circles, ellipses, or polygons can be applied.

[0031] The height threshold H and the depth threshold L are preset according to the minimum size of the target object to be assumed, and are stored in the ROM 202 or the like. For example, if the target object is a child, the height threshold H is preset to 50 [cm], the depth threshold L is preset to 30 [cm], and so on.

[0032] The distance acquisition unit 215 obtains distance information of all target objects by calculation based on the principle of triangulation from the parallax regarding the corresponding points between the reference image and the reference image. More specifically, the distance of the target object is the distance from the stereo camera 1 to the target object.

[0033] When the distance of the target object is equal to or less than a predetermined distance threshold D based on the acquired distance information of the target object, the distance acquisition unit 215 associates the height, depth, width, distance, and time of the target object as the attribute information of the target object and stores it in the data holding unit 213.

[0034] The relative acceleration acquisition unit 216 acquires relative acceleration information when the distance of the target object is equal to or less than a predetermined distance threshold D based on the distance of the target object. Specifically, the relative acceleration acquisition unit 216 obtains the relative velocity information of the target object based on the difference between the distance indicated by the previously acquired distance information held by the data holding unit 213 and the distance indicated by the newly acquired distance information, and the elapsed time from the previous distance information acquisition time to the current distance information acquisition time.

[0035] Then, the relative acceleration acquisition unit 216 obtains the relative acceleration information of the target object based on the difference between the relative velocity indicated by the previously acquired relative velocity information held by the data holding unit 213 and the relative velocity indicated by the newly acquired relative velocity information, and the elapsed time from the previous relative velocity information acquisition time to the current relative velocity information acquisition time.

[0036] The recording control unit 22 controls the recording operation of the time-series images to the recording unit 23 based on the calculation result by the parameter calculation unit 21. In the present embodiment, the recording unit 23 records the reference image among the pair of time-series images, but the recording unit 23 may record the reference image.

[0037] The trigger determination unit 221 gives a trigger for the recording unit 23 to start recording the reference image based on the distance information of the object acquired by the distance acquisition unit 215 and the relative acceleration information acquired by the relative acceleration acquisition unit 216. For example, when the relative acceleration indicated by the relative acceleration information acquired this time is greater than the previous relative acceleration, the trigger determination unit 221 gives a trigger to start recording.

[0038] The marking unit 222 performs a process of marking the reference image recorded by the recording unit 23 with a marking representing the object.

[0039] The recording unit 23 records the reference image marked by the marking unit 222 in response to the trigger given from the trigger determination unit 221. The reference image recorded by the recording unit 23 is used for accident cause analysis and the like.

[0040] <Operation example> (Overall operation example of the drive recorder 100) FIG. 4 is a flowchart showing an example of the overall operation of the drive recorder 100. FIG. 4 shows an operation triggered by the timing when the stereo camera 1 in the drive recorder 100 starts shooting a pair of time-series images. The stereo camera 1 may start shooting in response to the start of the engine of the automobile on which the drive recorder is mounted, or may start shooting in response to an operation input for starting shooting by the driver of the automobile.

[0041] First, in step S41, the drive recorder 100 acquires a pair of time-series images by the stereo camera 1, that is, a reference image and a reference image.

[0042] Subsequently, in step S42, the drive recorder 100 acquires the distance information of the object and the relative acceleration information of the object by the processing unit 2 based on the reference image and the reference image acquired by the stereo camera 1. Also, when a trigger is given by the trigger determination unit 221 in the processing unit 2, the drive recorder 100 records the reference image by the recording unit 23.

[0043] Subsequently, in step S43, the drive recorder 100 determines whether to end the operation. If it is determined to end (step S43, Yes), the operation ends. On the other hand, if it is determined not to end (step S43, No), the operations after step S41 are performed again.

[0044] In this way, the drive recorder 100 can record the reference image in the recording unit 23.

[0045] (Processing example by processing unit 2) FIG. 5 is a flowchart showing an example of the processing by the processing unit 2. FIG. 5 explains the processing of step S42 in FIG. 4 in more detail.

[0046] First, in step S51, the correction processing unit 211 corrects various errors such as lens distortion due to distortion, etc. in each of the reference image and the reference image acquired by the stereo camera 1, the mounting position of the camera, or the mounting angle of the camera.

[0047] Subsequently, in step S52, the disparity calculation unit 212 acquires the disparity regarding the corresponding points between the reference image corrected by the correction processing unit 211 and the reference image.

[0048] Subsequently, in step S53, the recognition processing unit 214 generates a V-map based on the disparity information stored in the data holding unit 213.

[0049] Subsequently, in step S54, the recognition processing unit 214 estimates the road surface shape using the V-map, and generates a road surface height map showing the road surface height with the depth of the image on the horizontal axis and the height of the road surface on the vertical axis. Also, the recognition processing unit 214 generates a depth map showing the depth of the image with the width of the image on the horizontal axis and the depth of the image on the vertical axis.

[0050] Subsequently, in step S55, the recognition processing unit 214 determines whether the object is a target to be marked based on the height and depth of the object located at a position higher than the height of the road surface shown in the road surface height map.

[0051] In step S55, if it is determined that the object is not present (step S55, No), the processing unit 2 ends the process. On the other hand, if it is determined that the object is present (step S55, Yes), in step S56, the distance acquisition unit 215 acquires distance information of all the objects based on the principle of triangulation from the parallax regarding the corresponding points between the reference image and the reference image.

[0052] Subsequently, in step S57, the distance acquisition unit 215 determines whether the distance of the object is equal to or less than the distance threshold D. For example, when the relative speed of the object is 60 [km / h], since it advances approximately 17 [m] per second, the distance threshold D is preset to 34 [m] in order to accumulate data for at least 2 seconds. That is, the distance threshold D is set based on the upper limit value of the assumed relative speed.

[0053] In step S57, if it is determined that the distance is not less than the distance threshold D (step S57, No), the processing unit 2 ends the process. On the other hand, if it is determined that the distance is equal to or less than the distance threshold D (step S57, Yes), in step S58, the recognition processing unit 214 causes the data holding unit 213 to store the attribute information of the object.

[0054] Subsequently, in step S59, the relative acceleration acquisition unit 216 determines whether the attribute information of the same object is already stored in the data holding unit 213. For example, the relative acceleration acquisition unit 216 makes this determination by comparing the attribute information acquired by the recognition processing unit 214 with the attribute information stored in the data holding unit 213.

[0055] In step S59, if it is determined that the information is not stored (step S59, No), the processing unit 2 ends the process. On the other hand, if it is determined that the information is stored (step S59, Yes), in step S60, the relative acceleration acquisition unit 216 determines whether the relative speed information of the object is already stored in the data holding unit 213.

[0056] In step S60, if it is determined that the information has not been saved (step S60, No), in step S61, the relative acceleration acquisition unit 216 acquires the relative velocity information of the object.

[0057] Subsequently, in step S62, the relative acceleration acquisition unit 216 causes the data holding unit 213 to save the relative velocity information of the object. Then, the processing unit 2 ends the processing.

[0058] On the other hand, in step S60, if it is determined that the relative velocity information of the object has been saved (step S60, Yes), in step S63, the relative acceleration acquisition unit 216 acquires the relative acceleration information of the object and outputs the acquired relative acceleration information to the recording control unit 22.

[0059] Subsequently, in step S64, the trigger determination unit 221 determines whether the relative acceleration information of the object has already been saved in the data holding unit 213.

[0060] In step S64, if it is determined that the information has not been saved (step S64, No), the processing unit 2 ends the processing. On the other hand, if it is determined that the information has been saved (step S64, Yes), in step S65, the trigger determination unit 221 determines whether the previously acquired relative acceleration is greater than the currently acquired relative acceleration.

[0061] In step S65, if it is determined that it is not greater (step S65, No), the processing unit 2 ends the processing. On the other hand, if it is determined that it is greater (step S65, Yes), in step S66, the marking unit 222 performs a process of applying a marking representing the object to the reference image. Then, the trigger determination unit 221 gives a trigger for starting the recording of the reference image to the recording unit 23. The recording unit 23 records the reference image with the marking in response to the trigger given from the trigger determination unit 221.

[0062] In this way, the processing unit 2 can give the recording unit 23 a trigger to start recording the reference image.

[0063] <Example of processing result by processing unit 2> With reference to FIGS. 6 to 9, an example of the processing result by the processing unit 2 will be described.

[0064] FIG. 6 is a diagram showing an example of a reference image 6 acquired by the right camera 11. The reference image 6 includes a road surface image 61 showing the road surface, a pedestrian image 62 showing a pedestrian walking on the road surface, and an oncoming vehicle image 63 showing an oncoming vehicle traveling on the road surface. The left camera 12 includes the road surface image 61, the pedestrian image 62, and the oncoming vehicle image 63, and acquires a reference image having a parallax with respect to the reference image 6.

[0065] FIG. 7 is a diagram showing an example of a V map 7 generated based on the reference image and the reference image. The horizontal axis of the V map 7 indicates parallax, and the vertical axis indicates the number of appearances. The number of appearances can also be said to be the number of pixels having the same or substantially the same parallax. The black dot plots indicate the number of appearances once. In the example shown in FIG. 7, the larger the parallax, the larger the number of appearances.

[0066] In the region corresponding to the road surface in the reference image and the reference image, the number of appearances increases according to the tendency corresponding to the road surface shape according to the parallax. Therefore, the recognition processing unit 214 can acquire the road surface shape information 71 by extracting a component indicating the tendency in the V map 7.

[0067] FIG. 8 is a diagram showing an example of a height map 8 acquired based on the V map 7 by the recognition processing unit 214. The horizontal axis indicates the depth of the reference image (the vertical direction of the image), and the vertical axis indicates the height.

[0068] FIG. 9 is a diagram showing an example of a depth map 9 acquired based on the V map 7 by the recognition processing unit 214. The horizontal axis indicates the width of the reference image (the horizontal direction of the image), and the vertical axis indicates the depth (the vertical direction of the image).

[0069] The recognition processing unit 214 determines whether an object located at a position higher than the road surface height 81 in the height map 8 is an object to be marked based on the height and depth of the object.

[0070] The recognition processing unit 214 detects objects 82 and 83 located at positions higher than the road surface height 81 in the height map 8. Further, the recognition processing unit 214 extracts an object image region 92 corresponding to the object 82 and an object image region 93 corresponding to the object 83 in the depth map 9.

[0071] When the height of the object 82 is equal to or greater than the height threshold H and the depth is equal to or greater than the depth threshold L, the recognition processing unit 214 determines that the object 82 is an object. Similarly, when the height of the object 83 is equal to or greater than the height threshold H and the depth is equal to or greater than the depth threshold L, the recognition processing unit 214 determines that the object 83 is an object. The recognition processing unit 214 labels the regions corresponding to the objects in the reference image and the reference image. Note that labeling refers to an analysis method for extracting connected regions in an image.

[0072] FIG. 10 is a diagram showing an example of a reference image 6a marked based on distance information and acceleration information obtained from the reference image and the reference image. The reference image 6a includes a road surface image 61, a pedestrian image 62, a pedestrian marking 64 indicating the pedestrian, an oncoming vehicle image 63, and an oncoming vehicle marking 65 indicating the oncoming vehicle. The recording unit 23 records the reference image 6a with such markings.

[0073] <Operation and Effect of the Drive Recorder 100> Conventionally, in a drive recorder, when vibration equal to or higher than a predetermined level is detected by an acceleration sensor provided in a moving body such as an automobile, a trigger for starting image recording is given, etc. are known.

[0074] However, when an acceleration sensor is used, the acceleration sensor may detect vibrations corresponding to road surface unevenness or passing through a railroad crossing while the vehicle is running, thereby erroneously giving a trigger for starting image recording. In such a case, not only is the image recorded uselessly, but there is also a concern that when an accident actually occurs, the free capacity of the recording unit will be exhausted and the images to be recorded cannot be recorded.

[0075] The drive recorder 100 (information processing device) according to the present embodiment includes a stereo camera 1 that acquires a plurality of time-series images, and a recording unit 23 that records a reference image (time-series image) acquired by the stereo camera 1. The drive recorder 100 also includes a distance acquisition unit 215 that acquires distance information of an object based on the parallax regarding corresponding points between the reference image and a reference image (between a plurality of time-series images), a relative acceleration acquisition unit 216 that acquires relative acceleration information based on the distance of the object, and a trigger determination unit 221 that gives a trigger for starting recording by the recording unit 23 based on the distance information and relative acceleration information of the object.

[0076] The drive recorder 100 determines whether to give a trigger by acquiring relative acceleration information from the reference image and the reference image, so it is less affected by vibrations corresponding to road surface unevenness or passing through a railroad crossing, and can suppress erroneously giving a trigger for starting recording of the reference image. In addition, the drive recorder 100 can give a trigger for starting recording with higher accuracy while further suppressing malfunction at the start of recording by using both trigger determination using the stereo camera 1 and trigger determination using an acceleration sensor.

[0077] Also, for example, when relative acceleration information is acquired for all objects included in the reference image and the reference image, the processing load on the processing unit of the drive recorder may increase.

[0078] In contrast, when the distance of the object is equal to or less than a predetermined distance threshold D based on the distance information of the object acquired by the distance acquisition unit 215, the drive recorder 100 acquires the relative acceleration information of the object. When the object exists at a position far from the automobile equipped with the drive recorder 100, the possibility of the automobile coming into contact with the object is low, and there is also little need to give a recording start trigger to the recording unit 23. Therefore, when the object exists at a position farther than the distance threshold D, the drive recorder 100 can reduce the processing load on the processing unit 2 by not performing the process of acquiring the relative acceleration information of the object. Further, the drive recorder 100 can reduce the storage capacity of the data holding unit 213 by storing the attribute information of the object in the data holding unit 213 only when the distance of the object is equal to or less than a predetermined distance threshold D. By reducing the processing load on the processing unit 2 and the storage capacity of the data holding unit 213, the cost of the drive recorder 100 can be reduced.

[0079] Also, in the present embodiment, the stereo camera 1 acquires a reference image and a reference image including a road surface and an object on the road surface. The distance acquisition unit 215 acquires the distance information of an object whose height from the road surface is equal to or higher than a height threshold H based on a V map 7 (map) showing the relationship between the parallax in the reference image and the reference image acquired by the stereo camera 1 and the number of appearances for each parallax. The relative acceleration acquisition unit 216 acquires the relative acceleration information of an object whose height from the road surface is equal to or higher than a height threshold H based on the V map 7.

[0080] When low objects such as small stones and small cargos exist on the road surface, the possibility of an accident occurring is lower compared to the case where there are pedestrians, vehicles ahead, oncoming vehicles, etc., and there is also less need to give a trigger for starting recording to the recording unit 23. Therefore, when there is an object on the road surface that is lower than the height threshold H, the driving recorder 100 can reduce the processing load on the processing unit 2 by not performing the process of acquiring the relative acceleration information of the object. Also, the driving recorder 100 can reduce the storage capacity of the data storage unit 213 by storing only the attribute information of the object whose height from the road surface is equal to or higher than the height threshold H in the data storage unit 213. By reducing the processing load on the processing unit 2 and the storage capacity of the data storage unit 213, the driving recorder 100 can be made less costly.

[0081] The driving recorder 100 also has a marking unit 222 that marks the reference image recorded in the recording unit 23 based on the height, depth, and width of the object and the distance of the object. Thereby, an observer who observes the reproduced reference image can easily visually recognize the object included in the reference image, and thus can easily analyze the cause of an accident or the like based on the reference image recorded in the recording unit 23.

[0082] <Other preferred embodiments> As described above, examples of the embodiments of the present invention have been described. However, the present invention is not limited to such specific embodiments, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims.

[0083] In the above-described embodiment, a configuration in which the stereo camera 1 includes two cameras has been exemplified. However, the stereo camera 1 may include three or more cameras. Also, although a configuration in which the stereo camera 1 includes two cameras arranged left and right in the horizontal direction has been exemplified, the stereo camera 1 may include two or more cameras arranged vertically up and down.

[0084] In the above-described embodiments, a drive recorder mounted on an automobile is exemplified as the information processing apparatus, but the present invention is not limited thereto. For example, an information processing apparatus mounted on a moving body such as an aircraft, a ship, or a railway, which records peripheral images of the moving body, can also obtain the same effects as those of the above-described embodiments.

[0085] In addition to the stereo camera 1 and the processing unit 2, the drive recorder 100 in the embodiment may further include a vehicle control device that controls the speed of the automobile or the like based on the processing result, a display device that displays the processing result, or an audio control unit that emits sound based on the processing result.

[0086] The embodiment also includes an information processing method. For example, the information processing method is an information processing method by an information processing apparatus, in which the information processing apparatus acquires a plurality of time-series images by a stereo camera, records the time-series images acquired by the stereo camera by a recording unit, acquires distance information of the object based on a parallax related to corresponding points between the plurality of time-series images by a distance acquisition unit, acquires relative acceleration information based on the distance of the object by a relative acceleration acquisition unit, and gives a trigger for starting recording by the recording unit based on the distance information of the object and the relative acceleration information by a trigger determination unit. By such an information processing method, the same effects as those of the above-described drive recorder 100 can be obtained.

[0087] The embodiment also includes a program. For example, the program causes a computer to execute a process of acquiring a plurality of time-series images by a stereo camera, recording the time-series images acquired by the stereo camera by a recording unit, acquiring distance information of the object based on a parallax related to corresponding points between the plurality of time-series images by a distance acquisition unit, acquiring relative acceleration information based on the distance of the object by a relative acceleration acquisition unit, and giving a trigger for starting recording by the recording unit based on the distance information of the object and the relative acceleration information by a trigger determination unit. By such a program, the same effects as those of the above-described drive recorder 100 can be obtained.

[0088] Each function of the embodiment described above can be realized by one or more processing circuits. Here, the "processing circuit" in this specification refers to a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, an ASIC (Application Specific Integrated Circuit) designed to execute each function described above, a DSP (digital signal processor), an FPGA (field programmable gate array), and devices such as conventional circuit modules.

Explanation of Signs

[0089] 1 Stereo camera 11 Right camera 12 Left camera 2 Processing unit 21 Parameter calculation unit 211 Correction processing unit 212 Disparity calculation unit 213 Data holding unit 214 Recognition processing unit 215 Distance acquisition unit 216 Relative acceleration acquisition unit 22 Recording control unit 221 Trigger determination unit 222 Marking unit 23 Recording unit 6 Reference image (an example of a time-series image) 61 Road surface image 62 Pedestrian image 63 Oncoming vehicle image 64 Pedestrian marking 65 Oncoming vehicle marking 7 V map (an example of a map) 71 Road surface shape information 8 Height map 81 Road surface height 82, 83 Objects 9 Depth map 92, 93 Object image regions 100 Drive recorder (an example of an information processing device)

Prior Art Documents

Patent Documents

[0090]

Patent Document 1

Claims

1. A stereo camera that acquires a plurality of time-series images each including a road surface and an object on the road surface, and a processing unit that gives a trigger for recording the time-series images acquired by the stereo camera, and has the processing unit estimates the road surface shape based on a map showing the relationship between the parallax regarding corresponding points between the plurality of time-series images acquired by the stereo camera and the number of appearances for each parallax, determines that an object lower than a predetermined height threshold located at a position higher than the height of the road surface is not a target for recording the time-series images, acquires distance information of an object that is equal to or higher than a predetermined height threshold located at a position higher than the height of the road surface, acquires relative speed information of the object based on the difference between the previously acquired distance and the newly acquired distance indicated by the distance information of the acquired time-series images and the elapsed time from the previous time to the current time, acquires relative acceleration information of the object based on the difference between the previously acquired relative speed and the newly acquired relative speed indicated by the relative speed information of the acquired time-series images and the elapsed time from the previous time to the current time, An information processing apparatus that gives a trigger for starting recording of the acquired time-series images based on the relative acceleration information of the object.

2. The information processing apparatus according to claim 1, wherein the processing unit gives a trigger for starting recording of the acquired time-series images based on the relative acceleration of an object whose distance is equal to or less than a predetermined distance threshold and whose height is equal to or higher than a predetermined height threshold located at a position higher than the height of the road surface.

3. The processing unit determines whether the distance of the object is equal to or less than a predetermined distance threshold from the distance information of the object that is equal to or higher than a predetermined height threshold located at a position higher than the height of the road surface, The information processing apparatus according to claim 1 or 2, which acquires the relative acceleration information of the object whose distance is equal to or less than a predetermined distance threshold.

4. The processing unit determines whether the previously acquired relative acceleration of the object is greater than the newly acquired relative acceleration of the object based on the relative acceleration information of the object, The information processing apparatus according to any one of claims 1 to 3, which performs marking on the time-series images to be recorded when the previously acquired relative acceleration is greater than the newly acquired relative acceleration.

5. An information processing method by an information processing apparatus, wherein the information processing apparatus A stereo camera acquires a plurality of time-series images each including a road surface and an object above the road surface. A processing unit that provides a trigger for recording the time-series images acquired by the stereo camera. Estimate the road surface shape based on a map showing the relationship between the parallax regarding corresponding points between the plurality of time-series images acquired by the stereo camera and the number of occurrences for each parallax. Determine that an object at a position lower than a predetermined height threshold that is higher than the height of the road surface is not an object for which the time-series images are to be recorded. Obtain distance information of an object that is at a position higher than the height of the road surface and is equal to or higher than a predetermined height threshold. Based on the difference between the previously acquired distance and the newly acquired distance indicated by the distance information of the acquired time-series images, and the elapsed time from the previous time to the current time, obtain relative velocity information of the object. Based on the difference between the previously acquired relative velocity and the newly acquired relative velocity indicated by the relative velocity information of the acquired time-series images, and the elapsed time from the previous time to the current time, obtain relative acceleration information of the object. An information processing method for providing a trigger for starting recording of the acquired time-series images based on the relative acceleration information of the object.

6. A stereo camera acquires a plurality of time-series images each including a road surface and an object above the road surface. A processing unit that provides a trigger for recording the time-series images acquired by the stereo camera. Estimate the road surface shape based on a map showing the relationship between the parallax regarding corresponding points between the plurality of time-series images acquired by the stereo camera and the number of occurrences for each parallax. Determine that an object at a position lower than a predetermined height threshold that is higher than the height of the road surface is not an object for which the time-series images are to be recorded. Obtain distance information of an object that is at a position higher than the height of the road surface and is equal to or higher than a predetermined height threshold. Based on the difference between the previously acquired distance and the newly acquired distance indicated by the distance information of the acquired time-series images, and the elapsed time from the previous time to the current time, obtain relative velocity information of the object. Based on the difference between the previously acquired relative velocity and the newly acquired relative velocity indicated by the relative velocity information of the acquired time-series images, and the elapsed time from the previous time to the current time, obtain relative acceleration information of the object. Based on the relative acceleration information of the object, provide a trigger for starting recording of the acquired time-series images. A program for causing an information processing apparatus to execute the processing.

Images