Information processing device, object detection device, information processing method and program

Abstract

To specify distance to an object by an image photographed from a ship.SOLUTION: An information processing device includes: an acquisition unit for acquiring a captured image captured by an imaging unit installed in a ship; a target specification unit for specifying an object in the captured image; a depression angle specifying unit for specifying a depression angle of the imaging unit; and a distance specification unit for specifying distance between the ship and the object based on the captured image and the depression angle.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an object detection apparatus, an information processing method, and a program.

Background Art

[0002] Patent Document 1 discloses an information processing apparatus that detects the presence or absence of a ship and the type of the ship from image data using an image recognition model that detects ships included in an image taken of the sea.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the information processing apparatus described in Patent Document 1 has a problem that it does not have a configuration for specifying the distance and azimuth to the detected ship, the traveling azimuth of the detected ship, and the speed.

[0005] The present disclosure has been made to solve the above problems, and an object thereof is to provide an information processing apparatus, an object detection apparatus, an information processing method, and a program capable of specifying the distance to an object from an image taken from a ship.

Means for Solving the Problems

[0006] In order to solve the above problems, an information processing apparatus according to the present disclosure includes an acquisition unit that acquires a captured image captured by a capturing unit installed on a ship, an object identification unit that identifies an object in the captured image, an elevation angle identification unit that identifies the elevation angle of the capturing unit, and a distance identification unit that identifies the distance between the ship and the object based on the captured image and the elevation angle.

[0007] The object detection device according to the present disclosure includes the information processing device and the imaging unit.

[0008] The information processing method according to the present disclosure includes steps of acquiring a captured image captured by an imaging unit installed on a ship, identifying an object in the captured image, identifying a depression angle of the imaging unit, and identifying a distance between the ship and the object based on the captured image and the depression angle.

[0009] The program according to the present disclosure causes a computer to execute steps of acquiring a captured image captured by an imaging unit installed on a ship, identifying an object in the captured image, identifying a depression angle of the imaging unit, and identifying a distance between the ship and the object based on the captured image and the depression angle.

Advantages of the Invention

[0010] According to the information processing device, object detection device, information processing method, and program of the present disclosure, the distance to an object can be identified from an image captured from a ship.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

MODE FOR CARRYING OUT THE INVENTION

[0012] (Configuration of Object Detection Device) Hereinafter, an information processing apparatus, an object detection apparatus, an information processing method, and a program according to embodiments of the present disclosure will be described with reference to FIGS. 1 to 14. FIG. 1 is a block diagram showing a schematic configuration of an object detection apparatus according to an embodiment of the present disclosure. FIG. 2 is a plan view schematically showing an image range of a photographing unit according to an embodiment of the present disclosure. FIG. 3 is a schematic diagram for explaining a configuration example of a shaking sensor according to an embodiment of the present disclosure. FIGS. 4 and 5 are flowcharts showing operation examples of an image analysis server according to an embodiment of the present disclosure. FIGS. 6 to 13 are schematic diagrams for explaining operation examples of an image analysis server according to an embodiment of the present disclosure. FIG. 14 is a schematic block diagram showing the configuration of a computer according to an embodiment of the present disclosure. In each figure, the same or corresponding components are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

[0013] As shown in FIG. 1, the object detection device 10 includes an image analysis server 1, a photographing unit 2, a shaking sensor 3, a PC (Personal Computer) 4 for displaying image analysis results, a hub 5, and a plurality of communication cables 51. The image analysis server 1 is an example of a configuration of the information processing device of the present disclosure. The image analysis server 1 is communicatively connected to the photographing unit 2, the shaking sensor 3, and the PC 4 for displaying image analysis results via the hub 5 and a plurality of communication cables 51. Further, the image analysis server 1 is communicatively connected to the automatic ship control system 6 via a communication cable 61. The object detection device 10 and the automatic ship control system 6 are mounted on the ship 100. The object detection device 10 identifies objects such as other ships, buoys, fishing gear, and other objects located within a predetermined detection range, calculates the distance, position, etc. from the ship 100 to the object, and outputs the calculation results and the images of the objects taken to the PC 4 for displaying image analysis results, the automatic ship control system 6 of the ship 100, etc.

[0014] The photographing unit 2 includes a plurality of cameras 21 and a plurality of cameras 22. The cameras 21 and 22 have different angles of view. The cameras 21 and 22 are, for example, infrared cameras. The plurality of cameras 21 and the plurality of cameras 22 are communicatively connected to the image analysis server 1 via the communication cable 51 and the hub 5, and the captured images taken by the plurality of cameras 21 and the plurality of cameras 22 are transmitted to the image analysis server 1 at any time. The plurality of cameras 21 and the plurality of cameras 22 are installed at locations where it is possible to overlook the bridge of the ship 100 and the surrounding area outside. The installation directions of the cameras 21 and 22 with respect to the hull are set in consideration of the detection range of the object detection device 10 and the angles of view of the cameras 21 and 22. When installing a plurality of cameras 21 and cameras 22, the photographing directions are set so that the image ranges of the respective cameras 21 and 22 are continuous as much as possible. A camera with specifications suitable for the detection distance to the object of the object detection device 10 and the size of the object is selected. The number of cameras and the angle of view corresponding to the detection range of the object detection device 10 are set.

[0015] FIG. 2 schematically shows the image range of the photographing unit 2. In the example shown in FIG. 2, the photographing unit 2 is configured as a camera system that combines a plurality of cameras with different angle of views, and includes six cameras 21 and two cameras 22. The six cameras 21 are long-distance cameras with a horizontal angle of view (field of view) of 8.6 degrees, and the image ranges (photographing ranges) of the respective cameras 21 are image ranges 21A, 21B, 21C, 21D, 21E, and 21F. The two cameras 22 are short-distance cameras with a horizontal angle of view of 69.0 degrees, and the image ranges of the respective cameras 22 are image ranges 22A and 21B.

[0016] In the case of a ship, the required detection distance from an object for collision avoidance ranges widely from several kilometers to several tens of meters. Also, a horizontal field of view of several tens of degrees in the forward direction is required. Generally, due to the physical limitations of industrial cameras, a long-distance camera has a narrow field of view of several degrees of the angle of view, and a short-distance camera has a relatively wide angle of view. In the present embodiment, in order to cover from a long distance to a short distance, which is necessary for peripheral monitoring of the ship 100, a plurality of long-distance cameras and short-distance cameras are installed to secure a field of view range.

[0017] The motion sensor 3 detects the motion (heave, roll, pitch, etc.) of the ship 100 (hereinafter also referred to as hull motion information), and transmits the detection value to the image analysis server 1. The motion sensor 3 includes four GNSS (Global Navigation Satellite System) receivers 31, antennas 31a, 31b, 31c, and 31d of each GNSS receiver 31, an inertial measurement device 32, and a controller (not shown).

[0018] As shown in FIG. 3, among the four GNSS receivers 31, the antennas 31b, 31c, and 31d of three of the GNSS receivers 31 are installed such that, for example, with the antenna 31b as a reference, the antenna 31c is installed in the pitch axis direction of the hull coordinate system, and the antenna 31d is installed in the roll axis direction. The three GNSS receivers 31 connected to the antennas 31b, 31c, and 31d use the GNSS receiver 31 connected to the antenna 31b as a Moving-RTK (Real Time Kinematic) Base, and the two GNSS receivers 31 connected to the antennas 31c and 31d as Moving-RTK Rovers to measure the three-dimensional relative position vector Rr and the three-dimensional relative position vector Rp. The motion sensor 3 calculates the roll angle, pitch angle, and yaw angle of the ship 100 based on the three-dimensional relative position vector Rr and the three-dimensional relative position vector Rp. In addition, the GNSS receiver 31 connected to the antenna 31a measures the three-dimensional absolute position coordinates Pw in the World coordinate system. The motion sensor 3 calculates heave (vertical oscillation) from the roll angle, pitch angle, and the three-dimensional absolute position coordinates Pw. Note that the inertial measurement device 32 is, for example, a MEMS (Micro Electro Mechanical Systems) gyro-accelerometer and is used to acquire short-time attitude change information.

[0019] In the calculation of the distance to an object to be described later, the installation height of the cameras 21 and 22 above the water surface is one of the parameters. The cameras 21 and 22 are fixed to the upper part of the ship, but the draft of the ship 100 changes depending on the loading state of the ship 100 and the like, and accordingly, the height of the cameras 21 and 22 from the water surface changes. Therefore, the heights of the cameras 21 and 22, which are calculation parameters, always need to be corrected, and the real-time correction is performed based on the heave measured value calculated by the motion sensor 3. In addition, the roll and pitch calculated by the motion sensor 3 are used in the calculation of the virtual horizon position when the horizon detection described later is not possible, to ensure the calculation accuracy of the distance to the object, the azimuth of the object, the speed of the object, and the traveling direction. Note that if the horizon can be detected, highly accurate distance calculation based on the horizon position can be performed only by image analysis, and thus correction related to the ship's motion is not particularly necessary.

[0020] On the one hand, the image analysis server 1 is configured as a functional configuration composed of a combination of hardware such as a processor and a memory, and software such as a program executed by the processor. It includes an acquisition unit 11, an object identification unit 12, a depression angle identification unit 13, and a distance identification unit 14.

[0021] The acquisition unit 11 acquires the captured image captured by the imaging unit 2 installed on the ship 100. In addition, the acquisition unit 11 acquires the hull motion information output by the motion sensor 3.

[0022] The object identification unit 12 identifies an object in the captured image. The object identification unit 12 performs image recognition using, for example, a trained machine learning model to identify the position and area (frame) of the object and the horizon. The trained machine learning model is, for example, a model learned by supervised learning using a training dataset in which a plurality of images including objects such as ships, buoys, fishing gear, and other objects are labeled with identification information of the objects. It is a model that inputs image information and outputs information indicating the presence or absence of an object, the position and area of the object in the captured image.

[0023] The depression angle identification unit 13 identifies the depression angle of the imaging unit 2. At this time, the depression angle identification unit 13 identifies the depression angle based on the positional relationship between the horizon and the object in the captured image. In addition, when the horizon cannot be detected in the captured image, the depression angle identification unit 13 sets a virtual horizon and identifies the depression angle based on the positional relationship between the virtual horizon and the object. For example, when the imaging direction is facing the land side, the captured image may not include a horizon. In such a case, the depression angle identification unit 13 virtually sets a horizon. Note that the depression angle identification unit 13 identifies the depression angle based on the detection value of the motion sensor 3 that detects the motion of the ship 100. In addition, the depression angle identification unit 13 identifies the depression angle based on the installation height of the imaging unit 2.

[0024] The distance identification unit 14 identifies the distance between the ship 100 and the object based on the captured image and the depression angle. In addition, the distance identification unit 14 integrates objects across the plurality of cameras 21 and 22 based on the position information including the identified distance.

[0025] (Operation of Object Detection Device) Next, with reference to the flowcharts and the like shown in FIGS. 4 and 5, an operation example of the object detection device 10 shown in FIG. 1 will be described. Note that the flow shown in FIG. 4 and the flow shown in FIG. 5 are connected to each other by connectors A and B. The processes shown in FIGS. 4 and 5 are repeatedly executed at a predetermined cycle. Also, it is assumed that the installation height of each of the cameras 21 and 22 is 25 m from the water surface (sea surface). Further, it is assumed that the example of the captured image used in the description is a captured image by the camera 21 having 640 pixels horizontally (angle of view: 8.6 degrees) and 512 pixels vertically (angle of view: 6.6 degrees).

[0026] When the processes shown in FIGS. 4 and 5 are started, in the object detection device 10, first, the acquisition unit 11 of the image analysis server 1 acquires a plurality of captured images (captured by the plurality of cameras 21 and 22) captured by the imaging unit 2 (step S101). Next, the acquisition unit 11 acquires the hull motion information output by the motion sensor 3 (step S102).

[0027] Next, the object identification unit 12 identifies the presence, position, and region of the object in one of the plurality of captured images (step S103). For example, the object identification unit 12 inputs a captured image IMG1 as shown in FIG. 6 to a learned machine learning model, and identifies information representing the position and region of the object (ship (SHIP), buoy (BUOY), etc.) included in the captured image IMG1.

[0028] Next, the object identification unit 12 determines whether the object has been identified (step S104). If the object has not been identified (step S104: No), the object identification unit 12 determines whether all of the plurality of captured images have been processed (step S113). If all of the plurality of captured images have not been processed (step S113: No), the object identification unit 12 identifies the presence, position, and region of the object in the next captured image among the plurality of captured images (step S103).

[0029] On the other hand, when the object is specified (Step S104: Yes), the object specifying unit 12 detects a horizontal line in the captured image (Step S105). For example, the object specifying unit 12 performs image recognition on the captured image IMG1 as shown in FIG. 6, and specifies information representing the position and area of the horizontal line (HORIZON) included in the captured image IMG1.

[0030] Next, the depression angle specifying unit 13 determines whether a horizontal line has been detected (Step S106). When a horizontal line is detected in the captured image (Step S106: Yes), the depression angle specifying unit 13 calculates a horizontal line shooting depression angle θh from the radius of the earth and the heave detected by the motion sensor 3 (Step S107).

[0031] As shown in FIG. 7, the heights of the cameras 21 and 22 with respect to the sea surface are values obtained by adding or subtracting the absolute value of the heave from the installation height of 25 m. Also, the horizontal line shooting depression angle θh is calculated by the following formula.

[0032] θh = cos -1 (radius of the earth / (radius of the earth + camera installation height ± heave))

[0033] Next, as shown in FIG. 8, the depression angle specifying unit 13 measures the vertical distance dH between the horizontal line and the object OJ1 on the captured image IMG2, and converts it into an angle θd formed by the horizontal line shooting direction and the object shooting direction (Step S108).

[0034] Next, as shown in FIG. 9, the depression angle specifying unit 13 adds the horizontal line shooting depression angle θh and the angle θd formed by the horizontal line shooting direction and the object shooting direction, and calculates the object shooting depression angle θt (Step S109).

[0035] Next, as shown in FIG. 9, the distance specifying unit 14 calculates the distance to the object OJ1 based on the object shooting depression angle θt, the height of the shooting unit, and the heave by the following formula (Step S110).

[0036] Distance to the object = (camera installation height ± heave) / tan(object shooting depression angle θt)

[0037] On the other hand, when no horizontal line is detected in the captured image (step S106: No), as shown in FIG. 10, the depression angle specifying unit 13 calculates a virtual horizontal line shooting depression angle θh' from the earth radius and the heave and pitch detected by the motion sensor 3 by the following formula (step S121).

[0038] θ’=cos -1 (earth radius / (earth radius + camera installation height ± heave))

[0039] Virtual horizontal line shooting depression angle θh' = θ’ ± pitch

[0040] Next, as shown in FIG. 11, the depression angle specifying unit 13 adds a virtual horizontal line (dashed line) at a position dHr' corresponding to the virtual horizontal line shooting depression angle θh' on the captured image IMG3, and rotates by the roll amount detected by the motion sensor 3 around the reference point in the captured image (the rotated virtual horizontal line (dotted line)) (step S122).

[0041] Here, even when the horizontal line cannot be detected, it is estimated as the position when it is supposed to be reflected, and is used as the reference line in distance calculation. When multiple cameras are installed, based on the virtual horizontal line of one camera, the virtual horizontal line positions in other cameras are estimated from the shooting angles and installation relationships of each camera. Also, it is desirable to perform a calibration operation in advance so that the virtual horizontal line position matches as much as possible the horizontal line position, virtual horizontal line position, and behavior reflected in the captured image in a situation where the horizontal line can be detected.

[0042] Next, as shown in FIG. 11, the depression angle specifying unit 13 measures the vertical distance dH' between the virtual horizontal line and the object OJ1 on the captured image IMG3, and converts it into an angle θd' formed by the virtual horizontal line shooting direction and the object shooting direction (step S123).

[0043] Next, as shown in FIG. 12, the depression angle specifying unit 13 adds the virtual horizontal line shooting depression angle θh' and the angle θd' formed by the virtual horizontal line shooting direction and the object shooting direction to calculate the object shooting depression angle θt' (step S124).

[0044] Next, as shown in FIG. 12, the distance specifying unit 14 calculates the distance to the target object OJ1 based on the target object shooting depression angle θt, the shooting unit height, and the heave by the following formula (step S125).

[0045] Distance to the target object = (Camera installation height ± Heave) / tan (Target object shooting depression angle θt’)

[0046] After calculating the distance to the target object in step S110 or step S125, the distance specifying unit 14 calculates the azimuth of the target object (step S111). The azimuth of the target object is calculated from the position of the target object in the captured image based on the azimuth of the projected camera with respect to the hull, the angle of view of the camera, and the hull sway information.

[0047] Next, the distance specifying unit 14 calculates the traveling direction and speed of the target object (step S112). The traveling direction and speed of the target object are continuously calculated based on the moving distance of the target object in the captured images at regular time intervals and the distance between the target object and the own ship (for example, the distance to the target object calculated in step S110 or step S125). Also, for the purpose of stabilizing the wobbling at that time, the calculated values at each time are appropriately averaged by arithmetic processing.

[0048] Next, the target specifying unit 12 determines whether all of the plurality of captured images have been processed (step S113). On the other hand, if all of the plurality of captured images have not been processed (step S113: No), the target specifying unit 12 specifies the presence, position, and area of the target object in the next captured image among the plurality of captured images (step S103). On the other hand, if all of the plurality of captured images have been processed (step S113: Yes), the distance specifying unit 14 performs integrated processing of the target object detection results (step S114).

[0049] In step S114, when the same object appears in a plurality of cameras, the distance specifying unit 14 integrates the detection results as a single object as much as possible based on the fact that the position information of the object is close to a certain extent and the discrimination result by image recognition. For example, as shown in FIG. 13, in adjacent captured images IMG11 and IMG12, when the positioning information (azimuth and distance) of the detection frame F1 for the object OJ11 and the detection frame F2 for the object OJ12 are within a certain range, the distance specifying unit 14 treats them as one target (object OJ2) (with the same ID (identification code)). Further, during panning, the distance specifying unit 14 calculates the overall center of gravity from the head and the tail in the moving direction.

[0050] Next, the distance specifying unit 14 outputs information of the object (presence or absence of the object, ID of the object, distance, azimuth, speed, traveling direction, type (ship, buoy, etc.), etc.) and the captured image to the PC 4 for displaying the image analysis result, the automatic ship control system 6, etc., and ends the processes shown in FIGS. 4 and 5.

[0051] (Function and effect) As described above, according to the present embodiment, the distance to the object can be specified from the image captured from the ship.

[0052] As described above, the embodiments of the present invention have been described with reference to the drawings. However, the specific configuration is not limited to the above embodiments, and design changes and the like within the scope not departing from the gist of the present invention are also included.

[0053] 〈Computer configuration〉 FIG. 14 is a schematic block diagram showing the configuration of a computer according to an embodiment of the present disclosure. The computer 90 includes a processor 91, a main memory 92, a storage 93, and an interface 94. The above-described image analysis server (information processing device) 1 is implemented in a computer 90. Then, the operations of the above-described respective processing units are stored in a storage 93 in the form of a program. The processor 91 reads the program from the storage 93, expands it in the main memory 92, and executes the above processing according to the program. Further, the processor 91 secures a storage area corresponding to each of the above-described storage units in the main memory 92 according to the program.

[0054] The program may be for realizing a part of the functions to be exhibited by the computer 90. For example, the program may exhibit functions by combination with other programs already stored in the storage, or by combination with other programs implemented in other devices. In other embodiments, the computer may include, in addition to or instead of the above configuration, a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device). Examples of the PLD include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), FPGA (Field Programmable Gate Array), and the like. In this case, part or all of the functions realized by the processor may be realized by the integrated circuit.

[0055] Examples of the storage 93 include HDD (Hard Disk Drive), SSD (Solid State Drive), magnetic disk, magneto-optical disk, CD-ROM (Compact Disc Read Only Memory), DVD-ROM (Digital Versatile Disc Read Only Memory), semiconductor memory, and the like. The storage 93 may be an internal medium directly connected to the bus of the computer 90, or may be an external medium connected to the computer 90 via the interface 94 or a communication line. Further, when this program is distributed to the computer 90 via a communication line, the receiving computer 90 may expand the program in the main memory 92 and execute the above processing. In at least one embodiment, the storage 93 is a non-transitory tangible storage medium.

[0056] <Appendix> The image analysis server 1 described in the embodiments of the present disclosure is understood as follows, for example.

[0057] (1) The image analysis server 1 (information processing apparatus) according to the first aspect includes an acquisition unit 11 that acquires a captured image captured by the imaging unit 2 installed on the ship 100, an object identification unit 12 that identifies an object in the captured image, an elevation angle identification unit 13 that identifies the elevation angle of the imaging unit 2, and a distance identification unit 14 that identifies the distance between the ship 100 and the object based on the captured image and the elevation angle. According to this aspect, the image analysis server 1 (information processing apparatus) can identify the distance to the object from the image captured from the ship 100.

[0058] (2) The image analysis server 1 (information processing apparatus) according to the second aspect is the image analysis server 1 (information processing apparatus) of (1), and the elevation angle identification unit 13 identifies the elevation angle based on the positional relationship between the horizontal line and the object in the captured image. According to this aspect, the image analysis server 1 (information processing apparatus) can accurately identify the distance to the object from the image captured from the ship 100.

[0059] (3) The image analysis server 1 of the third aspect is the image analysis server 1 (information processing device) of (1), and when the depression angle specifying unit 13 cannot detect a horizontal line in the captured image, a virtual horizontal line is set, and the depression angle is specified based on the positional relationship between the virtual horizontal line and the object. According to this aspect, the image analysis server 1 (information processing device) can accurately specify the distance to the object from the image captured from the ship 100 even when the horizontal line is not included in the captured image.

[0060] (4) The image analysis server 1 (information processing device) of the fourth aspect is the image analysis server 1 (information processing device) of any one of (1) to (3), and the depression angle specifying unit 13 specifies the depression angle based on the detection value of the shaking sensor 3 that detects the shaking of the ship 100. According to this aspect, the image analysis server 1 (information processing device) can accurately specify the distance to the object from the image captured from the ship 100.

[0061] (5) The image analysis server 1 (information processing device) of the fifth aspect is the image analysis server 1 (information processing device) of any one of (1) to (4), and the depression angle specifying unit 13 specifies the depression angle based on the installation height of the imaging unit 2.

[0062] (6) The image analysis server 1 (information processing device) of the sixth aspect is the image analysis server 1 (information processing device) of any one of (1) to (5), and the imaging unit 2 includes a plurality of cameras 21 and 22 with different viewing angles.

[0063] (7) The image analysis server 1 (information processing device) of the seventh aspect is the image analysis server 1 (information processing device) of any one of (1) to (5), the imaging unit 2 includes a plurality of cameras 21 and 22, and the distance specifying unit 14 integrates based on the position information including the specified distance of the object across the plurality of cameras 21 and 22.

[0064] (8) The object detection device 10 according to the eighth aspect includes any one of the image analysis servers 1 (information processing devices) from (1) to (7) and the imaging unit 2. According to this aspect, the object detection device 10 can specify the distance to the object based on the image taken from the ship 100.

[0065] (9) The information processing method according to the ninth aspect includes the steps of acquiring a captured image captured by the imaging unit 2 installed on the ship 100, specifying an object in the captured image, specifying the depression angle of the imaging unit 2, and based on the captured image and the depression angle, specifying the distance between the ship 100 and the object. According to this aspect, the information processing method can specify the distance to the object based on the image taken from the ship 100.

[0066] (10) The program according to the tenth aspect causes a computer to execute the steps of acquiring a captured image captured by the imaging unit 2 installed on the ship 100, specifying an object in the captured image, specifying the depression angle of the imaging unit 2, and based on the captured image and the depression angle, specifying the distance between the ship 100 and the object. According to this aspect, the program can specify the distance to the object based on the image taken from the ship 100.

Explanation of Reference Numerals

[0067] 1... Image analysis server (information processing device) 2... Imaging unit 3... Vibration sensor 4... PC for displaying image analysis results 5... Hub 6... Automatic steering system 10... Object detection device 11... Acquisition unit 12... Object specification unit 13... Depression angle specification unit 14... Distance specification unit 21... Camera 21A... Image range 21B... Image range 21C... Image range 21D…Image range 21E…Image range 21F…Image range 22…Camera 22A…Image range 31…GNSS receiver 31a…Antenna 31b…Antenna 31c…Antenna 31d…Antenna 32…Inertial measurement unit 51…Communication cable 61…Communication cable 90…Computer 91…Processor 92…Main memory 93…Storage 94…Interface 100…Ship F1…Detection frame F2…Detection frame IMG1…Captured image IMG2…Captured image IMG3…Captured image IMG11…Captured image IMG12…Captured image OJ1…Object OJ2…Object OJ11…Object OJ12…Object Pw…Three-dimensional absolute position coordinates Rp…Three-dimensional relative position vector Rr…Three-dimensional relative position vector

Claims

1. an acquisition unit that acquires images captured by an image capture unit installed on the ship; an object identification unit that identifies an object in the captured image; a depression angle determination unit that determines an object photographing depression angle that is the depression angle of the photographing unit with respect to the object based on a horizontal photographing depression angle that is the depression angle of the photographing unit with respect to the horizon or a virtual horizontal photographing depression angle that is the depression angle of the photographing unit with respect to a virtual horizontal line and the object in the photographed image; A distance determination unit that determines a distance between the ship and the object based on the object photographing depression angle; An information processing device comprising:

2. The depression angle specification unit specifies a photographing depression angle of the object based on a positional relationship between the horizon and the object in the photographed image. The information processing device according to claim 1 .

3. When the depression angle determination unit cannot detect a horizon in the photographed image, the depression angle determination unit sets a virtual horizon and determines a photographing depression angle of the object based on a positional relationship between the virtual horizon and the object. The information processing device according to claim 1 .

4. The depression angle determination unit determines the horizontal line shooting depression angle or the virtual horizontal line shooting depression angle based on a detection value of a motion sensor that detects motion of the ship. The information processing device according to claim 1 .

5. The depression angle determination unit determines the horizontal line shooting depression angle or the virtual horizontal line shooting depression angle based on the installation height of the shooting unit. The information processing device according to claim 1 .

6. The photographing unit includes a plurality of cameras with different angles of view. The information processing device according to claim 1 .

7. The photographing unit includes a plurality of cameras, The distance determination unit integrates the objects captured by the plurality of cameras based on position information including the determined distances. The information processing device according to claim 1 .

8. An information processing device according to any one of claims 1 to 7; The imaging unit; An object detection device comprising:

9. acquiring an image captured by an image capturing unit installed on a ship; identifying an object in the captured image; A step of specifying an object photographing depression angle, which is the depression angle of the photographing unit with respect to the object, based on a horizontal photographing depression angle, which is the depression angle of the photographing unit with respect to the horizon, or a virtual horizontal photographing depression angle, which is the depression angle of the photographing unit with respect to a virtual horizontal line, and the object in the photographed image; determining a distance between the vessel and the object based on the object photographing depression angle; An information processing method comprising:

10. acquiring an image captured by an image capturing unit installed on a ship; identifying an object in the captured image; A step of specifying an object photographing depression angle, which is the depression angle of the photographing unit with respect to the object, based on a horizontal photographing depression angle, which is the depression angle of the photographing unit with respect to the horizon, or a virtual horizontal photographing depression angle, which is the depression angle of the photographing unit with respect to a virtual horizontal line, and the object in the photographed image; determining a distance between the vessel and the object based on the object photographing depression angle; A program that causes a computer to execute the following.

Images