Apparatus, method and program

Abstract

To accurately determine whether an object is moving.SOLUTION: A system 3 includes: a position acquisition unit 33 that acquires positional information representing a position of an own ship; an azimuth acquisition unit 34 that acquires azimuth information representing an azimuth of an object relative to the own ship; and a determination unit 35 that determines, based on the positional information and azimuth information obtained at least three time points, whether the object is moving. The azimuth acquisition unit 34 includes: an imaging azimuth acquisition block 341 that acquires an imaging azimuth of when the object is imaged from the own ship; and a calculation block 342 that calculates, based on the imaging azimuth, the azimuth of the object relative to the own ship.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an apparatus, a method, and a program. [Background technology]

[0002] Patent Documents 1 to 3 state, "...the information acquisition module...can detect obstacle information of at least one obstacle. The information acquisition module may be an image collection device, an attitude sensor, a radar sensor, a lidar sensor, a sonar sensor, a radio positioning device, or an Automatic Identification System (AIS). ...The obstacle information includes the obstacle's movement speed, position, movement direction, size of the obstacle, and distance from the ship. ...The obstacle's movement method (dynamic or static) can be determined from the obstacle information detected by the information acquisition module." (Patent Document 1, paragraphs 0027 and 0029) [Prior art document] [Patent documents] [Patent Document 1] JP 2021-149940 A [Patent Document 2] JP 2020-164110 A [Patent Document 3] JP 2012-155359 A Summary of the Invention

[0003] In a first aspect of the present invention, an apparatus is provided that includes a position acquisition unit that acquires position information indicating the position of the ship, an orientation acquisition unit that acquires orientation information indicating the orientation of an object relative to the ship, and a determination unit that determines whether the object is moving based on the position information and the orientation information at at least three points in time.

[0004] In the above device, the orientation acquisition unit may include an imaging orientation acquisition unit that acquires the imaging orientation when the object is imaged from the ship, and a calculation unit that calculates the orientation of the object relative to the ship based on the imaging orientation.

[0005] The above device may further include an image acquisition unit that acquires images taken from the ship itself, and an identification unit that identifies the object within the image, and the calculation unit may calculate the orientation of the object relative to the ship itself based further on the position of the object within the image.

[0006] In the above device, the calculation unit may calculate the orientation of the object relative to the ship based further on the angle of view of the image.

[0007] Any of the above devices may further include an image acquisition unit that acquires an image taken from the ship itself, an identification unit that identifies the object within the image, and a display control unit that displays the image of the object and changes the display mode of the object within the image depending on the determination result by the determination unit.

[0008] In any of the above devices, the position acquisition unit may acquire the position information from a GPS signal.

[0009] In any of the above devices, the judgment unit may include an intersection position identification unit that identifies the position of the intersection of straight lines extending from the position of the ship toward the direction of the object at each of the at least three points in time, and a determination unit that determines whether the object is moving depending on whether the positions of the at least two intersections identified by the intersection position identification unit are the same.

[0010] In the above device, the determination unit may determine that the object is not moving if, among the at least three points in time, a first intersection between a straight line extending from the position of the ship at a first point in time toward the direction of the object and a straight line extending from the position of the ship at a second point in time toward the direction of the object are the same as a second intersection between a straight line extending from the position of the ship at the second point in time toward the direction of the object and a straight line extending from the position of the ship at a third point in time toward the direction of the object.

[0011] Any of the above devices may further include an estimation unit that, in response to determining that the object is moving, estimates at least one of the object's movement path, the object's movement speed, and the object's position at a time different from the at least three times based on a straight line extending from the ship's position at each of the at least three times toward the object's direction and the time of each of the at least three times.

[0012] In a second aspect of the present invention, a method is provided comprising a position acquisition step of acquiring position information indicating the position of the ship, an orientation acquisition step of acquiring orientation information indicating the orientation of an object relative to the ship, and a determination step of determining whether the object is moving based on the position information and the orientation information at at least three points in time.

[0013] In a third aspect of the present invention, a program is provided that causes a computer to function as a position acquisition unit that acquires position information indicating the position of the ship, an orientation acquisition unit that acquires orientation information indicating the orientation of an object relative to the ship, and a determination unit that determines whether the object is moving based on the position information and the orientation information at at least three points in time.

[0014] The above summary of the invention does not list all of the necessary features of the present invention, and subcombinations of these features may also constitute inventions. [Brief explanation of the drawings]

[0015] [Figure 1] 1 shows a system 1 according to an embodiment. [Figure 2] Indicates the relative positions of the ship and the target when the target is moving. [Figure 3] This shows the relative positions of the ship and the target when the target is stationary. [Figure 4] The operation of device 3 is shown. [Figure 5] 22 illustrates an example computer 2200 in which aspects of the present invention may be embodied, in whole or in part. DETAILED DESCRIPTION OF THE INVENTION

[0016] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0017] <1. System 1> 1 shows a system 1 according to this embodiment. The system 1 assists in maneuvering the ship itself, and includes a camera 2 and a device 3.

[0018] <1.1.Camera 2> Camera 2 captures images from its own ship. Camera 2 may be capable of measuring the image capture position and image capture direction. The image capture position may be the position of camera 2 when the image was captured, and may be indicated by longitude and latitude, for example. The image capture direction may be the direction of the image capture relative to a reference direction, and may be indicated by an angle, for example. In this embodiment, as an example, the image capture direction may be the direction relative to the magnetic meridian (also called magnetic direction). Camera 2 may add EXIF ​​(Exchangeable Image File Format) data to the captured image indicating the date and time of capture, the image capture position, and the image capture direction. The EXIF ​​data may further indicate the focal length.

[0019] The shooting direction of camera 2 may be fixed or variable relative to the hull. The focal length of camera 2, in other words, the angle of view, may be fixed or variable. Camera 2 may capture still images in response to operation by the crew, or may constantly capture moving images. Camera 2 may be a visible light camera, or may be an infrared or ultraviolet (X-ray, for example) camera.

[0020] The images captured by camera 2 may include as subjects objects (also referred to as targets) whose movement is unknown. Whether an object is moving may refer to whether it is moving relative to a fixed position on the Earth, and may be synonymous with whether it has ground speed. The targets may be ships, buoys, land, reefs, drifting ice, people, etc.

[0021] Camera 2 may provide captured images to device 3. If camera 2 captures still images, the provided images may be still images, and if camera 2 captures moving images, the provided images may be frames within the moving images.

[0022] <1.2.Device 3> The device 3 includes an image acquisition unit 31, an identification unit 32, a position acquisition unit 33, a direction acquisition unit 34, a determination unit 35, an estimation unit 36, a display control unit 37, and a display unit 38. Each unit of the device 3 may be installed on the ship.

[0023] <1.2.1. Image Acquisition Unit 31> The image acquisition unit 31 acquires images captured by the ship itself. The image acquisition unit 31 may acquire images from the camera 2. EXIF ​​data may be added to the acquired images.

[0024] The image acquisition unit 31 may acquire only images that include the target object as a subject. For example, when an image is captured by the camera 2 in response to an operation by a crew member who has visually recognized the target object, the image acquisition unit 31 may acquire only images that include the target object as a subject. Alternatively, the image acquisition unit 31 may acquire images regardless of whether the image includes the target object as a subject. For example, when images (for example, frames of a moving image) are captured by the camera 2 regardless of the presence of the target object, the image acquisition unit 31 may acquire each captured image.

[0025] The image acquisition section 31 may supply the acquired image to the identification section 32. The image acquisition section 31 may supply the acquired image to the display control section 37, and cause the display section 38 to display the image.

[0026] <1.2.2. Specific part 32> The identification unit 32 identifies an object in the image acquired by the image acquisition unit 31. The identification unit 32 may identify any one of the objects detected by image recognition other than the sea surface as the object. The identification unit 32 may identify an object closer to the center of the image as the object, or may identify a larger object as the object. The identification unit 32 may identify any object as the object in response to a sailor specifying the object in the image displayed on the display unit 38. The identification unit 32 does not need to identify the object if there are no objects other than the sea surface in the image.

[0027] In response to identifying an object in an image, the identification unit 32 may supply position information of the object in the image to the display control unit 37. In response to identifying an object in an image, the identification unit 32 may supply the image and position information of the object in the image to the position acquisition unit 33 and the orientation acquisition unit 34.

[0028] <1.2.3.Position acquisition unit 33> The position acquisition unit 33 acquires position information indicating the position of the ship. In this embodiment, the position of the ship may be indicated by longitude and latitude, for example. In this embodiment, the position acquisition unit 33 may acquire position information of the camera 2 at the time of image capture from the EXIF ​​data of the image, and use this information as the ship's position information.

[0029] The position acquisition unit 33 may supply the acquired position information to the determination unit 35. As an example in the present embodiment, the position acquisition unit 33 may further acquire shooting date and time information indicating the date and time of image capture from the EXIF ​​data, associate the information with the position information, and supply the information to the determination unit 35.

[0030] <1.2.4. Orientation acquisition unit 34> The orientation acquisition unit 34 acquires orientation information indicating the orientation of an object relative to the ship. The orientation of an object relative to the ship may be the direction of a line connecting the ship and the object relative to a reference orientation, and may be expressed as an angle, for example. In this embodiment, as an example, the orientation of an object relative to the ship may be a compass orientation relative to a magnetic meridian. The orientation acquisition unit 34 has an image capture orientation acquisition unit 341 and a calculation unit 342.

[0031] <1.2.4-1. Imaging orientation acquisition unit 341> The imaging orientation acquisition unit 341 acquires the imaging orientation when an image of an object is captured from the ship. The imaging orientation acquisition unit 341 may acquire the imaging orientation from EXIF ​​data. In addition, the imaging orientation acquisition unit 341 may calculate the angle of view of the image from the focal length indicated in the EXIF ​​data. The imaging orientation acquisition unit 341 may supply the imaging orientation and position information of the object in the image supplied from the identification unit 32 to the calculation unit 342. The imaging orientation acquisition unit 341 may further supply the calculated angle of view to the calculation unit 342.

[0032] <1.2.4-2. Calculation unit 342> The calculation unit 342 calculates the orientation of the object relative to the ship based on the image capture orientation. For example, the calculation unit 342 may calculate the image capture orientation as the orientation of the object.

[0033] The calculation unit 342 may calculate the orientation of the object relative to the ship based on the image capture orientation and the position of the object in the image. For example, the calculation unit 342 may calculate the orientation of the object by adding or subtracting an azimuth angle corresponding to the amount of displacement of the object from the center of the image to the azimuth angle of the image capture orientation.

[0034] When the angle of view of the image is supplied from the imaging orientation acquisition unit 341, the calculation unit 342 may calculate the orientation of the object relative to the ship based on the imaging orientation, the position of the object in the image, and the angle of view of the image. For example, the calculation unit 342 may calculate an azimuth angle corresponding to the amount of displacement of the object from the center of the image from the angle of view of the image, and add or subtract this to the azimuth angle of the imaging orientation to calculate the orientation of the object.

[0035] The calculation unit 342 may supply orientation information indicating the orientation of the object to the determination unit 35. In the present embodiment, as an example, the calculation unit 342 may acquire image capture date and time information from the EXIF ​​data, associate it with the orientation information, and supply it to the determination unit 35.

[0036] <1.2.5. Judgment section 35> The determination unit 35 determines whether the object is moving. The determination unit 35 may make this determination based on the position information of the ship and the direction information of the object relative to the ship at at least three points in time. The determination unit 35 has an intersection position identification unit 351 and a determination unit 352.

[0037] <1.2.5-1. Intersection position identification unit 351> The intersection position identifying unit 351 identifies the position of the intersection between straight lines (also called own ship-object straight lines) extending from the position of the own ship toward the bearing of the object at each of at least three points in time.

[0038] The intersection position identification unit 351 may acquire at least three combinations of location information and orientation information associated with the same shooting date and time information from among the location information supplied from the location acquisition unit 33 and the orientation information supplied from the orientation acquisition unit 34.

[0039] For each combination of position information and direction information, the intersection position identifying unit 351 may calculate a straight line from the position indicated by the position information toward the direction indicated by the direction information as the own ship-object line. As an example, the intersection position identifying unit 351 may calculate n own ship-object lines in response to acquiring n combinations (n ​​is a natural number greater than or equal to 3). The intersection position identifying unit 351 may calculate the own ship-object line within a coordinate plane with latitude and longitude as coordinate axes.

[0040] The intersection position identifying unit 351 may identify the intersections of two own ship-object lines selected from the multiple calculated own ship-object lines. As an example, the intersection position identifying unit 351 may identify n-1 intersections in response to calculating n own ship-object lines. The intersection position identifying unit 351 may supply position information of the identified intersections to the determination unit 352. The intersection position identifying unit 351 may further supply information indicating each calculated own ship-object line (also referred to as own ship-object line information) to the determination unit 352, together with image capture date and time information associated with the position information and direction information that formed the own ship-object line.

[0041] <1.2.5-2. Determination unit 352> The determination unit 352 determines whether the object is moving or not depending on whether the positions of at least two intersections identified by the intersection position identification unit 351 are the same. For example, the determination unit 352 may determine that the object is moving depending on whether the positions of the two intersections are different. The determination unit 352 may determine that the object is stationary depending on whether the positions of the at least two intersections are the same.

[0042] The determination unit 352 may supply the result of the determination (also referred to as the judgment result) to the display control unit 37. In response to determining that the object is moving, the determination unit 352 may associate the own ship-object straight line information supplied from the intersection position identification unit 351 with the photographing date and time information and supply them to the estimation unit 36.

[0043] <1.2.6. Estimation part 36> In response to determining that the object is moving, the estimation unit 36 ​​estimates at least one of the object's movement path, the object's movement speed, and the object's position. The estimated object's position may be the object's position at a time different from each time point corresponding to the position information and orientation information used by the determination unit 35 (in this embodiment, the time point indicated by the photographing date and time information, as an example). The estimation unit 36 ​​may perform estimation based on the ship-object straight line at each of at least three time points and the time of each of the time points. The estimation unit 36 ​​may perform estimation based on the assumption that the object is moving in a straight line at a constant speed.

[0044] For example, the estimation unit 36 ​​may determine, as candidate movement paths for the object, each of the lines that intersect with each of the multiple own ship-to-object lines indicated in the own ship-to-object line information supplied from the determination unit 352. As an example, when the determination unit 352 supplies the estimation unit 36 ​​with own ship-to-object line information for three own ship-to-object lines, the estimation unit 36 ​​may determine, as candidate movement paths, each of the multiple lines that intersect with each of these three own ship-to-object lines. The estimation unit 36 ​​may exclude the own ship's movement path from the candidate movement paths.

[0045] The estimation unit 36 ​​may estimate one of the candidate movement paths as the movement path of the object based on the ratio of the intervals between the photographing dates and times corresponding to the own ship-object straight line. The estimation unit 36 ​​may estimate, as the movement path of the object, a candidate movement path among the multiple candidate movement paths in which the ratio of the intervals between the intersections with each own ship-object straight line matches the ratio of the intervals between the photographing dates and times. When there are multiple estimated movement paths, the estimation unit 36 ​​may increase the number of own ship-object straight lines used and perform further estimation to narrow down the candidate movement paths.

[0046] The estimation unit 36 ​​may estimate the movement speed of the object from the distance between the intersections of the estimated movement path and the two own ship-object straight lines and the interval between the image capture dates and times corresponding to the two own ship-object straight lines. The estimation unit 36 ​​may estimate the movement speed of the object as the result of dividing the distance between the intersections of the estimated movement path and the two own ship-object straight lines by the interval between the image capture dates and times corresponding to the two own ship-object straight lines.

[0047] The estimation unit 36 ​​may estimate the position of the object at a time different from the time indicated by the photographing date and time information, based on the estimated movement path and movement speed. The estimation unit 36 ​​may estimate the position of the object at any time designated by the crew, or may estimate the position of the object a preset reference time after the current time. The estimation unit 36 ​​may supply the estimation result to the display control unit 37.

[0048] <1.2.7. Display control unit 37> The display control unit 37 displays an image of the object, and changes the display mode of the object in the image according to the determination result by the determination unit 35. The display control unit 37 may change at least one of the display color, brightness, contrast, or sharpness of the object in the image when it is determined that the object is moving and when it is determined that the object is not moving. The display control unit 37 may display a moving object so that it is more visible than a non-moving object.

[0049] In response to the determination that the object is moving, the display control unit 37 may also display the estimation result supplied from the estimation unit 36. As an example in the present embodiment, the display control unit 37 may cause the display unit 38 to display various information.

[0050] <1.2.8. Display section 38> The display unit 38 may display various information in accordance with the control of the display control unit 37.

[0051] According to the above-described device 3, whether an object is moving can be determined based on the position information of the ship at least three times and the azimuth information of the object relative to the ship, so it is possible to determine whether an object is moving without using radar. Therefore, even small ships that cannot be equipped with radar can determine whether an object is moving. Furthermore, it is possible to determine whether an object that cannot be detected by radar is moving.

[0052] Furthermore, since the orientation of the object relative to the ship is calculated based on the imaging orientation when the object is imaged from the ship, the orientation of the object can be obtained with a simple configuration.

[0053] Furthermore, an image of an object is captured from the ship, the object is detected in the image, and the orientation of the object relative to the ship is calculated based on the position of the object in the image. Therefore, the accurate orientation of the object can be obtained with a simple configuration.

[0054] Furthermore, since the orientation of the object is calculated based on the angle of view of the image, it is possible to obtain a more accurate orientation of the object.

[0055] Furthermore, the positions of the intersections of the lines pointing from the ship's position toward the target's direction at at least three points in time are identified, and whether the target is moving is determined based on whether the positions of the at least two of the identified intersections are the same. This allows for accurate determination of whether the target is moving.

[0056] In addition, an image of the object is displayed, and the display mode of the object in the image changes depending on the judgment result by the judgment unit, so that the appearance of the object in the image and whether the object is moving or not can be grasped at the same time.

[0057] Furthermore, when it is determined that the object is moving, at least one of the direction of movement of the object, the speed of movement of the object, and the position of the object is estimated, thereby making it possible to reliably avoid the moving object.

[0058] 2. Principles of Judgment and Estimation Figure 2 shows the relative positions of the ship and the target when the target is moving.

[0059] In this diagram, the ship is moving and is located at positions P1 to P3 from time t1 to t3. The target object is also moving from time t1 to t3 and is located at positions O1 to O3 from time t1 to t3. The azimuth of the target object relative to the ship at times t1 to t3 is θ1 to θ3. The dashed dotted line in the diagram indicates the magnetic meridian pointing toward magnetic north.

[0060] Here, assuming that the target is stationary between time t1 and time t2, the intersection I of the own ship-target line L1 from position P1 heading to θ1 and the own ship-target line L2 from position P2 heading to θ2 is L1-L2If we assume that the object was stationary between time t2 and time t3, the intersection I of the own ship-object line L2 from position P2 heading to θ2 and the own ship-object line L3 from position P3 heading to θ3 will be L2-L3 If the object is stationary between time t1 and time t3, the intersection point I L1-L2 and intersection I L2-L3 However, in the example shown in this figure, the intersection I L1-L2 and intersection I L2-L3 This makes it possible to determine that the object is moving.

[0061] In this case, the distance T between the time t1 corresponding to the own ship-object line L1 and the time t2 corresponding to the own ship-object line L2 is t1-t2 and the interval T between the time t2 corresponding to the own ship-object line L2 and the time t3 corresponding to the own ship-object line L3. t2-t3 The ratio of the spacing to T t1-t2 :T t2-t3 On the other hand, among the multiple candidate paths K that intersect with the own ship-object straight lines L1 to L3, the candidate path K that matches the actual path of the object is the one that intersects with the three own ship-object straight lines L1 to L3 at points Q1 to Q3, and the distance D between the intersection points Q1 and Q2 is Q1-Q2 and the distance D between the intersections Q2 and Q3 Q2-Q3 Relative to D Q1-Q2 :D Q2-Q3 The ratio T t1-t2 :T t2-t3 and is estimated as the actual movement path Kj of the object. In addition, the distance between the intersection points Q1 and Q2 of the estimated movement path Kj and the own ship-object straight line L1, L2 is the interval T t1-t2 The moving speed of the object is estimated by dividing by . In addition, the position of the object at any point in time is estimated from the estimated moving path Kj and moving speed.

[0062] Figure 3 shows the relative positions of the ship and the target when the target is stationary. In this figure, the ship is moving and is located at positions P4 to P6 from time t4 to t6. The target is stationary from time t4 to t6, remaining at position O. The orientation of the target relative to the ship is θ4 to θ6 at times t4 to t6, respectively.

[0063] In the example shown in this figure, the target is stationary between time t4 and time t5, so the intersection I between the ship-target line L4 heading from position P4 to the heading θ4 and the ship-target line L5 heading from position P5 to the heading θ5 is L4-L5 In addition, since the object is stationary between time t5 and time t6, the intersection I of the own ship-object line L5 from position P5 heading in the direction θ5 and the own ship-object line L6 from position P6 heading in the direction θ6 is L5-L6 The object is stationary between time t4 and time t6, so the intersection point I L4-L5 and intersection I L5-L6 This is the same as the above. As a result, the object is determined to be stationary.

[0064] <3.Operation> 4 shows the operation of the device 3. The device 3 may assist in maneuvering the ship by performing the processes of steps S11 to S23. Note that the device 3 may constantly acquire images using the image acquisition unit 31 and attempt to identify objects in the images using the identification unit 32, and this operation may be initiated in response to an object being identified by the identification unit 32. If multiple objects are identified by the identification unit 32, this operation may be performed for each of the objects.

[0065] In step S11, the position acquisition unit 33 acquires position information indicating the position of the ship. The position acquisition unit 33 may acquire the position of the ship at the time of capturing the most recent image in which the target object was identified by the identification unit 32. In this embodiment, as an example, the position acquisition unit 33 may acquire the position information from the EXIF ​​data of the image.

[0066] In step S13, the orientation acquisition unit 34 acquires orientation information indicating the orientation of the object relative to the ship. The orientation acquisition unit 34 may acquire the orientation information at the time of capturing the most recent image in which the object was identified by the identification unit 32. In the present embodiment, as an example, the orientation acquisition unit 34 may calculate the orientation information based on the image capturing orientation acquired from the EXIF ​​data of the image.

[0067] In step S15, the determination unit 35 determines whether position information and direction information have been acquired for three time points. If it is determined that position information and direction information have not been acquired for three time points (step S15; No), the process may proceed to step S11. In this case, the ship's position information acquired again in step S11 may differ from the previous position information depending on the ship's movement. If it is determined that position information and direction information have been acquired for three time points (step S15; Yes), the process may proceed to step S17. Note that the number of time points determined in step S15 may be four or more.

[0068] In step S17, the determination unit 35 determines whether the target is moving based on the acquired ship position information and orientation information at each time point. For example, the intersection position identification unit 351 of the determination unit 35 may identify the position of the intersection between the ship-target straight lines at each time point. Furthermore, the determination unit 352 of the determination unit 35 may determine whether the target is moving based on whether the positions of at least two intersections identified by the intersection position identification unit 351 are the same. As an example, the determination unit 352 may determine that the target is not moving based on the fact that the first intersection between the ship-target straight line at the first time point and the second time point is the same as the second intersection between the ship-target straight line at the second time point and the third time point, among the times at which the ship position information and orientation information were acquired.

[0069] In step S21, the estimation unit 36 ​​estimates at least one of the movement path of the object, the movement speed of the object, and the position of the object. Note that if it is determined in step S17 that the object is not moving, the process may proceed to step S23 without performing the process of step S21.

[0070] In step S23, the display control unit 37 acquires an image of the object and displays it on the display unit 38, and changes the display mode of the object in the image depending on the determination result of whether the object is moving. If estimation is performed by the estimation unit 36 ​​in step S21, the display control unit 37 may also display the estimation result on the display unit 38.

[0071] According to the above operation, it is determined that the object is not moving if the first intersection between the ship and object line at the first and second time points is the same as the second intersection between the ship and object line at the second and third time points. This reduces the burden of determination and speeds up the determination compared to when it is determined that the object is not moving if three or more intersections are the same.

[0072] <4. Modifications> In the above embodiment, the device 3 has been described as having the estimation unit 36 ​​and the display control unit 37, but it may not have either of them. For example, if the device 3 does not have the display control unit 37, the determination unit 35 may output the determination result to the outside.

[0073] Furthermore, although the position acquisition unit 33 has been described as acquiring the ship's position information from the EXIF ​​data of the image, it may also acquire the ship's position information from a GPS (Global Positioning System) signal received by the camera 2 itself. In this case, the ship's position information can be acquired even when the camera 2 is not capable of positioning. The position acquisition unit 33 may store and associate the ship's position information with the acquisition time of the position information, read out the position information corresponding to the shooting date and time of the image in which the object was identified by the identification unit 32, and supply it to the determination unit 35.

[0074] Furthermore, although the imaging orientation acquisition unit 341 has been described as acquiring the imaging orientation from EXIF ​​data, the imaging orientation may be acquired by other methods. For example, the imaging orientation acquisition unit 341 may acquire the magnetic bearing of the ship's direction of travel and the orientation of the camera 2 relative to the ship's hull, and may calculate the imaging orientation by adding or subtracting the orientation of the camera 2 relative to the hull to the magnetic bearing. The imaging orientation acquisition unit 341 may store and associate the imaging orientation with the acquisition time of the imaging orientation, read out the imaging orientation corresponding to the date and time of capture of the image in which the target object was identified by the identification unit 32, and supply the image to the calculation unit 342.

[0075] Various embodiments of the present invention may also be described with reference to flowcharts and block diagrams, where the blocks may represent (1) stages of a process in which operations are performed or (2) sections of an apparatus responsible for performing the operations. Particular stages and sections may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable medium, and / or a processor provided with computer-readable instructions stored on a computer-readable medium. Dedicated circuitry may include digital and / or analog hardware circuitry, and may include integrated circuits (ICs) and / or discrete circuits. Programmable circuitry may include reconfigurable hardware circuitry, including logical AND, OR, XOR, NAND, NOR, and other logic operations, flip-flops, registers, memory elements such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like.

[0076] A computer-readable medium may include any tangible device capable of storing instructions that are executed by an appropriate device, such that the computer-readable medium having instructions stored thereon comprises an article of manufacture containing instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable media may include electronic, magnetic, optical, electromagnetic, and semiconductor storage media. More specific examples of computer-readable media may include floppy disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-ray (RTM) disc, memory stick, integrated circuit card, and the like.

[0077] The computer readable instructions may include either assembler instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0078] The computer-readable instructions may be provided to a processor or programmable circuitry of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, either locally or over a wide-area network (WAN) such as a local area network (LAN), the Internet, etc., which executes the computer-readable instructions to create means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0079] 5 illustrates an example of a computer 2200 in which aspects of the present invention may be embodied, in whole or in part. Programs installed on the computer 2200 may cause the computer 2200 to function as or perform operations associated with an apparatus or one or more sections of the apparatus according to embodiments of the present invention, and / or to perform a process or steps of a process according to embodiments of the present invention. Such programs may be executed by the CPU 2212 to cause the computer 2200 to perform specific operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

[0080] A computer 2200 according to this embodiment includes a CPU 2212, a RAM 2214, a graphics controller 2216, and a display device 2218, which are interconnected by a host controller 2210. The computer 2200 also includes input / output units such as a communication interface 2222, a hard disk drive 2224, a DVD-ROM drive 2226, and an IC card drive, which are connected to the host controller 2210 via an input / output controller 2220. The computer also includes legacy input / output units such as a ROM 2230 and a keyboard 2242, which are connected to the input / output controller 2220 via an input / output chip 2240.

[0081] The CPU 2212 operates according to programs stored in the ROM 2230 and RAM 2214, thereby controlling each unit. The graphics controller 2216 acquires image data generated by the CPU 2212 into a frame buffer or the like provided in the RAM 2214 or into the graphics controller 2216 itself, and causes the image data to be displayed on the display device 2218.

[0082] The communication interface 2222 communicates with other electronic devices via a network. The hard disk drive 2224 stores programs and data used by the CPU 2212 in the computer 2200. The DVD-ROM drive 2226 reads programs or data from the DVD-ROM 2201 and provides the programs or data to the hard disk drive 2224 via the RAM 2214. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0083] The ROM 2230 stores therein a boot program or the like that is executed by the computer 2200 upon activation, and / or programs that depend on the hardware of the computer 2200. The input / output chip 2240 may also connect various input / output units to the input / output controller 2220 via a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0084] The programs are provided by a computer-readable medium such as a DVD-ROM 2201 or an IC card. The programs are read from the computer-readable medium, installed in the hard disk drive 2224, RAM 2214, or ROM 2230, which are also examples of computer-readable media, and executed by the CPU 2212. Information processing described in these programs is read by the computer 2200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be configured by realizing information manipulation or processing in accordance with the use of the computer 2200.

[0085] For example, when communication is performed between the computer 2200 and an external device, the CPU 2212 may execute a communication program loaded into the RAM 2214 and instruct the communication interface 2222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 2212, the communication interface 2222 reads transmission data stored in a transmission buffer processing area provided in the RAM 2214, the hard disk drive 2224, the DVD-ROM 2201, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer processing area or the like provided on the recording medium.

[0086] The CPU 2212 may also cause all or a necessary portion of a file or database stored on an external recording medium such as the hard disk drive 2224, the DVD-ROM drive 2226 (DVD-ROM 2201), an IC card, etc. to be read into the RAM 2214, and perform various types of processing on the data on the RAM 2214. The CPU 2212 then writes back the processed data to the external recording medium.

[0087] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and may undergo information processing. The CPU 2212 may perform various types of processing on data read from the RAM 2214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write the results back to the RAM 2214. The CPU 2212 may also search for information in a file, database, etc. on the recording medium. For example, if multiple entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored on the recording medium, the CPU 2212 may search for an entry that matches a condition specified by the attribute value of the first attribute from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0088] The above-described programs or software modules may be stored in a computer-readable medium on or near the computer 2200. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable medium, thereby providing the programs to the computer 2200 via the network.

[0089] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0090] It should be noted that the execution order of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a subsequent process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order. [Explanation of symbols]

[0091] 1 System 2 Cameras 3 equipment 31 Image acquisition unit 32 Specific part 33 Position acquisition part 34 Direction acquisition part 35 Judgment section 36 Estimation part 37 Display control unit 38 Display section 341 Imaging direction acquisition unit 342 Calculation Unit 351 Intersection position identification part 352 Decision Section 2200 Computer 2201 DVD-ROM 2210 host controller 2212 CPU 2214 RAM 2216 Graphics Controller 2218 Display Device 2220 Input / Output Controller 2222 communication interface 2224 hard disk drive 2226 DVD-ROM drive 2230 ROM 2240 I / O chip 2242 keyboard

Claims

1. a position acquisition unit that acquires position information indicating the position of the ship; a direction acquisition unit that acquires direction information indicating the direction of an object relative to the ship; a determination unit that determines whether the object is moving based on the position information and the orientation information at least at three points in time; Equipped with The determination unit an intersection position identifying unit that identifies the position of an intersection between straight lines pointing from the position of the ship toward the bearing of the object at each of the at least three points in time; a determination unit that determines whether the object is moving based on whether the positions of at least two intersections identified by the intersection position identification unit are the same; and An apparatus having:

2. The determination unit a first intersection between a straight line extending from the position of the ship at a first time point toward the bearing of the object and a straight line extending from the position of the ship at a second time point toward the bearing of the object; and a second intersection between a straight line extending from the position of the ship at the second time point toward the bearing of the object and a straight line extending from the position of the ship at a third time point toward the bearing of the object; The apparatus of claim 1 , further comprising: determining that the object is not moving in response to the fact that the first and second values ​​of the first and second values ​​of the second and third ...

3. The orientation acquisition unit an image capturing direction acquisition unit that acquires an image capturing direction when the target object is captured from the ship; a calculation unit that calculates the orientation of the object relative to the ship based on the imaging orientation; 10. The apparatus of claim 1, comprising:

4. an image acquisition unit that acquires images captured from the ship; an identification unit that identifies the object in the image; Furthermore, The apparatus according to claim 3 , wherein the calculation unit calculates the orientation of the object relative to the ship based further on the position of the object in the image.

5. The device according to claim 4 , wherein the calculation unit calculates the orientation of the object relative to the ship based further on the angle of view of the image.

6. an image acquisition unit that acquires images captured from the ship; an identification unit that identifies the object in the image; a display control unit that displays an image of the object and changes a display mode of the object in the image according to a determination result by the determination unit; The apparatus of claim 1 further comprising:

7. The device according to claim 1 , wherein the location acquisition unit acquires the location information from a GPS signal.

8. The device described in claim 1 further includes an estimation unit that, in response to a determination that the object is moving, estimates at least one of the object's movement path, the object's movement speed, and the object's position at a time different from the at least three time points based on a straight line from the ship's position at each of the at least three time points toward the object's bearing and the time of each of the at least three time points.

9. A position acquisition step in which position information indicating the position of the ship is acquired by a computer; a direction acquisition step of acquiring direction information indicating the direction of an object relative to the ship by a computer; a determining step of determining, by a computer, whether the object is moving based on the position information and the orientation information at least three times; Equipped with The determining step includes: an intersection position identifying step of identifying the position of an intersection between straight lines pointing from the position of the ship toward the bearing of the object at each of the at least three points in time; a determining step of determining whether the object is moving or not depending on whether the positions of at least two intersections identified by the intersection position identifying step are the same or different; A method having the following.

10. Computer, a position acquisition unit that acquires position information indicating the position of the ship; a direction acquisition unit that acquires direction information indicating the direction of an object relative to the ship; a determination unit that determines whether the object is moving based on the position information and the orientation information at least at three points in time; It functions as The determination unit an intersection position identifying unit that identifies the position of an intersection between straight lines pointing from the position of the ship toward the bearing of the object at each of the at least three points in time; a determination unit that determines whether the object is moving based on whether the positions of at least two intersections identified by the intersection position identification unit are the same; and A program with.

Images