Information Processing Apparatus, Monitoring Apparatus, Information Processing Method, and Program

Abstract

To provide an information processing system, a monitoring device, an information processing method, and a program capable of detecting a target approaching a ship.SOLUTION: An information processing system includes: an acquisition unit that acquires a captured image captured by an imaging unit installed in a ship; an identification unit that identifies a nearest point of a target, which is image-recognized in the captured image, relative to the imaging unit; a first approach determination unit that determines whether or not the nearest point falls within a first approach area which is an area on the side of the ship in the captured image; and a first angle determination unit that determines whether or not a first angle in the captured image of the nearest point determined to fall within the first approach area remains unchanged or gets smaller. The first angle is an angle at which a direction oriented to the nearest point is oblique to an imaging axis of the imaging unit.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, a monitoring apparatus, an information processing method, and a program.

Background Art

[0002] Patent Document 1 describes the following monitoring apparatus. That is, the monitoring apparatus described in Patent Document 1 interlocks a radar apparatus and a ship navigation support apparatus. When an operation of designating the azimuth and distance from the own ship of a target ship is performed on one display screen, the position of the target ship designated at the same azimuth and the same distance is displayed on the other display screen. Further, the ship navigation support apparatus has a display screen for displaying a view of the sea photographed by a camera apparatus. When an operation of designating the position of a target ship is performed on one of the display screens of the radar apparatus and the ship navigation support apparatus, the camera apparatus enlarges and photographs the view in the direction including the target ship, and the target ship is enlarged and displayed on the display screen of the ship navigation support apparatus.

[0003] According to the monitoring apparatus described in Patent Document 1, for example, when a target ship is specified on the display screen of the radar apparatus, the target ship on the radar image of the ship navigation support apparatus is specified. Then, by looking at the target ship interference zone in the predicted course of the target ship, it can be determined whether there is a possibility of collision. Therefore, a navigation plan can be made on the display screen of the radar apparatus. Further, by enlarging and photographing the specified target ship with the camera apparatus and enlarging and displaying the target ship on the display screen of the ship navigation support apparatus, it is possible to know what kind of ship the target ship is.

[0004] Note that the ship navigation support apparatus described in Patent Document 1 is also called an Automatic Radar Plotting Aid (ARPA).

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, the monitoring device described in Patent Document 1 uses radar to identify the position of an approaching target. Therefore, for example, in the case of a target that is difficult to detect with radar, there is a problem that the approach of the target to the ship may not be detected.

[0007] The present disclosure has been made to solve the above problems, and an object thereof is to provide an information processing device, a monitoring device, an information processing method, and a program capable of detecting a target approaching a ship.

Means for Solving the Problems

[0008] To solve the above problems, an information processing device according to the present disclosure includes an acquisition unit that acquires a captured image captured by a photographing unit installed on a ship, a specifying unit that specifies a closest point to the photographing unit in a target recognized by image recognition in the captured image, a first approach determination unit that determines whether the closest point is within a first approach area that is an area on the ship side in the captured image, and a first angle determination unit that determines whether a first angle in the captured image regarding the closest point determined to be within the first approach area remains the same or decreases, where the first angle is an angle at which the direction facing the closest point is inclined with respect to the photographing axis of the photographing unit.

[0009] An information processing method according to the present disclosure includes a step of acquiring a captured image captured by a photographing unit installed on a ship, a step of specifying a closest point to the photographing unit in a target recognized by image recognition in the captured image, a step of determining whether the closest point is within a first approach area that is an area on the ship side in the captured image, and a step of determining whether a first angle in the captured image regarding the closest point determined to be within the first approach area remains the same or decreases, where the first angle is an angle at which the direction facing the closest point is inclined with respect to the photographing axis of the photographing unit.

[0010] A program according to the present disclosure causes a computer to execute steps of: obtaining a captured image captured by a capturing unit installed on a ship; specifying a closest point on a target recognized in the captured image with respect to the capturing unit; determining whether the closest point is within a first approaching region that is a region on the ship side in the captured image; and determining whether a first angle in the captured image regarding the closest point determined to be within the first approaching region remains the same or decreases, where the first angle is an angle at which a direction facing the closest point inclines with respect to the imaging axis of the capturing unit.

Advantages of the Invention

[0011] According to the information processing apparatus, monitoring apparatus, information processing method, and program of the present disclosure, a target approaching a ship can be detected.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Mode for Carrying Out the Invention

[0013] (Configuration of Monitoring Device) Hereinafter, an information processing apparatus, a monitoring apparatus, an information processing method, and a program according to embodiments of the present disclosure will be described with reference to FIGS. 1 to 12. FIG. 1 is a block diagram showing a configuration example of a monitoring apparatus according to an embodiment of the present disclosure. FIG. 2 is a block diagram showing a functional configuration example of an information processing apparatus according to an embodiment of the present disclosure. FIG. 3 is a schematic diagram showing an example of a captured image of a camera according to an embodiment of the present disclosure. FIGS. 4 to 7 are schematic diagrams for explaining an information processing apparatus according to an embodiment of the present disclosure. FIGS. 8 to 11 are flowcharts showing operation examples of an information processing apparatus according to an embodiment of the present disclosure. And, FIG. 12 is a schematic block diagram showing the configuration of a computer according to an embodiment of the present disclosure. In addition, the same or corresponding components in each figure are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

[0014] As shown in FIG. 1, a monitoring apparatus 100 according to an embodiment of the present disclosure includes an information processing apparatus 1 and a camera 2. The information processing apparatus 1 can be configured using a computer such as a notebook PC (personal computer), a tablet terminal, a smartphone, and its peripheral devices. The information processing apparatus 1 includes a processing apparatus 11, an input apparatus 12, a display apparatus 13, and a speaker 14. The processing apparatus 11 includes a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and the like.

[0015] The input device 12 includes, for example, a keyboard, a mouse, a touch panel integrally configured with the display device 13, a voice recognition device, and the like. The display device 13 is, for example, a liquid crystal display, an organic EL (electroluminescence) display, or the like. The speaker 14 outputs an alarm sound, a synthesized voice signal, and the like.

[0016] As shown in FIG. 3, for example, the camera 2 is installed on the ship 200 so as to be able to photograph the sea 300 around the ship 200. The captured image 2a shown in FIG. 3 includes another ship 400 sailing slightly to the left in front of the ship 200. Note that the camera 2 can be attached not only in the directly forward direction but in any direction of 360°. Further, the monitoring device 100 may include a plurality of cameras 2. Further, the camera 2 may be configured to photograph in all directions of 360°. Further, the camera 2 can be a visible light camera, an infrared camera, or a combination thereof. The camera 2 repeatedly captures images at a predetermined period, for example, and transmits information indicating the captured images to the information processing device 1 via a predetermined communication line. In the present embodiment, it is assumed that the horizontal direction of the position of each pixel included in the captured image 2a is the X direction and the vertical direction is the Y direction. Further, the camera 2 is an example of the imaging unit according to the present disclosure.

[0017] As will be described later, the monitoring device 100 performs image recognition on the captured image captured by the camera 2 using the information processing device 1, and determines the approach of the target object recognized in the image to the ship by the behavior of the position of the target object in the image, thereby detecting the approach of the target object. Here, the target objects to be detected are, for example, other ships, buoys, floating objects, and the like.

[0018] (Functional configuration of information processing device) Next, with reference to FIG. 2 and the like, the functional configuration of the information processing apparatus 1 will be described. The information processing apparatus 1 includes the following components as a functional configuration composed of a combination of hardware such as a computer and peripheral devices that constitute the information processing apparatus 1, and software such as programs executed by the computer. That is, the information processing apparatus 1 includes an acquisition unit 101, an image recognition unit 102, a specifying unit 103, a first proximity determination unit 104, a first angle determination unit 105, a first continuation determination unit 106, a second proximity determination unit 107, a second angle determination unit 108, a second continuation determination unit 109, an alarm output unit 110, a canceling unit 111, a setting unit 112, and a display unit 113.

[0019] The acquisition unit 101 repeatedly acquires, for example, at a predetermined cycle, a captured image 2a captured by the camera 2 installed on the ship 200.

[0020] The image recognition unit 102 performs image recognition processing on the captured image 2a, recognizes a target having a predetermined feature (hereinafter, when referring to the drawings, etc., the ship 400 will be described as an example of a target), and specifies a recognition frame RF1 surrounding the ship 400 as shown in FIG. 6, for example. The recognition frame RF1 is a rectangle (rectangular shape) with all four corners being right angles. The recognition frame RF1 can be specified, for example, by the coordinates (XR1, YR1) of the left end of the lower side, the coordinates (XR2, YR2) of the right end, and the coordinates (XR3, YR3) of the left end of the upper side. Alternatively, the recognition frame RF1 can be specified, for example, by the coordinates (XR1, YR1) of the left end of the lower side and the lengths of two sides. There is no limitation on the method of image recognition processing in the image recognition unit 102. For example, by image recognition processing using pattern matching or a learned machine learning model, targets such as other ships, buoys, and floating objects are recognized, and the recognition frame RF1 surrounding the recognized target is specified.

[0021] In addition, when the image recognition unit 102 recognizes a target, it assigns a recognition code for identifying the target to each target. Then, the image recognition unit 102 specifies the same target in a plurality of consecutive captured images 2a, and records the change in the coordinates of each target in association with each recognition code. Also, the image recognition unit 102 can perform determination processing on whether or not a target is included in the captured image 2a.

[0022] The specific part 103 specifies the closest point to the camera 2 on the ship 400 recognized by image recognition in the captured image 2a. The closest point is the point that is closest to the camera 2 within the recognition frame RF1 of the ship 400 on the captured image 2a. In the example shown in FIG. 7, the closest point MN is the point closer to the imaging axis PA of the camera 2 extending in the vertical direction (Y direction) from the center CB at the lower end of the captured image 2a among the points of the coordinates (XR1, YR1) at the left end of the lower side of the recognition frame RF1 or the points of the coordinates (XR2, YR2) at the right end. In this case, the closest point MN is the point of the coordinates (XR2, YR2) at the right end.

[0023] The first approach determination unit 104 determines whether the closest point MN is within the first approach region, which is the region on the ship 200 side in the captured image 2a. In the present embodiment, the determination of whether the target is approaching the ship 200 is made by providing two types of approach regions. In the example shown in FIG. 4, a first approach region FR and a second approach region SR are set within the captured image 2a. The first approach region FR is the region below the boundary line FL indicated by the one-dot chain line. The second approach region SR is the region below the boundary line SL indicated by the two-dot chain line. Note that the first approach region FR and the second approach region SR are regions at a predetermined distance from the lower end of the captured image 2a. Also, the second approach region SR is a region closer to the ship 200 than the first approach region FR.

[0024] Note that it is desirable to be able to set the first approach region FR and the second approach region SR arbitrarily, for example, by inputting them as parameters. The mounting height of the camera 2 varies depending on the ship 200 to be mounted. Therefore, the distance on the captured image 2a from the ship 200 to the first approach region FR and the second approach region SR changes for each ship 200. The second approach region SR can be set, for example, by inputting as parameters the number of pixels (P1 to P5) from the lower end of the captured image 2a at a plurality of points (FIG. 4 shows an example of 5 points) at regular intervals (X1 to X5). The first approach region FR can be set in the same way. Note that there is no limitation on the setting method of the first approach region FR and the second approach region SR as long as the first approach region FR includes the second approach region SR.

[0025] The first angle determination unit 105 determines whether the first angle θ1 (FIG. 7) in the captured image 2a regarding the nearest point MN determined to be within the first proximity region FR remains the same or becomes smaller. The first angle θ1 is the angle at which the direction facing the nearest point MN is inclined with respect to the imaging axis PA of the camera 2. In the present embodiment, the direction facing the nearest point MN is the direction extending from the center CB at the lower end of the captured image 2a to the nearest point MN. Further, in the present embodiment, the determination of the angle by the first angle determination unit 105 is executed based on an angle region having a certain angle range. FIG. 5 shows an example of the setting of the angle region. In the example shown in FIG. 5, with the center CB at the lower end of the captured image 2a as the center, the direction of the imaging axis PA is set to 0°, and 90° is defined on each side, with 5° increments, and the left angle regions RL1 to RL18 and the right angle regions RR1 to RR18 are set. As shown in FIG. 7, when the nearest point MN is located in the same region (angle region RL3 in the example of FIG. 7), the first angle determination unit 105 determines that the first angle θ1 remains the same. Further, when the region where the nearest point MN is located moves from, for example, the angle region RL3 to the angle region RL2, the first angle determination unit 105 determines that the first angle θ1 has become smaller. Each angle region includes the maximum angle of the angle region and does not include the minimum angle of the angle region. For example, the angle region RL1 is an angle region greater than 0° and less than or equal to 5°. Note that 0° is not included in either the angle region RL1 or the angle region RR1, and the first angle determination unit 105 determines that the case where the first angle θ1 remains 0° is the case where the first angle θ1 remains the same. By the determination by the first angle determination unit 105, it is possible to recognize whether the target is a target approaching or a target moving away. In addition, when the nearest point MN frequently switches between the point with the left end coordinates (XR1, YR1) and the point with the right end coordinates (XR2, YR2) near 0°, both may be evaluated as the nearest point to make a safer judgment.

[0026] The first continuation determination unit 106 determines that the determination that the first angle θ1 remains the same or becomes smaller continues for a predetermined time or more.

[0027] The second proximity determination unit 107 determines whether the nearest point MN is within the second proximity area SR.

[0028] The second angle determination unit 108 determines whether the second angle in the captured image 2a regarding the nearest point MN determined to be within the second proximity area SR remains the same or becomes smaller. The second angle is the angle at which the direction facing the nearest point MN located within the second proximity area SR is inclined with respect to the imaging axis PA of the camera 2. The second angle corresponds to the first angle θ1 shown in FIG. 7 when the nearest point MN is located within the second proximity area SR. Note that the second angle determination unit 108 may use the same angular regions RL1 to RL18 and RR1 to RR18 as those used by the first angle determination unit 105 described with reference to FIG. 5 as the criteria for determination, or may use different angular regions as the criteria.

[0029] The second continuation determination unit 109 determines that the determination that the second angle remains the same or becomes smaller continues for a predetermined time or more. The predetermined time used as the criterion by the second continuation determination unit 109 is usually set shorter than the predetermined time used as the criterion by the first continuation determination unit 106.

[0030] The alarm output unit 110 outputs an alarm signal related to the first proximity area FR based on the determination result of the first proximity determination unit 104, the determination result of the first angle determination unit 105, and the determination result of the first continuation determination unit 106. The alarm output unit 110 further outputs an alarm signal related to the second proximity area SR based on the determination result of the second proximity determination unit 107, the determination result of the second angle determination unit 108, and the determination result of the second continuation determination unit 109, preferentially over the alarm signal related to the first proximity area FR.

[0031] When the closest point MN of the ship 400 enters the first approach area FR and the first angle θ1 is located within the same angular region (or is the same angle) or the state where the first angle θ1 becomes smaller continues for a predetermined time, the alarm output unit 110 outputs an alarm for the first approach area FR. For example, an image indicating approach is displayed around the image of the ship 400 displayed on the display device 13, or an alarm sound or alarm voice is output from the speaker 14. Further, when the closest point MN of the ship 400 enters the second approach area SR and the state where the second angle is located within the same angular region (or is the same angle) or the second angle becomes smaller continues for a predetermined time, the alarm output unit 110 outputs an alarm for the second approach area SR. For example, an image indicating approach is displayed around the image of the ship 400 displayed on the display device 13, or an alarm sound or alarm voice is output from the speaker 14. Note that it is desirable that the alarm for the second approach area SR be an alarm indicating a higher level of urgency than the alarm for the first approach area FR. For the alarm for the second approach area SR, for example, the size of the image display is increased, the period of image blinking is shortened, or the period of alarm sound emission is shortened. Also, even when the alarm signal related to the first approach area FR satisfies the condition for sounding an alarm, when the alarm signal related to the second approach area SR satisfies the condition for sounding an alarm, the alarm output unit 110 does not sound the alarm related to the first approach area FR and outputs only the alarm related to the second approach area SR.

[0032] Note that when the alarm output unit 110 outputs an alarm on an image, for example, when there are a plurality of targets, alarms for the plurality of targets can be output in parallel.

[0033] The cancellation unit 111 cancels the alarm signal for each object recognized in the image. For example, in the captured image 2a shown in FIG. 6, when the inside of the recognition frame RF1 is clicked, the cancellation unit 111 displays a button or the like for canceling the alarm, and when the button is clicked, the alarm for the ship 400 is canceled. The alarm can be canceled at any time after the recognition frame RF1 is displayed as shown in FIG. 6, for example. It may be after the alarm is issued or before the alarm is issued. When the alarm is canceled, no alarm is issued for the object. Note that the cancellation of the alarm may be made cancellable. Also, it may be possible to cancel or cancel the alarm for a plurality of objects at once.

[0034] The setting unit 112 makes new settings or changes to the settings of the first approach area FR, the second approach area SR, the angular area, and a predetermined time (for continuous determination) in response to an input operation on the input device 12.

[0035] The display unit 113 superimposes an image showing the recognition frame RF1, the boundary line FL, the boundary line SL, the alarm signal, etc. on the captured image 2a and displays it on the display device 13 as shown in FIG. 6, for example.

[0036] (Operation example of the information processing device) An operation example of the information processing device 1 will be described with reference to FIGS. 8 to 11. The process shown in FIG. 8 is repeatedly executed for every one or more frames in accordance with the frame period of the camera 2, for example. The process shown in FIG. 9 is executed at an arbitrary timing in response to an operation by the operator. The processes shown in FIGS. 10 and 11 are repeatedly executed at a predetermined period longer than the period of the process shown in FIG. 8, for example. Note that the flows shown in FIGS. 10 and 11 are coupled to each other by the connectors C1 and C2.

[0037] When the process shown in FIG. 8 starts, first, the acquisition unit 101 acquires a captured image (step S11). Next, the image recognition unit 102 performs image recognition on the target (step S12). Next, the image recognition unit 102 determines whether the captured image 2a contains a target (step S13). If the captured image 2a contains a target (step S13: YES), the image recognition unit 102 determines whether there is a new target (step S14). If there is a new target (step S14: YES), the image recognition unit 102 assigns a recognition code to the new target (step S15).

[0038] If there is no new target (step S14: NO) or after step S15, the display unit 113 superimposes a recognition frame RF1 surrounding each target on the captured image 2a (step S16). Next, the alarm output unit 110 outputs an alarm according to the set state of the alarm output (step S17). In this embodiment, as the set state of the alarm output, for each target, three states of output, non-output, and cancel are set. Also, for output and non-output, there are set states of output and non-output of the alarm signal for the first approach area FR and output and non-output of the alarm signal for the second approach area SR. The alarm output unit 110 outputs the alarm signal for the first approach area FR or the alarm signal for the second approach area SR according to the set state when there is an output setting of the alarm signal that is not set to cancel.

[0039] If there is no target (step S13: NO) or after step S17, the display unit 113 superimposes an image indicating the first approach area FR and the second approach area SR on the captured image 2a (step S18). Next, the display unit 113 displays the image on the display screen of the display device 13 (step S19), and ends the process shown in FIG. 8.

[0040] Also, in the process shown in FIG. 9, when there is an alarm cancellation operation (step S21: YES), the cancellation unit 111 cancels the alarm for the specified target (step S22).

[0041] Also, in the processes shown in FIGS. 10 and 11, for example, the specifying unit 103 determines whether there is an object for which the alarm has not been canceled (step S101). If there is an object for which the alarm has not been canceled (step S101: YES), for example, the specifying unit 103 determines whether all the uncanceled objects have been determined (step S102). If there is no object for which the alarm has not been canceled (step S101: NO), or if all the uncanceled objects have been determined (step S102: YES), for example, the specifying unit 103 ends the processes shown in FIGS. 10 and 11. If not all the uncanceled objects have been determined (step S102: NO), for example, the specifying unit 103 selects an object to be determined (step S103). Next, the specifying unit 103 specifies the closest point of the selected object (step S104).

[0042] Next, the second approach determination unit 107 determines whether the closest point is within the second approach area (step S105). If the closest point is within the second approach area (step S105: YES), the second angle determination unit 108 determines whether the angle area of the closest point remains the same or has become smaller (step S106). If the closest point is not within the second approach area (step S105: NO), the second approach determination unit 107 sets the alarm output state for the second approach area for the object to non-output (step S109). If the angle area of the closest point does not remain the same and has not become smaller (step S106: NO), the second angle determination unit 108 sets the alarm output state for the second approach area for the object to non-output (step S109).

[0043] If the angle area of the closest point remains the same or has become smaller (step S106: YES), the second continuation determination unit 109 determines whether the above state has continued for a second time or more (step S107). The above state is the state where step S105 is YES and step S106 is YES.

[0044] When the above state continues for the second time or longer (step S107: YES), the second continuation determination unit 109 sets the state of the alarm output for the second approach area for the target to output (step S108). When the above state has not continued for the second time or longer (step S107: NO), the second continuation determination unit 109 sets the state of the alarm output for the second approach area for the target to non-output (step S109).

[0045] After step S108 or step S109, the first approach determination unit 104 determines whether the closest point is within the first approach area (step S110). When the closest point is within the first approach area (step S110: YES), the first angle determination unit 105 determines whether the angle area of the closest point remains the same or becomes smaller (step S111). When the closest point is not within the first approach area (step S111: NO), the first approach determination unit 104 sets the state of the alarm output for the first approach area for the target to non-output (step S115). When the angle area of the closest point does not remain the same and does not become smaller (step S111: NO), the first angle determination unit 105 sets the state of the alarm output for the first approach area for the target to non-output (step S115).

[0046] When the angle area of the closest point remains the same or becomes smaller (step S111: YES), the first continuation determination unit 106 determines whether the above state has continued for the first time or longer (step S112). The above state is the state where step S110 is YES and step S111 is YES.

[0047] When the above state has continued for the first time or longer (step S112: YES), for example, the first continuation determination unit 106 determines whether the alarm output state of the second approach area is set to output (step S113). When the alarm output state of the second approach area is not set to output (step S113: NO), for example, the first continuation determination unit 106 sets the state of the alarm output for the first approach area for the target to output (step S114).

[0048] If the above state has not continued for more than the first time (step S112: NO), or if the alarm output state in the second proximity area is set to output (step S113: YES), for example, the first continuation determination unit 106 sets the alarm output state of the first proximity area for the target to non-output (step S115).

[0049] After step S114 or step S115, for example, the specifying unit 103 determines whether all the uncanceled targets have been determined (step S102).

[0050] Through the above processing, the information processing apparatus 1 can detect a target approaching the ship. Further, an alarm signal can be output according to the detection result.

[0051] (Function, effect and supplementary explanation) In this embodiment, without calculating the distance to the target (other ship, buoy, floating object), the approach is determined and an alarm is issued based on the behavior of the position on the screen of the closest pixel of the image-recognized target. At this time, the information processing apparatus 1 outputs an alarm when the following conditions are satisfied. First, for example, it is a condition that a part of the image-detected target enters the image range below the boundary line FL of the first proximity area FR. Next, as shown in FIG. 7, the angle between the center CB at the lower end of the captured image (monitoring screen) and the closest point of the target (defined as 0° in the vertical direction at the center of the screen and 90° to the left and right respectively) is calculated, and the condition is that the state where the angle in the angular region at a certain angular interval remains the same or becomes smaller continues for a predetermined time or more. When these conditions are satisfied, the information processing apparatus 1 outputs an alarm signal (first approach alarm for the first proximity area FR). After that, if all of the image-detected targets are within the range above the boundary line FL, or if the angle of the angular region becomes larger, the alarm disappears. Note that the predetermined time is, for example, in units of seconds and can be variably set.

[0052] Also, in this embodiment, the information processing apparatus 1 includes the first continuation determination unit 106, so that a target having an approaching tendency over a predetermined time can be determined. Therefore, the information processing apparatus 1 can identify a target that requires an alarm.

[0053] Also, in the present embodiment, the information processing apparatus 1 includes a second approach determination unit 107, and thus can perform determination related to the second approach region SR independently of the determination related to the first approach region FR. Therefore, with the second approach determination unit 107, a target approaching the ship 200 so rapidly that it cannot be caught up with by the determination related to the first approach region FR can be alerted by the determination related to the first approach region FR.

[0054] Also, in the present embodiment, the information processing apparatus 1 includes a second angle determination unit 108, and thus can determine the approach of a target based on the behavior of the closest point recognized in the image.

[0055] Also, in the present embodiment, the information processing apparatus 1 includes a second continuation determination unit 109, and thus can determine a target having an approaching tendency over a predetermined time. Therefore, the information processing apparatus 1 can identify a target that requires an alert.

[0056] Also, in the present embodiment, in the information processing apparatus 1, since the predetermined time in the determination of the second continuation determination unit 109 is set shorter than the predetermined time in the first continuation determination unit 106, according to this aspect, a target approaching the ship 200 so rapidly that it cannot be caught up with by the determination related to the first approach region FR can be alerted by the determination related to the first approach region FR.

[0057] Also, in the present embodiment, since an alert signal related to the second approach region SR is output preferentially over an alert signal related to the first approach region FR, an alert related to the second approach region can be prioritized over an alert related to the first approach region FR. Therefore, the information processing apparatus 1 can preferentially alert a target approaching the ship 200.

[0058] Also, in the present embodiment, the information processing apparatus 1 includes a cancel unit 111, and thus an operator can exclude a target from the alert targets when an alert is not necessarily required, such as for a ship operating at a short distance from the own ship in the same fleet.

[0059] Below the boundary line SL of the second proximity area SR, as with the same processing, the determination time can be set independently of the boundary line FL, and an alarm signal (a second proximity alarm for the second proximity area SR) is output. The alarm cancellation is the same as the processing for the boundary line FL.

[0060] For the same image detection target, if the conditions for alarm output within the image range below the boundary line SL are satisfied during the determination of alarm output within the image range below the boundary line FL, the determination of alarm output within the image range below the boundary line FL is set to non-output. From the time when the closest point of the image detection target exits outside the image range below the boundary line SL, the determination of alarm output within the image range below the boundary line FL is performed again.

[0061] Incidentally, collision accident avoidance of ships is generally performed by radar and AIS (Automatic Identification System) in addition to visual observation. There are small ships such as fishing boats and pleasure boats, buoys, and floating objects that are difficult to detect with radar, and there are also ships without AIS. Detection of such targets depends greatly on visual observation. According to the monitoring device 100 of the present embodiment, visual monitoring work can be supported. According to the present embodiment, human errors in target detection can be compensated for, and the labor burden can be reduced.

[0062] Note that the monitoring device 100 according to the present embodiment, as a countermeasure against the shaking of the monitoring image when a camera such as a monitoring camera is equipped on a ship that sways on the ocean, 1) mechanical correction by a gimbal or a stabilizer, 2) correction by electronic calculation using an electronic circuit built into the camera body or image processing software, 3) measurement of the hull shaking by an IMU (Inertial Measurement Unit) and combination with correction of the distance calculation to the image recognition target can be applied.

[0063] (Other Embodiments) As described above, the embodiments of the present disclosure have been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and design changes and the like within the scope not departing from the gist of the present disclosure are also included.

[0064] (Computer Configuration) FIG. 12 shows a schematic block diagram showing the configuration of the computer according to the above embodiment. The computer 90 includes a processor 91, a main memory 92, a storage 93, and an interface 94. The above-described information processing apparatus 1 is implemented in the computer 90. Then, the operations of the above-described respective processing units are stored in the storage 93 in the form of a program. The processor 91 reads the program from the storage 93 and expands it in the main memory 92, and executes the above processing according to the program. Further, the processor 91 secures a storage area corresponding to each of the above-described storage units in the main memory 92 according to the program.

[0065] The program may be for realizing a part of the functions to be exhibited by the computer 90. For example, the program may exhibit functions in combination with other programs already stored in the storage, or in combination with other programs implemented in other devices. In other embodiments, the computer may include a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) in addition to or instead of the above configuration. Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), FPGA (Field Programmable Gate Array), and the like. In this case, some or all of the functions realized by the processor may be realized by the integrated circuit.

[0066] Examples of the storage 93 include HDD (Hard Disk Drive), SSD (Solid State Drive), magnetic disk, magneto-optical disk, CD-ROM (Compact Disc Read Only Memory), DVD-ROM (Digital Versatile Disc Read Only Memory), semiconductor memory, and the like. The storage 93 may be an internal medium directly connected to the bus of the computer 90, or may be an external medium connected to the computer 90 via the interface 94 or a communication line. Further, when this program is distributed to the computer 90 via a communication line, the receiving computer 90 may expand the program in the main memory 92 and execute the above processing. In at least one embodiment, the storage 93 is a non-transitory tangible storage medium.

[0067] <Appendix> The information processing apparatus 1 described in the above embodiment is understood as follows, for example.

[0068] (1) The information processing apparatus 1 according to the first aspect includes an acquisition unit 101 that acquires a captured image captured by a photographing unit (camera 2) installed on the ship 200, a specifying unit 103 that specifies the closest point to the photographing unit (camera 2) among the targets recognized in the captured image, a first proximity determination unit 104 that determines whether the closest point is within a first proximity region FR that is a region on the ship side in the captured image, and a first angle determination unit 105 that determines whether a first angle θ1 in the captured image regarding the closest point determined to be within the first proximity region FR remains the same or becomes smaller. The first angle θ1 is an angle at which the direction facing the closest point is inclined with respect to the photographing axis PA of the photographing unit (camera 2). According to this aspect and the following aspects, the information processing apparatus 1 determines proximity based on the behavior of the position on the screen of the closest pixel of the target recognized by image recognition without calculating the distance to the target. Therefore, the information processing apparatus 1 can detect a target approaching the ship 200.

[0069] (2) The information processing apparatus 1 according to the second aspect is the information processing apparatus 1 in (1), wherein the first proximity area FR is an area at a predetermined distance from the lower end of the captured image.

[0070] (3) The information processing apparatus 1 according to the third aspect is the information processing apparatus 1 in (1) or (2), wherein the direction facing the nearest point is a direction extending from the center CB at the lower end of the captured image to the nearest point.

[0071] (4) The information processing apparatus 1 according to the fourth aspect is the information processing apparatus 1 in any one of (1) to (3), further comprising a first continuation determination unit 106 that determines that the determination that the first angle θ1 remains the same or becomes smaller continues for a predetermined time or more. According to this aspect, a target having an approaching tendency over a predetermined time can be determined. Therefore, the information processing apparatus 1 can identify a target that requires an alarm.

[0072] (5) The information processing apparatus 1 according to the fifth aspect is the information processing apparatus 1 in any one of (1) to (4), further comprising a second proximity determination unit 107 that determines whether the nearest point is within a second proximity area SR closer to the ship than the first proximity area FR. According to this aspect, a determination regarding the second proximity area SR can be made independently of the determination regarding the first proximity area FR. Therefore, a target that approaches the ship 20 so rapidly that it cannot be caught by the determination regarding the first proximity area FR can be alarmed by the determination regarding the first proximity area FR.

[0073] (6) The information processing apparatus 1 according to the sixth aspect is the information processing apparatus 1 in (5), further comprising a second angle determination unit 108 that determines whether a second angle in the captured image regarding the nearest point determined to be within the second proximity area remains the same or becomes smaller. According to this aspect, the information processing apparatus 1 can determine the approach of a target based on the behavior of the nearest point recognized in the image.

[0074] The information processing apparatus 1 according to the seventh aspect is the information processing apparatus 1 according to the sixth aspect, and further includes a second continuation determination unit 109 that determines that the determination that the second angle remains the same or becomes smaller continues for a predetermined time or longer. According to this aspect, it is possible to determine a target that has a tendency to approach over a predetermined time. Therefore, the information processing apparatus 1 can identify a target that requires an alarm.

[0075] The information processing apparatus 1 according to the eighth aspect is the information processing apparatus 1 according to the fourth aspect, and further includes a second approach determination unit 107 that determines whether the closest point is within a second approach region SR that is closer to the ship than the first approach region FR, a second angle determination unit 108 that determines whether the second angle in the captured image regarding the closest point determined to be within the second approach region SR remains the same or becomes smaller, and a second continuation determination unit 109 that determines that the determination that the second angle remains the same or becomes smaller continues for a predetermined time or longer. The predetermined time in the determination of the second continuation determination unit 109 is set shorter than the predetermined time in the first continuation determination unit 106. According to this aspect, a target that approaches the ship 200 so fast that it cannot be caught up with by the determination regarding the first approach region FR can be alarmed by the determination regarding the first approach region FR.

[0076] The information processing apparatus 1 according to the ninth aspect is the information processing apparatus 1 according to the fourth aspect, and further includes an alarm output unit 110 that outputs an alarm signal regarding the first approach region FR based on the determination result of the first approach determination unit 104, the determination result of the first angle determination unit 105, and the determination result of the first continuation determination unit 106.

[0077] (10) The information processing apparatus 1 according to the tenth aspect is the information processing apparatus 1 in (9), further comprising: a second proximity determination unit 107 that determines whether the nearest point is within a second proximity region SR that is closer to the ship than the first proximity region FR; a second angle determination unit 108 that determines whether a second angle in the captured image regarding the nearest point determined to be within the second proximity region remains the same or decreases; and a second continuation determination unit 109 that determines that the determination that the second angle remains the same or decreases continues for a predetermined time or longer. The alarm output unit 110 further outputs an alarm signal related to the second proximity region SR preferentially over an alarm signal related to the first proximity region FR based on the determination result of the second proximity determination unit 107, the determination result of the second angle determination unit 108, and the determination result of the second continuation determination unit 109. According to this aspect, an alarm related to the second proximity region SR can be prioritized over an alarm related to the first proximity region FR. Therefore, the information processing apparatus 1 can preferentially alarm a target approaching the ship 200.

[0078] (11) The information processing apparatus 1 according to the eleventh aspect is the information processing apparatus 1 in (9) or (10), further comprising a cancellation unit 111 that cancels the alarm signal for each of the image-recognized targets. According to this aspect, an operator can exclude a target from being alarmed when an alarm is not necessarily required, such as for a ship operating at a short distance from the own ship within the same fleet.

[0079] (12) The monitoring apparatus 100 according to the twelfth aspect includes any one of the information processing apparatuses 1 in (1) to (10) and the imaging unit (camera 2). According to this aspect, the monitoring apparatus 100 determines proximity based on the behavior of the position on the screen of the nearest pixel of the image-recognized target without calculating the distance to the target. Therefore, the monitoring apparatus 100 can detect a target approaching the ship 200.

[0080] (13) The information processing method according to the 13th aspect includes steps of: acquiring a captured image captured by an imaging unit (camera 2) installed on a ship 200; specifying a nearest point with respect to the imaging unit (camera 2) on a target recognized by image recognition in the captured image; determining whether the nearest point is within a first approaching region which is a region on the ship side in the captured image; and determining whether a first angle θ1 in the captured image regarding the nearest point determined to be within the first approaching region remains the same or becomes smaller, where the first angle θ1 is an angle at which the direction facing the nearest point is inclined with respect to the imaging axis PA of the imaging unit (camera 2). According to this aspect, the information processing method determines approach by the behavior of the position on the screen of the nearest pixel of the target recognized by image recognition without calculating the distance to the target. Therefore, the information processing method can detect a target approaching the ship 200.

[0081] (14) The program according to the 14th aspect is a program that causes a computer 90 to execute steps of: acquiring a captured image captured by an imaging unit (camera 2) installed on a ship 200; specifying a nearest point with respect to the imaging unit (camera 2) on a target recognized by image recognition in the captured image; determining whether the nearest point is within a first approaching region which is a region on the ship side in the captured image; and determining whether a first angle θ1 in the captured image regarding the nearest point determined to be within the first approaching region remains the same or becomes smaller, where the first angle θ1 is an angle at which the direction facing the nearest point is inclined with respect to the imaging axis PA of the imaging unit (camera 2). According to this aspect, the program determines approach by the behavior of the position on the screen of the nearest pixel of the target recognized by image recognition without calculating the distance to the target. Therefore, the program can detect a target approaching the ship 200.

Explanation of Signs

[0082] 1 Information processing device 2 Camera 2a Captured image 11 Processing device 12 Input device 13 Display device 14 Speaker 90 Computer 91 Processor 92 Main memory 93 Storage 94 Interface 100 Monitoring device 101 Acquisition unit 102 Image recognition unit 103 Identification unit 104 First proximity determination unit 105 First angle determination unit 106 First continuation determination unit 107 Second proximity determination unit 108 Second angle determination unit 109 Second continuation determination unit 110 Alarm output unit 111 Cancellation unit 112 Setting unit 113 Display unit 200 Ship 300 Sea 400 Ship CB Center FL Boundary line FR First proximity area MN Nearest point P1 Number of pixels P2 Number of pixels P3 Number of pixels P4 Number of pixels P5 Number of pixels PA Shooting axis RF1 Recognition frame RL1~18 Angle areas RR1~18 Angle areas SL Boundary line SR Second proximity area X1~5 Fixed intervals XR1~3 Coordinates YR1~3 Coordinates θ1 First angle

Claims

1. An acquisition unit that acquires a captured image captured by an imaging unit installed on a ship; A specifying unit that specifies the closest point to the imaging unit among the targets recognized in the captured image; A first proximity determination unit that determines whether the closest point is within a first proximity region that is a region on the ship side in the captured image; A first angle determination unit that determines whether a first angle in the captured image regarding the closest point determined to be within the first proximity region remains the same or decreases; Comprising: The first angle is an angle at which the direction facing the closest point is inclined with respect to the imaging axis of the imaging unit An information processing apparatus.

2. The first proximity region is a region at a predetermined distance from the lower end of the captured image The information processing apparatus according to claim 1.

3. The direction facing the closest point is a direction extending from the center of the lower end of the captured image to the closest point The information processing apparatus according to claim 1 or 2.

4. A first continuation determination unit that determines that the determination that the first angle remains the same or decreases continues for a predetermined time or more The information processing apparatus according to any one of claims 1 to 3, further comprising.

5. A second proximity determination unit that determines whether the closest point is within a second proximity region that is closer to the ship than the first proximity region The information processing apparatus according to any one of claims 1 to 4, further comprising.

6. A second angle determination unit that determines whether a second angle in the captured image regarding the closest point determined to be within the second proximity region remains the same or decreases The information processing apparatus according to claim 5, further comprising.

7. A second continuation determination unit that determines that the determination that the second angle remains the same or decreases continues for a predetermined time or more The information processing apparatus according to claim 6, further comprising.

8. A second proximity determination unit that determines whether the closest point is within a second proximity region that is closer to the ship than the first proximity region; A second angle determination unit that determines whether a second angle in the captured image regarding the closest point determined to be within the second proximity region remains the same or decreases; A second continuation determination unit that determines that the determination that the second angle remains the same or decreases continues for a predetermined time or more; Further comprising: The predetermined time in the determination of the second continuation determination unit is set shorter than the predetermined time in the first continuation determination unit The information processing apparatus according to claim 4.

9. An alarm output unit that outputs an alarm signal related to the first approach area based on the determination result of the first approach determination unit, the determination result of the first angle determination unit, and the determination result of the first continuation determination unit The information processing apparatus according to claim 4, further comprising the same.

10. A second approach determination unit that determines whether the closest point is within a second approach area that is closer to the ship than the first approach area; A second angle determination unit that determines whether a second angle in the captured image regarding the closest point determined to be within the second approach area remains the same or decreases; A second continuation determination unit that determines that the determination that the second angle remains the same or decreases continues for a predetermined time or longer, and further comprising: Based on the determination result of the second approach determination unit, the determination result of the second angle determination unit, and the determination result of the second continuation determination unit, the alarm output unit preferentially outputs an alarm signal related to the second approach area over the alarm signal related to the first approach area The information processing apparatus according to claim 9.

11. A cancel unit that cancels the alarm signal for each of the image-recognized targets The information processing apparatus according to claim 9 or 10, further comprising the same.

12. An information processing apparatus according to any one of claims 1 to 11, and the imaging unit, and A monitoring device comprising the same.

13. A step of acquiring a captured image captured by a imaging unit installed on a ship; A step of specifying a closest point to the imaging unit in a target recognized by image recognition in the captured image; A step of determining whether the closest point is within a first approach area that is an area on the ship side in the captured image; A step of determining whether a first angle in the captured image regarding the closest point determined to be within the first approach area remains the same or decreases, and including: The first angle is an angle at which the direction facing the closest point is inclined with respect to the imaging axis of the imaging unit An information processing method.

14. A step of acquiring a captured image captured by a imaging unit installed on a ship; A step of specifying a closest point to the imaging unit in a target recognized by image recognition in the captured image; A step of determining whether the closest point is within a first approach area that is an area on the ship side in the captured image; A step of determining whether a first angle in the captured image regarding the closest point determined to be within the first approach area remains the same or decreases, and A program that causes a computer to execute the same The first angle is an angle at which the direction facing the nearest point is inclined with respect to the imaging axis of the imaging unit. Program.

Images