Image analysis system, image analysis method, and program

Abstract

This provides an image analysis system capable of improving the accuracy of detecting the position of horizontal lines contained in an image. [Solution] The image analysis system comprises: an acquisition unit that acquires an overall image including a horizon line; a first estimation unit that calculates the positions of multiple points that are separated from each other in the horizontal direction and represent the position of the horizon line by compressing the overall image to a predetermined size using a first compression ratio and inputting it into a first horizon line position estimation model; an extraction unit that extracts multiple partial images from the overall image based on the positions of the multiple points; a second estimation unit that calculates the positions of the horizon line in the multiple partial images by inputting the multiple partial images into a second horizon line position estimation model, either without compression or by compressing them using a second compression ratio lower than the first compression ratio; a correction unit that corrects the positions of multiple points based on the positions of the horizon line in the multiple partial images; and a calculation unit that calculates an approximation line representing the position of the horizon line in the overall image based on the corrected positions of the multiple points.

Description

Technical Field

[0001] The present invention relates to an image analysis system, an image analysis method, and a program.

Background Art

[0002] Conventionally, a technique for detecting a marine target such as a ship in an image captured by a camera mounted on a ship by image recognition is known.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, an image captured by a camera mounted on a ship includes a horizon, and it is also required to detect the position of the horizon in the image.

[0005] The present invention has been made in view of the above problems, and its main object is to provide an image analysis system, an image analysis method, and a program capable of improving the detection accuracy of the position of the horizon included in an image.

Means for Solving the Problems

[0006] To solve the above problems, an image analysis system according to one aspect of the present invention includes: an acquisition unit that acquires an overall image including a horizontal line; a first estimation unit that calculates the positions of a plurality of points that are separated from each other in the horizontal direction and represent the position of the horizontal line by compressing the overall image to a predetermined size by a first compression ratio and inputting it into a first horizontal line position estimation model; an extraction unit that extracts a plurality of partial images from the overall image based on the positions of the plurality of points; a second estimation unit that calculates the position of the horizontal line in the plurality of partial images by inputting the plurality of partial images either without compression or by compressing them by a second compression ratio lower than the first compression ratio into a second horizontal line position estimation model; a correction unit that corrects the positions of the plurality of points based on the positions of the horizontal line in the plurality of partial images; and a calculation unit that calculates an approximation line representing the position of the horizontal line in the overall image based on the corrected positions of the plurality of points. This makes it possible to improve the detection accuracy of the position of the horizontal line included in the overall image.

[0007] In the above embodiment, a display unit may be further provided to display the overall image on which the approximation lines are arranged. This makes it easier for the user to grasp the position of the horizontal lines included in the overall image.

[0008] In the above embodiment, the correction unit may determine the amount of correction for the position of the point based on the vertical distance between the position of the horizontal line and the position of the point in the partial image. This makes it possible to correct the vertical misalignment between the horizontal line and the point in the partial image.

[0009] In the above embodiment, the first horizontal line position estimation model and the second horizontal line position estimation model may be the same trained model. This makes it possible to calculate the position of the horizontal line in both the overall image and the partial image using the same trained model.

[0010] In the above embodiment, the second estimation unit may calculate the positions of a plurality of points that are horizontally separated from each other, representing the position of the horizontal line in the partial image, as the position of the horizontal line in the partial image. This makes it possible to calculate the position of the horizontal line in the partial image, similar to the case of the whole image.

[0011] In the above embodiment, the correction unit may determine the amount of correction for the position of the point based on the vertical distance between the vertical average position of the plurality of points calculated in the partial image and the position of the point used as the reference for extracting the partial image. This makes it possible to correct the vertical misalignment between the horizontal line and the point in the partial image.

[0012] In the above embodiment, the extraction unit may extract a partial image of the same size as the predetermined size into which the overall image is compressed. This makes it possible to input the partial image into the horizontal position estimation model without compressing it.

[0013] In the above embodiment, the calculation unit may calculate the reliability of the approximation line based on the error of the approximation line. This makes it possible to calculate the reliability of the approximation line that represents the position of the horizontal line.

[0014] In the above embodiment, the first horizontal line position estimation model or the second horizontal line position estimation model may be a model trained by machine learning. This makes it possible to calculate the position of the horizontal line using a model trained by machine learning.

[0015] Furthermore, in another embodiment of the present invention, an image analysis method acquires an overall image including a horizontal line, compresses the overall image to a predetermined size using a first compression ratio and inputs it into a first horizontal line position estimation model to calculate the positions of a plurality of points that are horizontally separated from each other and represent the position of the horizontal line, extracts a plurality of partial images from the overall image using the positions of the plurality of points as references, inputs the plurality of partial images into a second horizontal line position estimation model either without compression or by compressing them using a second compression ratio lower than the first compression ratio, calculates the position of the horizontal line in each of the plurality of partial images, corrects the positions of the plurality of points based on the positions of the horizontal line in the plurality of partial images, and calculates an approximation line representing the position of the horizontal line in the overall image based on the corrected positions of the plurality of points. This makes it possible to improve the detection accuracy of the position of the horizontal line included in the overall image.

[0016] Furthermore, a program in another aspect of the present invention causes a computer to perform the following actions: acquire an overall image including a horizon line; compress the overall image to a predetermined size using a first compression ratio and input it into a first horizon line position estimation model to calculate the positions of a plurality of points that are horizontally separated from each other and represent the position of the horizon line; extract a plurality of partial images from the overall image using the positions of the plurality of points as references; input the plurality of partial images into a second horizon line position estimation model, either without compression or by compressing them using a second compression ratio lower than the first compression ratio; correct the positions of the plurality of points based on the positions of the horizon line in the plurality of partial images; and calculate an approximation line representing the position of the horizon line in the overall image based on the corrected positions of the plurality of points. This makes it possible to improve the detection accuracy of the position of the horizon line included in the overall image. [Effects of the Invention]

[0017] According to the present invention, it is possible to improve the accuracy of detecting the position of horizontal lines included in an image. [Brief explanation of the drawing]

[0018] [Figure 1] It is a diagram showing an example of a system. [Figure 2] It is a diagram showing an example of an image. [Figure 3] It is a diagram showing an example of a control unit. [Figure 4] It is a diagram showing an example of display. [Figure 5] It is a diagram showing an example of distance calculation. [Figure 6] It is a diagram showing an example of distance calculation. [Figure 7] It is a diagram showing an example of horizontal line detection. [Figure 8] It is a diagram showing an example of horizontal line detection. [Figure 9] It is a diagram showing an example of horizontal line detection. [Figure 10] It is a diagram showing an example of horizontal line detection. [Figure 11] It is a diagram showing an example of horizontal line detection. [Figure 12] It is a diagram showing an example of horizontal line detection.

Mode for Carrying Out the Invention

[0019] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In this specification and each drawing, elements that are the same as those described above with respect to the already shown drawings may be denoted by the same reference numerals, and detailed descriptions may be omitted as appropriate.

[0020] [System Overview] FIG. 1 is a block diagram showing a configuration example of a navigation support system 100. The navigation support system 100 is a system mounted on a ship. The navigation support system 100 is an image analysis system of the present embodiment. In the following description, a ship on which the navigation support system 100 is mounted is also referred to as "own ship", and other ships are also referred to as "other ships".

[0021] The navigation support system 100 comprises a control unit 1, a display unit 2, a radar 3, an AIS 4, a camera 5, a GNSS receiver 6, a compass 7, an ECDIS 8, a wireless communication unit 9, and a ship handling control unit 10. These devices are connected to a network N, such as a LAN, and are capable of network communication with each other.

[0022] The control unit 1 includes a computer comprising a CPU, RAM, ROM, non-volatile memory, and an input / output interface. The CPU of the navigation support device 1 performs information processing according to a program loaded from the ROM or non-volatile memory into the RAM.

[0023] The program may be supplied via an information storage medium such as an optical disc or memory card, or via a communication network such as the Internet or LAN.

[0024] The display unit 2 displays the display image generated by the control unit 1. The display unit 2 can also display radar images, camera images, or electronic charts.

[0025] The display unit 2 is, for example, a display device with a touch sensor, a so-called touch panel, which detects the position indicated on the screen by the user's finger or the like. However, it is not limited to this, and the indicated position may also be input by a pointing device such as a trackball.

[0026] Radar 3 emits radio waves around the ship and receives the reflected waves, generating echo data based on the received signals. Radar 3 also identifies targets from the echo data and generates TT data (Target Tracking Data) representing the target's position and speed.

[0027] The Automatic Identification System (AIS) 4 receives AIS data from other vessels or shore-based control systems in the vicinity of the vessel. While AIS is not the only option, a VHF Data Exchange System (VDES) may also be used. AIS data includes the identification code, name, position, course, speed, vessel type, length, and destination of other vessels.

[0028] Camera 5 is a digital camera that captures images of the outside from the ship and generates image data. Camera 5 is installed, for example, on the ship's bridge, facing the bow. Camera 5 is a so-called PTZ camera, which has, for example, pan-tilt and optical zoom functions.

[0029] Camera 5 may include an image recognition unit that estimates the position and type of maritime targets, such as ships, contained in the captured image using an object detection model. The image recognition unit is not limited to camera 5 and may be implemented in other devices such as control unit 1.

[0030] The GNSS receiver 6 detects the ship's position based on radio waves received from the GNSS (Global Navigation Satellite System). The compass 7 detects the ship's heading. The compass 7 is, for example, a GPS compass or a gyrocompass.

[0031] The ECDIS (Electronic Chart Display and Information System) 8 obtains the ship's position from the GNSS receiver 6 and displays the ship's position on the electronic chart. The ECDIS 9 also displays the ship's planned route on the electronic chart. A GNSS plotter may be used instead of the ECDIS.

[0032] The wireless communication unit 9 includes wireless equipment for realizing satellite communications. The wireless communication unit 9 also includes wireless equipment for realizing ship-to-shore or ship-to-ship wireless communications using, for example, ultra-high frequency, very high frequency, short frequency, or medium-high frequency.

[0033] The ship steering control unit 10 is a control device for achieving autonomous navigation and controls the ship's steering gear. The ship steering control unit 10 may also control the ship's engines.

[0034] In this embodiment, the control unit 1 and the display unit 2 are independent devices, but the control unit 1 and the display unit 2 may be an integrated device.

[0035] Furthermore, the control unit 1 is not limited to an independent device, but may be integrated with other devices such as ECDIS8. In other words, some or all of the functions of the control unit 1 may be implemented by other devices.

[0036] Furthermore, the display unit 2 is not limited to an independent device; the display unit of another device, such as ECDIS8, may be used as the display unit 2 that displays the display image generated by the control unit 1.

[0037] Furthermore, the control unit 1 and the display unit 2 may be installed, for example, at a land-based control center and used to monitor vessels navigating within a controlled area.

[0038] Figure 2 shows an example of an image WG generated by the camera 5 mounted on the ship. In the following explanation, the image WG will also be referred to as the "overall image WG". The overall image WG includes objects TG at sea such as other ships, and the horizon SH.

[0039] The horizon line SH is a line that crosses horizontally within the overall image WG. The horizon line SH can be, for example, a straight line or a curved line. The "horizon line" is the line that forms the boundary between the water surface and the sky, but it is not limited to this and may also include, for example, the boundary between the water surface and distant land.

[0040] Figure 3 is a block diagram showing an example configuration of the control unit 1. The control unit 1 comprises an image acquisition unit 11, a horizon line detection unit 12, a target detection unit 13, a distance calculation unit 14, and a display control unit 15. These functional units are realized by the CPU of the control unit 1 executing information processing according to a program.

[0041] The image acquisition unit 11 acquires the overall image WG generated by the camera 5. The horizon detection unit 12 detects the horizon SH included in the overall image WG. The target detection unit 13 detects the target TG included in the overall image WG.

[0042] The distance calculation unit 14 calculates the distance from the vessel to the target TG based on the recognition results of the horizon SH and the target TG. Specific examples of the horizon detection unit 12, the target detection unit 13, and the distance calculation unit 14 will be described later.

[0043] The display control unit 15 displays the overall image WG on the display unit 2, as well as the recognition results of the horizontal line SH and the target TG.

[0044] Figure 4 shows an example of screen display by the display unit 2. The screen of the display unit 2 displays the overall image WG, and on the overall image WG, an approximation line HL representing the position of the horizontal line SH and a boundary box BB surrounding the target TG are displayed. The approximation line HL can be, for example, a straight line or a curve.

[0045] Furthermore, the screen of display unit 2 may display information such as the distance from the vessel to the target TG. Note that the frame indicating the partial image PG and the arrow indicating the distance LT in the figure are for the purpose of explaining the distance calculation described later and do not need to be displayed on the screen.

[0046] [Distance Calculation] Figure 5 is a diagram illustrating a specific configuration and operation example related to calculating the distance from the vessel to the target TG.

[0047] The target detection unit 13 recognizes the region of the target TG in the overall image WG by inputting the overall image WG into the region recognition model. The target detection unit 13 is an example of the first image recognition unit. The region recognition model is a pre-trained model obtained through machine learning. However, it is not limited to this, and the region recognition model may also be a rule-based computation model.

[0048] The region recognition model is, for example, an object detection model such as YOLO (You Only Look Once) or SSD (Single Shot MultiBox Detector), which outputs a bounding box surrounding objects contained in the image. However, the region recognition model is not limited to these; it may also be a region segmentation model such as Semantic Segmentation or Instance Segmentation.

[0049] In this embodiment, the area of ​​the target TG is the boundary box BB surrounding the target TG (see Figure 4). The target detection unit 13 outputs the position (Bbox position), size (Bbox size), and shape (Bbox shape) of the boundary box BB as recognition results.

[0050] The position of the boundary box BB represents the distance SH from the horizontal line SH. Specifically, the position of the boundary box BB represents the vertical distance LT between the lower end of the boundary box BB and the horizontal line SH (see Figure 4). The lower end of the boundary box BB corresponds to the waterline of the target TG.

[0051] The size of the boundary box BB is its area. However, it is not limited to this; the size of the boundary box BB may also be its height or width. The shape of the boundary box BB is the ratio of its height to its width.

[0052] Furthermore, the target detection unit 13 outputs the recognition result to the partial image extraction unit 24. Based on the recognition result from the target detection unit 13, the partial image extraction unit 24 extracts a partial image PG containing the target TG from the overall image WG. As shown in Figure 4, the partial image PG is an image that includes the bounding box BB and covers an area larger than the bounding box BB, and is cropped so that its outer edge is a predetermined distance away from the bounding box BB.

[0053] The distance calculation unit 14 estimates the distance from the vessel to the target TG by inputting the recognition result from the target detection unit 13 and the partial image PG into the distance estimation model. The distance calculation unit 14 is an example of a second image recognition unit. The distance estimation model is also a machine learning-trained model. However, it is not limited to this, and the distance estimation model may also be a rule-based calculation model.

[0054] Specifically, the distance calculation unit 14 comprises an estimation main unit 21 and a type recognition unit 22. The estimation main unit 21 is composed of, for example, a neural network. However, it is not limited to this, and the estimation main unit 21 may be composed of a regression model. The type recognition unit 22 is composed of, for example, a convolutional neural network.

[0055] The type recognition unit 22 recognizes the type of target TG from the partial image PG and outputs the recognition result to the estimation main unit 21. The type of target TG is, for example, a type of vessel such as a pleasure boat, fishing boat, merchant ship, or tanker.

[0056] The estimation unit 21 calculates the distance from the ship to the target TG by inputting the position, size, and shape of the boundary box BB recognized by the target detection unit 13 and the type of target TG recognized by the type recognition unit 22 into a neural network.

[0057] The distance estimation model is generated by machine learning, using the position, size, and shape of the training bounding box, as well as partial images, as input data, and the corresponding distance from the ship to the target as training data.

[0058] Figure 6 is a flowchart showing an example of the procedure for calculating the distance from the vessel to the target TG, which is part of the image analysis method implemented in the navigation support system 100. The control unit 1 executes the information processing shown in the figure according to the program.

[0059] First, the control unit 1 acquires the overall image WG generated by the camera 5 (S11, processing as the image acquisition unit 11).

[0060] Next, the control unit 1 detects the horizontal line SH included in the overall image WG (S12, processing as the horizontal line detection unit 12).

[0061] Next, the control unit 1 inputs the overall image WG into the region recognition model and outputs the recognition result of the region of the target TG and the partial image PG that includes the target TG (S13, S14, processing as the target detection unit 13).

[0062] Next, the control unit 1 inputs the recognition result of the area of ​​the target TG and the partial image PG containing the target TG into the distance estimation model to calculate the distance from the ship to the target TG (S15, S16, processing as the distance calculation unit 14).

[0063] With the above steps completed, the process for calculating the distance from the vessel to the target TG is finished.

[0064] [Horizontal line detection] Figure 7 is a block diagram showing a specific configuration example of the horizontal line detection unit 12. Figure 8 is a diagram illustrating a specific operation example of the horizontal line detection unit 12. The horizontal line detection unit 12 comprises a first estimation unit 31, a partial image extraction unit 32, a second estimation unit 33, a point position correction unit 34, and an approximation line calculation unit 35.

[0065] The first estimation unit 31 compresses the overall image WG to a predetermined size and inputs it into the first horizontal line position estimation model 31m to calculate the positions of multiple detection points PT that are horizontally separated from each other and represent the position of the horizontal line SH. The number of detection points PT is, for example, 4 or 5, but is not limited to these.

[0066] The first horizontal position estimation model 31m is a pre-trained model obtained through machine learning. However, the first horizontal position estimation model 31m may also be a rule-based computation model.

[0067] Specifically, the first horizontal position estimation model 31m is composed of, for example, a convolutional neural network. The output layer of the first horizontal position estimation model 31m is configured to output the coordinates of multiple detection points PT.

[0068] This first horizontal position estimation model 31m is generated by machine learning, using training images as input data and the coordinates of points annotated with the horizontal positions contained in the training images as training data.

[0069] Incidentally, since the first horizontal line position estimation model 31m receives a compressed overall image WG as input, the positions of multiple detection points PT may not coincide with the horizontal line SH. If an approximation line HL is drawn based on such detection points PT, there is a risk that the horizontal line SH and the approximation line HL will differ (see Figure 9).

[0070] Therefore, in this embodiment, the error between the horizontal line SH and the approximation line HL is suppressed by correcting the position of the detection point PT, as described below.

[0071] The partial image extraction unit 32 extracts multiple partial images EG from the overall image WG, using the positions of multiple detection points PT as references. Specifically, the partial image extraction unit 32 extracts partial images EG such that the detection points PT are located in the center.

[0072] The second estimation unit 33 inputs each of the multiple partial images EG into the second horizontal position estimation model 33m, thereby calculating the position of the horizontal line SH in each of the multiple partial images EG. The second horizontal position estimation model 33m is also a model trained using machine learning.

[0073] In this embodiment, the first horizontal position estimation model 31m and the second horizontal position estimation model 33m are the same trained model.

[0074] Therefore, the second estimation unit 33, similar to the first estimation unit 31, calculates the positions of multiple detection points SP that are horizontally separated from each other, representing the position of the horizontal line SH in the partial image EG, as shown in Figure 10. The vertical average position of these detection points SP is taken as the position HE of the horizontal line SH in the partial image EG.

[0075] The second estimation unit 33 inputs multiple partial images EG into the second horizontal position estimation model 33m without compressing them. The partial images EG are extracted by the partial image extraction unit 32 at the same size as the overall image WG compressed by the first estimation unit 31.

[0076] The second estimation unit 33 may also compress multiple partial images EG using a compression ratio (second compression ratio) lower than the compression ratio (first compression ratio) used by the first estimation unit 31 to compress the overall image WG, and input these into the second horizontal position estimation model 33m.

[0077] In this way, by inputting the partial image EG into the second horizontal position estimation model 33m without compression or with a lower compression ratio, the position HE of the horizontal line SH in the partial image EG can be determined with high accuracy.

[0078] The point position correction unit 34 corrects the positions of multiple points PT based on the position HE of the horizontal line SH in multiple partial images EG.

[0079] Specifically, as shown in Figure 10, the point position correction unit 34 determines the correction amount for the position of the detected point PT based on the vertical distance RV between the position HE of the horizontal line SH in the partial image EG and the position HW of the detected point PT, which is calculated by the first estimation unit 31 and used as the reference for extracting the partial image EG.

[0080] The point position correction unit 34 determines the correction amount for the position HW of the detected point PT so that the distance RV between the position HW of the detected point PT and the position HE of the horizontal line SH becomes 0 or approaches 0.

[0081] For example, if the position HW of the detection point PT is located above the position HE of the horizontal line SH, the position HW of the detection point PT is corrected downwards, and if the position HW of the detection point PT is located below the position HE of the horizontal line SH, the position HW of the detection point PT is corrected upwards.

[0082] The approximation line calculation unit 35 calculates an approximation line HL representing the position of the horizontal line SH in the overall image WG based on the positions of the multiple corrected detection points PTr. The approximation line calculation unit 35 approximates the approximation line HL as, for example, a quadratic curve. However, it is not limited to this, and the approximation line calculation unit 35 may also approximate the approximation line HL as a straight line.

[0083] Furthermore, the approximation line calculation unit 35 calculates the reliability of the approximation line HL based on the error of the approximation line HL. The error is, for example, the sum of squared residuals, and the reliability is set so that it approaches 1 as the error is small and approaches 0 as the error is large. If the reliability is below a predetermined level, the estimation of the position of the horizontal line SH may be redone.

[0084] As described above, by calculating the approximation line HL based on the positions of multiple corrected detection points PTr, it becomes possible to reduce the error between the horizontal line SH and the approximation line HL in the overall image WG, and to make them match, as shown in Figure 11.

[0085] Figure 12 is a flowchart showing an example of the procedure for estimating the position of the horizon line HL, which is part of the image analysis method implemented in the navigation support system 100. The control unit 1 executes the information processing shown in the figure according to the program.

[0086] First, the control unit 1 acquires the overall image WG generated by the camera 5 (S21, processing as the image acquisition unit 11).

[0087] Next, the control unit 1 compresses the overall image WG to a predetermined size and inputs the compressed overall image WG into the first horizontal line position estimation model 31m to calculate the positions of multiple detection points PT that represent the position of the horizontal line SH (S22-S24, processing as the first estimation unit 31).

[0088] Next, the control unit 1 extracts multiple partial images EG from the overall image WG, using the positions of the multiple detection points PT as references (S25, processing as a partial image extraction unit 32).

[0089] Next, the control unit 1 inputs the partial image EG to the second horizontal line position estimation model 33m to calculate the position of the horizontal line SH in the partial image EG (S26-S27, processing as the second estimation unit 33), and corrects the position of the detection point PT based on the calculated position of the horizontal line SH (S28, processing as the point position correction unit 34).

[0090] These processes S26-S28 are performed for all partial images EG (S29).

[0091] Subsequently, the control unit 1 calculates an approximate line HL representing the position of the horizontal line SH in the overall image WG based on the corrected positions of the multiple detection points PTr (S30, processing as the approximate line calculation unit 35).

[0092] This completes the process for estimating the position of the horizontal line HL.

[0093] Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above, and various modifications are of course possible for those skilled in the art.

[0094] The following lists representative embodiments of the present invention.

[0095] (1) An acquisition unit that acquires the entire image including the horizon, A first estimation unit calculates the positions of multiple points that are separated horizontally and represent the position of the horizontal line by compressing the overall image to a predetermined size using a first compression ratio and inputting it into a first horizontal line position estimation model. An extraction unit that extracts multiple partial images from the overall image based on the positions of the multiple points mentioned above, A second estimation unit calculates the position of the horizontal line in each of the multiple partial images by inputting each of the multiple partial images into a second horizontal line position estimation model, either without compression or by compressing each of them with a second compression ratio lower than the first compression ratio. A correction unit that corrects the positions of the multiple points based on the positions of the horizontal line in the multiple partial images, A calculation unit calculates an approximate line representing the position of the horizontal line in the overall image based on the corrected positions of the plurality of points, An image analysis system equipped with the following features.

[0096] (2) The system further includes a display unit that displays the overall image on which the aforementioned approximation lines are arranged. (1) The image analysis system described above.

[0097] (3) The correction unit determines the amount of correction for the position of the point based on the vertical distance between the position of the horizontal line and the position of the point in the partial image. The image analysis system described in (1) or (2).

[0098] (4) The first horizontal position estimation model and the second horizontal position estimation model are the same trained model. An image analysis system as described in any of (1) to (3).

[0099] (5) The second estimation unit calculates the positions of a plurality of points that are separated from each other in the horizontal direction, which represent the position of the horizontal line in the partial image, as the position of the horizontal line in the partial image. (4) The image analysis system described above.

[0100] (6) The correction unit determines the amount of correction for the position of the point based on the vertical distance between the average vertical position of the plurality of points calculated in the partial image and the position of the point used as the reference for extracting the partial image. (5) The image analysis system described above.

[0101] (7) The extraction unit extracts a partial image of the same size as the predetermined size into which the overall image is compressed. An image analysis system as described in any of (4) to (6).

[0102] (8) The calculation unit calculates the reliability of the approximation line based on the error of the approximation line. An image analysis system as described in any of (1) through (7).

[0103] (9) The first horizontal position estimation model or the second horizontal position estimation model is a model trained using machine learning. An image analysis system as described in any of (1) through (8).

[0104] (10) Obtain the entire image including the horizon, By compressing the aforementioned overall image to a predetermined size using a first compression ratio and inputting it into the first horizontal line position estimation model, the positions of multiple points that are horizontally separated from each other and represent the position of the horizontal line are calculated. Using the positions of the aforementioned multiple points as references, multiple partial images are extracted from the overall image. The multiple partial images are input into the second horizontal line position estimation model either without compression or by compressing them at a second compression ratio lower than the first compression ratio, thereby calculating the position of the horizontal line in each of the multiple partial images. Based on the position of the horizontal line in the aforementioned multiple partial images, the positions of the aforementioned multiple points are corrected accordingly. Based on the corrected positions of the plurality of points, an approximation line representing the position of the horizontal line in the overall image is calculated. Image analysis methods.

[0105] (11) To obtain an overall image including the horizon, The overall image is compressed to a predetermined size using a first compression ratio and input into a first horizontal line position estimation model to calculate the positions of multiple points that are horizontally separated from each other and represent the position of the horizontal line. Extracting multiple partial images from the overall image based on the positions of the multiple points mentioned above, The positions of the horizontal lines in each of the multiple partial images are calculated by inputting each of the multiple partial images into a second horizontal line position estimation model, either without compression or by compressing each of them with a second compression ratio lower than the first compression ratio. Based on the position of the horizontal line in the plurality of partial images, the position of each of the plurality of points is corrected, and Based on the corrected positions of the plurality of points, calculate an approximation line representing the position of the horizontal line in the overall image. A program that causes a computer to execute something. [Explanation of Symbols]

[0106] 1 Control Unit, 2 Display Unit, 3 Radar, 4 AIS, 5 Camera, 6 GNSS Receiver, 7 Compass, 8 ECDIS, 9 Wireless Communication Unit, 10 Ship Maneuvering Control Unit, 11 Image Acquisition Unit, 12 Horizon Detection Unit, 13 Target Detection Unit, 14 Distance Calculation Unit, 15 Display Control Unit, 21 Estimation Main Unit, 22 Type Recognition Unit, 24 Partial Image Extraction Unit, 31 First Estimation Unit, 32 Partial Image Extraction Unit, 33 Second Estimation Unit, 34 Point Position Correction Unit, 35 Approximation Line Calculation Unit

Claims

1. An acquisition unit that acquires the entire image including the horizon, A first estimation unit calculates the positions of multiple points that are separated horizontally and represent the position of the horizontal line by compressing the overall image to a predetermined size using a first compression ratio and inputting it into a first horizontal line position estimation model. An extraction unit that extracts multiple partial images from the overall image based on the positions of the multiple points mentioned above, A second estimation unit calculates the position of the horizontal line in each of the multiple partial images by inputting each of the multiple partial images into a second horizontal line position estimation model, either without compression or by compressing each of them with a second compression ratio lower than the first compression ratio. A correction unit that corrects the positions of the multiple points based on the positions of the horizontal line in the multiple partial images, A calculation unit calculates an approximate line representing the position of the horizontal line in the overall image based on the corrected positions of the plurality of points, An image analysis system equipped with the following features.

2. The system further includes a display unit that displays the overall image on which the aforementioned approximation lines are arranged. The image analysis system according to claim 1.

3. The correction unit determines the amount of correction for the position of the point based on the vertical distance between the position of the horizontal line and the position of the point in the partial image. The image analysis system according to claim 1.

4. The first horizontal position estimation model and the second horizontal position estimation model are the same trained model. The image analysis system according to claim 1.

5. The second estimation unit calculates the positions of a plurality of points that are horizontally separated from each other, which represent the position of the horizontal line in the partial image, as the position of the horizontal line in the partial image. The image analysis system according to claim 4.

6. The correction unit determines the amount of correction for the position of the point based on the vertical distance between the average vertical position of the plurality of points calculated in the partial image and the position of the point used as the reference for extracting the partial image. The image analysis system according to claim 5.

7. The extraction unit extracts a partial image of the same size as the predetermined size into which the overall image is compressed. The image analysis system according to claim 4.

8. The calculation unit calculates the reliability of the approximation line based on the error of the approximation line. The image analysis system according to claim 1.

9. The first horizontal position estimation model or the second horizontal position estimation model is a model trained using machine learning. The image analysis system according to claim 1.

10. Obtain the entire image including the horizon, By compressing the aforementioned overall image to a predetermined size using a first compression ratio and inputting it into the first horizontal line position estimation model, the positions of multiple points that are horizontally separated from each other and represent the position of the horizontal line are calculated. Using the positions of the aforementioned multiple points as references, multiple partial images are extracted from the overall image. The plurality of partial images are input into the second horizontal line position estimation model either without compression or by compressing them using a second compression ratio lower than the first compression ratio, thereby calculating the position of the horizontal line in each of the plurality of partial images. Based on the position of the horizontal line in the aforementioned multiple partial images, the positions of the aforementioned multiple points are corrected accordingly. Based on the corrected positions of the plurality of points, an approximation line representing the position of the horizontal line in the overall image is calculated. Image analysis methods.

11. To obtain an overall image including the horizon, The overall image is compressed to a predetermined size using a first compression ratio and input into a first horizontal line position estimation model to calculate the positions of multiple points that are horizontally separated from each other and represent the position of the horizontal line. Extracting multiple partial images from the overall image based on the positions of the multiple points mentioned above, The positions of the horizontal lines in the plurality of partial images are calculated by inputting each of the aforementioned partial images into a second horizontal line position estimation model, either without compression or by compressing each of them with a second compression ratio lower than the first compression ratio. Based on the position of the horizontal line in the plurality of partial images, the positions of the plurality of points are corrected accordingly, and Based on the corrected positions of the plurality of points, calculate an approximation line representing the position of the horizontal line in the overall image. A program that causes a computer to execute something.

Images