Information processing apparatus, information processing method, and program

Abstract

Provided are an information processing apparatus, an information processing method, and a program capable of enhancing the safety during ship navigation in a new way. 【Solution means】The information processing apparatus according to the present disclosure includes a reception process of receiving detection target information including position information of a predetermined detection target detected by a detection unit of each first ship from a plurality of first ships that are sailing, a storage process of aggregating and storing the received detection target information, and a transmission process of transmitting at least any one of the received detection target information and predetermined output data generated under predetermined generation conditions using the detection target information to a predetermined second ship. The information processing apparatus includes a control unit that executes the processes.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Conventionally, in order to enhance the safety when a ship is navigating, a system such as AIS (Automatic Identification System) is known as a means for sharing the position information of one's own ship with other ships. However, among small ships such as fishing boats, many ships do not have AIS installed, and there are problems regarding information sharing.

[0003] In contrast, for example, Patent Document 1 proposes a system that transmits the position information of one's own ship to other ships using a device equipped with a GPS function so that even ships without AIS can share the position information of their own ships.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, it is realistically difficult to equip all ships with devices equipped with AIS or GPS.

[0006] Therefore, the present disclosure has been made in view of the above problems, and an object thereof is to provide an information processing apparatus, an information processing method, and a program capable of enhancing the safety during ship navigation by a new method.

Means for Solving the Problems

[0007] According to the present disclosure, a receiving process of receiving detection target information including position information of a predetermined detection target detected by a detection unit of each first ship from a plurality of first ships navigating, a storage process of aggregating and storing the received detection target information, and a transmission process of transmitting at least any one of the received detection target information and predetermined output data generated under generation conditions predetermined using the detection target information to a predetermined second ship, and an information processing apparatus including a control unit that executes the processes is provided.

Effect of the Invention

[0008] According to the present disclosure, it is possible to provide an information processing apparatus, an information processing method, and a program capable of enhancing the safety during ship navigation in a new method.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Mode for Carrying Out the Invention

[0010] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the present specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

[0011] FIG. 1 is a conceptual diagram of a system using the information processing apparatus of the present embodiment. In FIG. 1, ships A1 and A2 in navigation that can communicate with the information processing apparatus 100, ship B that cannot communicate with the information processing apparatus 100, floating object C, and person D are illustrated.

[0012] The information processing apparatus 100 shown in FIG. 1 functions as a management server (for example, a cloud server). As shown in FIG. 2, the information processing apparatus 100 includes a control unit 120, a storage unit 130, an input unit 140, an output unit 150, and a communication unit 160. In this example, the control unit 120, the storage unit 130, the input unit 140, the output unit 150, and the communication unit 160 are configured to be implemented in an information processing apparatus (computer), but the present invention is not limited thereto, and they may be divided into a plurality of apparatuses to form a system, or each unit (at least a part thereof) may be provided independently.

[0013] The control unit 120 includes a processor (arithmetic unit) such as a CPU, and is configured to be able to execute various information processes based on programs stored in the storage unit 130.

[0014] The storage unit 130 can store various information stored in advance, information generated by the control unit 120, operation information of a user input via the input unit 140, information received from ships, external devices, etc. via the communication unit 160, and the like. The storage unit 130 includes a main storage device configured of a volatile storage device such as a DRAM (Dynamic Random Access Memory), and an auxiliary storage device configured of a non-volatile storage device such as a flash memory or an HDD (Hard Disc Drive).

[0015] The storage unit 130 can store two-dimensional or three-dimensional map data (map information) corresponding to the real space, received information, various types of information generated based on the received information, various types of information regarding the detection target, condition information for various processes executed by the control unit, and the like. The various processes can store information regarding various conditions such as, for example, the matching process, position prediction process, target estimation process, etc., which will be described later. The storage unit 130 can store information regarding the shapes (such as three-dimensional model data) of various ships, various parameters indicating characteristics (characteristics regarding output, speed, turning amount), navigation history information, navigation route information, tide information, and the like.

[0016] The input unit 140 receives input information based on operations from the user or the like. The input unit 140 is composed of a mechanical button, switch, operation lever, touch panel, keyboard, mouse, and the like. The input unit 140 may be provided with a microphone capable of voice input or the like.

[0017] The output unit 150 outputs various types of information by means of an image (video), voice, and the like. The output unit 150 may be provided with, for example, a display unit such as a liquid crystal monitor or a touch panel for displaying an image, a voice output unit such as a speaker for displaying voice, a vibration generating unit (vibration device) for generating vibration, and the like.

[0018] The communication unit 160 is connected to a network such as the Internet or wireless communication, and receives and transmits data from ships, external devices (management servers, control devices, terminal devices used by users, etc.).

[0019] Note that the control unit 120 can estimate the position, type, size, orientation, etc. of an obstacle by, for example, analyzing the image data acquired by the camera (object estimation process). The control unit 120 may estimate the type of the obstacle by selecting any one from among the candidates of the types of obstacles stored in advance in the storage unit 130. The information on the type of the obstacle may include type information such as ships, specific ship classifications (fishing boats, water taxis, small multi-purpose boats, pleasure fishing boats, passenger ships, traffic boats, work boats, fire boats / guard boats, pleasure yachts, pleasure motor boats, special work boats, tankers, cargo ships, etc.), marine structures, people (including swimming, surfing, yachts, paddle boards, etc.), rocks, shores, driftwood, seaweed, marine organisms, etc. The information on the type of the obstacle is stored in association with the information on the size (area, height).

[0020] In the obstacle information estimation process, the control unit 120 may estimate the position, type, size, orientation, etc. of the obstacle by using a machine learning technique using a learning model learned in advance. When the control unit 120 estimates the position of the obstacle based on the image data acquired by the camera, after estimating the three-dimensional position (coordinates) of the obstacle in the camera coordinate system, based on the information on the position (coordinates) and the shooting direction of the camera, the camera coordinate system can be converted into the world coordinate system by coordinate conversion processing, and the position (coordinates) of the obstacle on the world coordinate system can be calculated. The control unit 120 can estimate the bow direction of the ship as an obstacle, the direction of the face of a person (front direction), the direction of an object used such as a surfboard (traveling direction) by analyzing the image data from the camera or analyzing three-dimensional data (point cloud data, model data, etc.).

[0021] The control unit 120 of the information processing device can execute a reception process of receiving detection target information including position information of a predetermined detection target detected by a detection unit of each of a plurality of first ships that are sailing, a storage process of aggregating and storing the received detection target information, and a transmission process of transmitting at least any one of the received detection target information and predetermined output data generated under generation conditions predetermined using the detection target information to a predetermined second ship. According to the present embodiment, the information detected by the first ship can be aggregated, stored, and transmitted to another second ship. As a result, for example, the position information of a ship that does not have AIS or GPS can be shared from the first ship to the second ship via the information processing device, so that the safety during navigation can be improved. Note that the ship A1 in FIG. 1 may function as the first ship and the ship A2 may function as the second ship, or vice versa. Also, the ship A1 and the ship A2 may function as the first ship, and the ship A1 and the ship A2 may also function as the second ship. Further, at least any one of the ship A1 and the ship A2 may function as the first ship, and another ship (not shown) may be the second ship. In any case, if detection targets such as the ship B, the floating object C, and the person D can be detected by any one of the first ships, they can be shared with other second ships that have not detected them, so that the safety during navigation can be improved.

[0022] For example, as shown in FIG. 3, the control unit 120 executes a reception process of acquiring detection target information from the first ship (S101). The reception process only needs to be able to receive the detection target information from at least one first ship. Also, as long as the control unit 120 of the information processing device does not stop the reception function, the information processing device is continuously maintained in a receivable state.

[0023] Next, the control unit 120 aggregates the detection target information and executes storage processing (S102). Aggregating and storing means collectively managing the information obtained from a plurality of different ships in a unified manner. The storage processing is performed every time reception processing is performed. The control unit 120 may assign unique identification information (such as a number or an ID including characters) to each detection target and store and accumulate the information in a detection target information table. Alternatively, map information in which the position (coordinates) of the detection target is reflected in the pre-stored map information may be stored.

[0024] FIG. 4 shows an example of a detection target information table. In the example of FIG. 4, as detection target information, information on the type, position, attitude, moving speed, moving direction, size, detection time (the time when the first ship detected), reception time (the time when the information processing device received), and transmitting ship (the ship that detected the information and transmitted it to the information processing device) of the detection target is aggregated and managed in association with the identification information. Note that the detection target information may include other information. For example, image information of each detection target captured by the camera of the first ship may be stored in association. Further, the detection target information received by the information processing device only needs to include at least position information. For example, the attitude, moving speed, etc. may be unknown. In the example of FIG. 4, for example, regarding the "driftwood" with the identification information "B0003" transmitted from the ship "A1", information regarding the attitude and movement cannot be detected and is unknown (indicated by "-" in the figure). Also, the information on the attitude and moving direction is expressed as a numerical value from 0° to 360° with one of the directions of east, west, south, and north being 0°, but it is not limited to this. In the illustrated example, the size is expressed in three levels: "large", "medium", and "small", but it is not limited to this. It may be two levels or less or four levels or more, and the levels may be expressed numerically, or the width (maximum width, diameter, etc.), height, area, volume, etc. may be expressed numerically. These information can be estimated by a known method by analyzing the image data acquired by the first ship. Such analysis may be performed by the first ship or by the information processing device that received the image data from the first ship.

[0025] Regardless of whether reception processing is performed or not, the control unit 120 may update the stored information repeatedly at predetermined intervals. For example, when the detection target is moving, the position information may be updated by predicting and storing the position at the current time based on the elapsed time from the past time.

[0026] Then, the control unit 120 executes a determination process for the ship to be transmitted (S103). The ship to be transmitted may be all ships existing within the communicable range, or may be only some ships. Some ships may be, for example, the ships that have transmitted a transmission request, but are not limited thereto, and only the ships that satisfy other predetermined conditions may be targeted. Other conditions may be, for example, ships that have transmitted their own ship information (ship type, position, shipowner information, destination information, etc.), or only the ships that have been registered with communication approval by the system side in advance. In this case, an identifier may be added to the transmitted information or encryption processing may be performed so that only specific ships can acquire the information. Such determination conditions for the ship to be transmitted are stored in the storage unit in advance.

[0027] Furthermore, the control unit 120 executes a transmission process for output information to a predetermined second ship (S104). The control unit 120 can transmit the detection target information received so far to the ship (second ship) determined in the determination process (S103) of the ship to be transmitted, for example. As the output information to be transmitted, the detection target information received from all the first ships may be transmitted in the original data format, or may be transmitted in a predetermined data format. Also, various data generated based on the detection target information received from the first ships may be transmitted.

[0028] Also, the transmission process may be repeated every time reception processing or storage processing is performed, may be performed every time the storage unit is updated, may be repeated at regular intervals, or may be performed in response to a reception request from the second ship.

[0029] Also, the second ship that is the target for performing the transmission process may be all ships that can communicate, or only some ships.

[0030] Here, the types of detection targets may include other ships, floating objects, people, and fixed objects such as rocks and shores. Other ships include, for example, fishing boats, water taxis, small multi-purpose boats, pleasure fishing boats, passenger ships, transportation ships, work ships, fire boats / guard boats, pleasure yachts, pleasure motorboats, special work ships, tankers, cargo ships, etc. Floating objects include, for example, driftwood, seaweed, marine organisms, etc. People include users of swimming, surfing, yachts, paddle boards, etc.

[0031] The detection target information is not limited to position information, and may include attitude (direction), presence or absence of movement, movement speed, movement route, type (other ship, floating object, person, etc.), size information, etc. Each type of information is associated with identification information (such as a unique ID) for each detection target and aggregated and stored. Also, the position information and attitude information may be stored in association with time information. Furthermore, various types of information such as position information may include not only the time at which the information was acquired (such as the current time at the detection time), but also predicted position information after a predetermined time. The size information can include at least any one of the maximum diameter (m) in plan view, area (m^2), volume (m^3), and height from the water surface (m).

[0032] The control unit 120 may transmit at least any one of the detection target information and the output data in response to the transmission request information received from the second ship. For example, the second ship transmits transmission request information (signal) via wireless communication, satellite communication, the Internet, etc. to request information regarding the detection target from the information processing device. When the information processing device receives the transmission request information, it can transmit the detection target information to the second ship that transmitted the transmission request information in the same manner.

[0033] In this embodiment, the transmission request information includes the own-ship position information of the second ship, and the control unit 120 may transmit information about a detection target located within a predetermined distance range from the own-ship position of the second ship to the second ship. Specifically, the control unit 120 can extract a detection target located within a range of a predetermined radius (such as 100 m, 200 m, 500 m, 1 km, etc.) centered on the position information (coordinate information) of the second ship from the information in the storage unit to generate output data. Further, the output data may include all the information of the detection targets included in the range, or may be composed of only some of the information. For example, among the detection target information shown in FIG. 4, it may be composed of only the position information, or may be composed of only the type and the position information, or may include other information such as image data. The transmission request information may include time information (for example, the current time when the second ship transmits a transmission request signal, etc.). In that case, the control unit of the information processing device may generate output data based on the time information. For example, only the information detected within a predetermined range (within 1 minute, within 10 minutes, within 30 minutes, within 1 hour, etc.) from the time may be extracted and transmitted to the second ship. Alternatively, according to the elapsed time from the detection time of each detection target shown in FIG. 4 to the transmission request time of the second ship, the position of each detection target may be predicted based on the moving speed and the moving direction, and output data including the predicted position may be generated and transmitted (position prediction processing). In this way, the control unit of the information processing device can generate information corresponding to the time information included in the transmission request information and transmit it to the second ship.

[0034] The control unit 120 may generate output data in which the detection target information is reflected on a pre-stored map and transmit it to the second ship in the transmission process. In this case, the map data can be displayed on a display unit such as a monitor provided on the second ship, and the position of the detection target can be easily grasped. Note that the map data is not limited to only the position information of the detection target, and various information such as the type, size, moving direction, and moving speed may be displayed in text, figures (icons, etc.). According to this, it is possible to confirm at a glance where and what kind of detection target exists. Also, on the map data pre-stored in the second ship, map data similar to the above output data may be generated by reflecting the position information and the like included in the detection target information received from the information processing device.

[0035] In the present embodiment, the detection target information includes the speed information (moving speed) and the traveling direction information (moving direction) of the detection target, and the control unit may generate output data including the predicted position information of the detection target predicted based on the speed information and the traveling direction information. According to this, since the movement of the detection target can be transmitted to the second ship, the safety can be further enhanced and the navigation efficiency of the second ship can be enhanced. That is, for example, if the moving direction of the detection target is a direction away from the second ship, there is no need to avoid it forcibly and there is no need to make a useless detour, so the moving efficiency of the second ship can be enhanced.

[0036] In the present embodiment, the control unit may further execute a matching process of integrating overlapping detection target information by comparing a plurality of received detection target information, and transmit output data based on the information after the matching process to the second ship. With such a configuration, for example, when the same detection target is detected by a plurality of first ships, it is possible to avoid the risk of misrecognizing that there are actually a plurality of one detection target. Information regarding the conditions of the matching process is pre-stored in the storage unit.

[0037] In this embodiment, the control unit may determine that two detected objects are the same when the position information of the two detected objects is within a predetermined distance in the matching process. For example, when the position information of "B0001" and "B0005" in FIG. 4 is within a predetermined distance range, the control unit may determine that they are the same detected object. In that case, "B0005" with the latest time information may be preferentially retained, and "B0001" may be deleted and updated.

[0038] In this embodiment, the control unit may determine that two detected objects are the same when the position information of the two detected objects and the information on the type of the detected object satisfy a predetermined condition in the matching process. For example, when the types of "B0001" and "B0005" in FIG. 4 are the same and the position information is within a predetermined distance range, the control unit may determine that they are the same detected object. In that case, "B0005" with the latest time information may be preferentially retained, and "B0001" may be deleted and updated. By adding not only the position information but also the type information to the matching conditions, a more accurate matching process becomes possible. Note that any information such as size information, moving speed, moving direction, etc. can be added as conditions, and the combination can also be arbitrarily changed.

[0039] In this embodiment, the control unit may store the received position information of the detected object in association with the time information in the storage process. The time information may include, for example, as shown in FIG. 4, either or both of the detection time when the detected object was detected by the first ship and the reception time when the information processing device received the detected object information.

[0040] In this embodiment, the detected object may include floating objects, may include people, or may include other objects such as fixed objects. In this way, information on obstacles that cannot be obtained by AIS or the like can be obtained.

[0041] In this embodiment, an information processing method is provided, in which an information processing apparatus executes a reception process of receiving detection target information including position information of a predetermined detection target detected by a detection unit of each of a plurality of first ships that are sailing, a storage process of aggregating and storing the received detection target information, and a transmission process of transmitting at least any one of the received detection target information and predetermined output data generated under predetermined generation conditions using the detection target information to a predetermined second ship.

[0042] In this embodiment, a program is provided that causes a control unit to execute a reception process of receiving detection target information including position information of a predetermined detection target detected by a detection unit of each of a plurality of first ships that are sailing, a storage process of aggregating and storing the received detection target information, and a transmission process of transmitting at least any one of the received detection target information and predetermined output data generated under predetermined generation conditions using the detection target information to a predetermined second ship.

[0043] In this embodiment, the detection target information may include information regarding the accuracy of the detection unit (including error information). For example, the accuracy information is expressed as 0 to 100% (or 0 to 1), and the control unit of the information processing apparatus can store the accuracy information and generate output information including the accuracy information. For example, when detection target information with different accuracy information is obtained for the position of the same detection target, the position information with a higher accuracy numerical value may be adopted and stored, or weighting corresponding to the accuracy numerical value may be performed, and the position between a plurality of detection target information may be calculated as the position information of the detection target. For example, based on the first position (x1, y1) information with a sensor detection accuracy of 0.6 and the second position (x2, y2) information with a sensor detection accuracy of 0.3, the control unit may estimate the position coordinates shifted toward the first position by the ratio of the accuracy difference rather than the midpoint between the first position and the second position as the position of the detection target. Such estimation conditions are stored in the storage unit in advance.

[0044] In this embodiment, the detection target information may include information on the type (kind) of a detection unit such as a sensor or a camera. The control unit of the information processing device can estimate the accuracy from the information on the type of the detection unit. Then, output information can be generated based on the accuracy in the same manner as described above.

[0045] In this embodiment, the information transmitted from the first ship may include the wind speed and direction obtained by the first ship (own ship), the estimated wave height, wavelength, tidal current, etc. By transmitting this information to the second ship as well, it can be shared. The second ship can acquire various detection information that cannot be obtained by its own ship, and furthermore, a safer and more efficient navigation plan becomes possible.

[0046] In this embodiment, the detection target information is not limited to the first ship only, and information detected by sensors or the like installed on land such as a harbor or a fixed buoy, or information collected from the air by a moving body such as a drone may be received. That is, it can be received from any device equipped with a detection unit and a transmission unit and shared (transmitted) to the second ship or the like.

[0047] In this embodiment, the second ship may be only a ship that does not have a detection unit of other ships or the like or whose detection unit does not function. According to this, the safety of a ship that cannot particularly detect the position of other ships or the like can be improved.

[0048] In this embodiment, the second ship may receive information such as the current position, destination position, waypoint, and the scheduled time of arrival at each position, and the navigation route information from the first ship and transmit it to the second ship.

[0049] Here, an example of the first ship (for example, ships A1, A2) that can communicate with the information processing device will be described. Note that the second ship may also have the same functions as the first ship. That is, it may be possible to display the detection target information received from the information processing device or generate a movement route based on the detection target information.

[0050] The first ship is equipped with detection units such as sensors and cameras, and can detect the positions of surrounding objects (detection targets). For example, the ship is equipped with a camera, and by analyzing the image captured by the camera, the relative positions (three-dimensional coordinates in a camera coordinate system with the camera as the origin) of ships, floating objects, people, etc. included in the image with respect to the own ship (for example, the camera) can be estimated by known techniques. In addition, the ship has an own ship position estimation function, and can estimate the position of the detection target in the real space (world coordinate system) based on the position information of the own ship and the position information of the target relative to the own ship. Note that the position estimation of the detection target may be performed by other known methods. Also, the position information of the own ship may be estimated, for example, as coordinate information as positioning data acquired by a GNSS module as the current position, or may be estimated by self-position estimation processing based on a comparison between map data (three-dimensional point cloud data) acquired from Lidar and map information stored in the storage unit.

[0051] The ship can store the information detected by the detection unit in association with the information of the detection time. Also, the detection information and the detection time information can be transmitted to the information processing device.

[0052] The ship may receive and acquire various information (ship type, size, attitude, moving speed, moving direction, route information, etc.) including the own ship position information of other ships, and may transmit the position information of the other ship to the information processing device. According to this, not only the detection information but also the received position information of other ships, etc. can be transmitted to the information processing device.

[0053] FIG. 5 is an example of an obstacle estimation system (hereinafter, also simply referred to as "system") provided in the first ship according to the present embodiment. The system of the present embodiment can be installed on any type of ship, and is particularly suitable for relatively small ships. Specifically, the system of the present embodiment can be adopted for fishing boats, water taxis, small multi-purpose ships, pleasure fishing boats, passenger ships, traffic ships, work ships, fire boats / guard boats, pleasure yachts, pleasure motor boats, special work ships, etc., and can also be adopted for large passenger ships, tankers, cargo ships, etc.

[0054] The system of this embodiment is an obstacle estimation system 1 for detecting obstacles (objects to be detected) when a ship is navigating. This system 1 includes, for example, a data acquisition unit 10, a control unit 20, a storage unit 30, an input unit 40, an output unit 50, a communication unit 60, and a ship motion control unit 70. Each component is connected by wire or wirelessly and can communicate with each other. Also, in this example, the control unit 20, the storage unit 30, the input unit 40, the output unit 50, and the communication unit 60 are configured to be implemented in an information processing device (computer), but it is not limited to this, and each unit may be independent.

[0055] The data acquisition unit 10 includes a plurality of data acquisition means such as various sensors and cameras, and acquires various sensor data and image data. The data acquisition unit can include, for example, cameras, Lidar (Light detection and ranging), marine radars, radar devices such as millimeter-wave radars, ultrasonic ranging sensors, positioning devices such as GNSS modules and QZSS modules, AIS, inertial measurement units (IMUs), inertial sensors, wind direction sensors, wind speed sensors, speed sensors (ground speed sensors, water speed sensors), azimuth sensors, acceleration sensors, gyro sensors, magnetic compasses, satellite compasses, temperature sensors, humidity sensors, barometric pressure sensors, altitude sensors, infrared sensors, etc. The data acquisition unit 10 can acquire information regarding the current position (coordinates), attitude (azimuth such as the bow direction), moving speed, moving direction, bow direction, and turning amount of the own ship, information on the surrounding environment, and information on obstacles such as other ships and people. AIS is a system for transmitting and receiving various information between ships and between ships and shore facilities. For example, ship information such as the position, attitude (course), moving speed, and destination of each ship can be exchanged by wireless communication. The azimuth sensor may be a magnetic azimuth sensor that calculates the bow azimuth using geomagnetism, a gyrocompass, a GPS compass, etc.

[0056] The control unit 20 includes a processor (arithmetic unit) such as a CPU, and is configured to be able to execute various information processes based on a program stored in the storage unit 30. The control unit 20 can execute a ship information estimation process for estimating the ship position information and the ship attitude information based on the data (first data) from the data acquisition unit 10. The position information of the ship can be estimated with the coordinate information as the positioning data acquired by the GNSS module as the current position. The position information of the ship may be estimated by a self-position estimation process based on a comparison between the map data (3D point cloud data) acquired from the Lidar and the map information stored in the storage unit. The position information of the ship is expressed by two-dimensional or three-dimensional coordinate information on a specific coordinate system, and may be expressed by latitude and longitude. The attitude of the ship is expressed by azimuth (angle) information in a two-dimensional or three-dimensional specific coordinate system. The attitude information may be expressed by the azimuth of east, west, south, and north.

[0057] The control unit 20 can execute a ship information correction process for correcting the ship position information and the ship attitude information based on the data (second data) from the data acquisition unit 10. The control unit 20 can correct the ship position information and the ship attitude information based on, for example, the speed data acquired from the Lidar or the camera.

[0058] The control unit 20 can execute a speed estimation process for estimating the speed of the ship based on the data from the data acquisition unit 10. For example, the distance traveled in the predetermined time can be calculated from the position information before the predetermined time (0.01 seconds, 0.1 seconds, 1 second, etc.) and the current position information, and the speed can be estimated from the information on the distance traveled in the predetermined time.

[0059] The control unit 20 can execute an obstacle information estimation process for estimating the position and size of an obstacle based on the ship position information and the ship attitude information corrected by the ship information correction process, and the data (third data) from the data acquisition unit 10. The control unit 20 can also estimate the type of the obstacle based on the ship position information and the ship attitude information corrected by the ship information correction process, and the data (third data) from the data acquisition unit 10.

[0060] The control unit 20 can estimate the type of the obstacle by, for example, selecting one from among the candidates for the type of obstacle stored in advance in the storage unit 30. The information on the type of obstacle can include information on types such as ships, specific ship classifications (fishing boats, water taxis, small multi-purpose boats, pleasure fishing boats, passenger ships, traffic boats, work boats, fire boats / guard boats, pleasure yachts, pleasure motor boats, special work boats, etc.), marine structures, people (including swimming, surfing, yachts, paddle boards, etc.), rocks, shores, driftwood, seaweed, marine organisms, etc. The information on the type of obstacle is stored in association with the information on the size (area, height).

[0061] The control unit 20 can estimate the position, attitude, type, and size of the obstacle by, for example, performing image analysis on the image data acquired by the camera. In the obstacle information estimation process, the control unit 20 may use a machine learning technique using a pre-trained learning model to estimate the position, attitude, type, and size of the obstacle. When the control unit 20 estimates the position of the obstacle based on the image data acquired by the camera, after estimating the three-dimensional position (coordinates) of the obstacle in the camera coordinate system, based on the information on the position (coordinates) and shooting direction of the camera, the camera coordinate system can be converted into the world coordinate system by coordinate conversion processing to calculate the position (coordinates) of the obstacle in the world coordinate system. The control unit 20 can estimate the bow direction of the ship as an obstacle, the direction of the face of a person (front direction), the direction of an object used such as a surfboard (traveling direction) by performing image analysis of the image data from the camera, or analysis of three-dimensional data (point cloud data, model data, etc.). The control unit 20 is not limited to the image data from the camera as described above, and can estimate the three-dimensional position and attitude of the obstacle on or under the water based on the sensor data of any one or a combination of a radar device, Lidar, etc.

[0062] The storage unit 30 may store, in advance, candidates for the types of obstacles, values of sizes (information regarding area in plan view and height) corresponding to each type, and size ranges (upper limit values, lower limit values). For example, the area range of a ship may be 3 m^2 or more and 9200 m^2 or less, and the height range of the ship (height above water level) may be 0.5 m or more and 100 m or less. Also, the area range of a person may be 0.2 m^2 or more and 5 m^2 or less, and the height range of the person (height above water level) may be 0.1 m or more and 3 m or less. The storage unit 30 stores information on the appropriate avoidance distance associated with obstacle information such as the position, type, and size of the obstacle. The storage unit 30 may also store information regarding a calculation formula for calculating the appropriate avoidance distance. For example, the appropriate avoidance distance may be calculated according to the size (plan view area) of the ship as an obstacle. Also, the parameters used in the calculation may include information such as the type, moving speed, and moving direction of the obstacle. The storage unit 30 may include information on a program that simulates the movement along the navigation route (planned movement route) of the own ship and the presence or absence of contact with an obstacle. The storage unit 30 stores information regarding each data acquisition means. The information regarding the data acquisition means may be, for example, the position of the data acquisition means on the own ship (relative position with respect to a specific point such as the center or centroid of the ship), relative attitude (for example, relative direction with respect to the bow direction of the own ship), etc., but is not limited thereto.

[0063] The control unit 20 may estimate the moving speed, moving direction, and attitude of an obstacle based on the own ship position information and own ship attitude information corrected by the own ship information correction process, the moving speed information and moving azimuth information of the own ship, and the data (third data) from the data acquisition unit 10. The control unit 20 can acquire relative moving speed information and moving direction information of an obstacle such as another ship with respect to the own ship based on the data from the Lidar. The control unit 20 can also acquire moving speed information, moving direction information, etc. of the obstacle using the data from the radar device.

[0064] In the obstacle information estimation process, the control unit 20 can analyze the image data acquired by the camera to estimate information such as the type, size, position, shape, and color of the obstacle. Further, the control unit 20 may estimate the obstacle information based on any of the information of the obstacle estimated from the image data and the obstacle-related information pre-stored in the storage unit. For example, the control unit 20 may estimate the size of the obstacle from the image data, refer to the storage unit, and determine (estimate) the candidate of the obstacle associated with the size range that matches the size as the type of the obstacle. Also, the type of the obstacle may be estimated by analyzing the image data acquired by the camera, and the size candidate associated with the type may be determined (estimated) as the size of the obstacle by referring to the storage unit. Further, not limited to the image data acquired by the camera, any one or a combination of the detection data acquired from the radar device and the detection data from the Lidar may be used to estimate any of the type, size, and position of the obstacle. For example, 3D data of the obstacle can be generated from the distance data to a number of points of the obstacle by the Lidar, and based on the 3D data of the obstacle, the 3D shape, type, size, position, etc. can be estimated. In this way, the control unit 20 can select from the obstacle candidates by comparing the information of the obstacle candidates stored in the storage unit 30 with the data related to the obstacle acquired from the data acquisition unit 10, and estimate the type, size, etc. of the detected obstacle. Note that the control unit 20 can also estimate the position and size of the obstacle based only on the data from the data acquisition unit 10.

[0065] The control unit 20 can store various information such as the generated information in the storage unit 30, update the information in the storage unit 30, output it from the output unit 50, or transmit it to an external device or another ship via the communication unit 60. Further, the control unit 20 can receive the information acquired by the data acquisition unit 10, the operation information of the user input via the input unit 40, and the information received by the communication unit 60, and execute information processing based on the information.

[0066] The storage unit 30 can store various information stored in advance, information acquired from the data acquisition unit 10, information received from an external device or another ship via the communication unit 60, information generated by the control unit 20, information input via the input unit 40, etc., and is composed of a non-volatile memory, a hard disk, etc. The storage unit 30 can store information regarding various parameters indicating the shape of the own ship (such as 3D model data), characteristics (characteristics regarding output, speed, turning amount), information regarding obstacles, 2D or 3D map information, navigation history information, navigation plan information, tide information, etc.

[0067] The input unit 40 receives input information based on operations from the user, etc. The input unit 40 is composed of a mechanical button, switch, operation lever, touch panel, etc. The input unit 40 may be provided with a microphone capable of voice input, etc.

[0068] The output unit 50 outputs various information as images (videos), voices, etc. The output unit 50 may be provided with, for example, a display unit such as a liquid crystal monitor or touch panel for displaying images, a voice output unit such as a speaker for displaying voices, a vibration generating unit (vibration device) for generating vibrations, etc.

[0069] The communication unit 60 is connected to a network such as the Internet or wireless communication, and can transmit data to an external device (management server, smartphone used by the user, tablet terminal, etc.) or another ship, or receive data from an external device.

[0070] The ship motion control unit 70 may include an auto-rudder for controlling steering, an auto-throttle for controlling a power unit (engine, motor, etc.) for rotating a propeller or thruster, etc. The ship motion control unit 70 controls the motion of the ship 70 (motions such as forward, backward, left and right movement, turning, etc.) based on instruction information from at least any one of the control unit 20, the storage unit 30, the input unit 40, and the communication unit 60, and can navigate the ship along a predetermined route.

[0071] FIG. 6 shows an example of the case where the present system is installed on ship A. In the example of FIG. 6, it includes a Lidar, a camera, a GNSS antenna, an AIS antenna, and a wind direction and speed meter as the data acquisition unit 10, a control unit as the control unit 20, a monitor as the output unit 50, and an auto-rudder and an auto-throttle as the ship motion control unit 70. In the example of FIG. 6, the Lidar and the camera are respectively installed on the front side (bow side) and the rear side (stern side) of the ship 100, so that data around the ship 100, particularly data on the front side and the rear side, can be detected with high accuracy. The configuration of the present system installed on the ship is not limited to the illustrated example, and other sensors or the like may be provided.

[0072] FIG. 7 shows an example of a processing flow related to the symmetry detection method according to the system of the present embodiment.

[0073] The control unit 20 executes a own ship information estimation process (S201) for estimating own ship position information and own ship attitude information based on the first data acquired from at least one data acquisition means, a own ship information correction process (S202) for correcting own ship position information and own ship attitude information based on the second data acquired from at least one data acquisition means, an obstacle information estimation process (S203) for estimating the position and size of an obstacle based on the own ship position information and own ship attitude information corrected in the own ship information correction process and the third data acquired from at least one data acquisition means, and an output information generation process (S204) for generating output information based on the position and size information of the obstacle estimated in the obstacle information estimation process. The first data, the second data, and the third data may each include data acquired from a common data acquisition means. Also, the first data, the second data, and the third data may each be data acquired from one data acquisition means, or may include data acquired from a plurality of data acquisition means. Note that the own ship information correction process is not essential, and the obstacle information estimation process may be performed after the own ship information estimation process.

[0074] In the own-ship information estimation process (S201), for example, the three-dimensional position of the own ship is estimated based on the coordinate information from the GNSS antenna as the data acquisition means, and the three-dimensional attitude information of the own ship is estimated based on the azimuth data acquired from the inertial measurement unit. The own-ship information estimation process (S201) may be executed based on the data acquired from only one data acquisition means, but from the viewpoint of improving the estimation accuracy, it is preferable to execute the estimation process based on the data acquired from a plurality of data acquisition means.

[0075] In the own-ship information correction process (S202), for example, based on the speed information acquired from at least one of the Lidar and the camera, the own-ship position information and the own-ship attitude information estimated in S101 are corrected. The control unit 20 estimates the position (coordinates) and the attitude at the current time (after a predetermined time has elapsed from that time) using, for example, the position (coordinates), the movement direction information, and the movement speed information at a time point before a predetermined time (0.01 seconds, 0.1 seconds, 1 second, etc.). Then, by comparing the information on the estimated position and attitude with the own-ship position information and the own-ship attitude information estimated in the own-ship information estimation process (S201), when they do not match, the numerical values between the two are adopted as the information on the accurate current position and the current attitude, whereby the own-ship position information and the own-ship attitude information can be corrected. Note that the own-ship information correction process may be other methods. For example, the own-ship position information and the own-ship attitude information may be corrected using the position information and the attitude information calculated by the self-position and attitude estimation process based on the image data from the camera, or they may be combined. Also, from the viewpoint of improving the accuracy of the correction process, it is preferable to execute the correction process based on the data acquired from a plurality of data acquisition means.

[0076] Next, in the obstacle information estimation process (S203), for example, based on the own ship position information and own ship attitude information corrected in the own ship information correction process (the position and attitude of the camera in the ship's coordinate system), the position and attitude of the camera are estimated, and coordinate conversion is performed using the relative position data of the obstacle on the camera coordinate system to estimate the position of the obstacle on the world coordinate system. Also, the image data of the camera is analyzed to estimate the type and size of the obstacle. The estimation of the position, type, and size of the obstacle may be performed alone or in combination using the detection data obtained from the radar device.

[0077] Then, in the output information generation process (S204), for example, output information including various information such as the position, type, moving speed, moving direction, size, etc. of the obstacle can be generated and transmitted to the information processing device 100. The transmission process may be repeatedly executed at predetermined intervals, or may be transmitted each time a new target is detected. Also, the output information may include image data or audio data, and the position of the obstacle may be displayed on the map of the own ship monitor, the type and size of the obstacle may be displayed in text or image, or an audio output may be made from the speaker to prompt attention. The content of the output information is not particularly limited. For example, navigation route information for avoiding the obstacle and navigating may be generated and transmitted to the information processing device 100 or the like, displayed on the monitor, or applied to the autopilot system for automatic navigation. The output information may include the current position, destination position, waypoint position, and the scheduled arrival times at the destination position and waypoint position.

[0078] The output information may include information on the distance to be kept away from the obstacle (appropriate avoidance distance). For example, when the size of the obstacle is small (such as 1 m^2 or less), the appropriate avoidance distance may be set small (such as 5 m, 10 m, etc.), and the larger the size of the obstacle (such as 10 m^2 or more), the larger the appropriate avoidance distance may be set (such as 20 m, 50 m, etc.). Also, for example, when the obstacle is a person, the appropriate avoidance distance may be set large (such as 20 m, 50 m, etc.), and when the obstacle is a floating object such as a buoy, driftwood, or seaweed, the appropriate avoidance distance may be set small (such as 5 m, 10 m, etc.). The appropriate avoidance distance may be determined based on the appropriate avoidance distance information pre-stored in the storage unit according to at least one of the type and size of the obstacle. In that case, the information on the type and size of the obstacle and the information on the appropriate avoidance distance are pre-associated and stored. Also, when the obstacle is another ship, the appropriate avoidance distance may be calculated based on the moving speed and moving direction of the other ship.

[0079] In this embodiment, the control unit 20 may perform various processes based on information (including error information) regarding the accuracy (correctness) of a detection unit (sensor, camera, etc.) stored in advance in the storage unit. For example, in the own-ship position estimation process, only the information of sensors with high accuracy, which is an accuracy numerical value equal to or higher than a predetermined threshold, may be preferentially used (information of sensors with low accuracy whose accuracy numerical value is less than the threshold is not used), or the detection information may be integrated and used based on the weighting stored in advance in the storage unit. For example, the accuracy information is expressed as 0 to 100% (which may be expressed as a decimal between 0 and 1), and the control unit of the information processing device can store or update the accuracy information based on the input information from the user. The various processes may include at least any one of the processes of estimating and correcting the own-ship position information and the own-ship attitude information, and the process of estimating obstacles. In this way, by preferentially using a detection unit with high accuracy for information processing, the accuracy of various processes such as the own-ship position, the own-ship attitude, and obstacle detection can be improved. For example, when the control unit 20 has a plurality of different sensors capable of estimating the own-ship position information, based on the weighting of the accuracy information, it can estimate the position determined based on the weighting of the accuracy information among the positions detected by the plurality of sensors. When different detection target information with different accuracy information is obtained for the position of the same detection target among the detected (estimated) plurality of own-ship position information, the position information with a high accuracy numerical value may be adopted and stored, or weighting corresponding to the accuracy numerical value may be performed, and the position between the plurality of detection target information may be calculated as the position information of the detection target. For example, based on the first position (x1, y1) information with a sensor detection accuracy of 0.6 and the second position (x2, y2) information with a sensor detection accuracy of 0.3, the control unit may estimate the position coordinates shifted toward the first position by the ratio of the accuracy difference rather than the midpoint between the first position and the second position as the position of the detection target. Such estimation conditions are stored in the storage unit in advance.

[0080] The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings. However, the technical scope of the present disclosure is not limited to such examples. It is obvious that those having ordinary knowledge in the technical field of the present disclosure can conceive of various modification examples or correction examples within the scope of the technical idea described in the claims, and it is naturally understood that these also belong to the technical scope of the present disclosure.

[0081] The devices described in this specification may be realized as a single device, or may be realized by a plurality of devices (such as cloud servers) partially or entirely connected by a network. For example, the control unit and the storage unit may be realized by different servers connected to each other by a network.

[0082] A series of processes by the devices described in this specification may be realized using any of software, hardware, and a combination of software and hardware. It is possible to create a computer program for realizing each function of the control unit according to this embodiment and install it on a PC or the like. In addition, a computer-readable recording medium storing such a computer program can also be provided. The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, etc. Further, the above computer program may be distributed via a network, for example, without using a recording medium.

[0083] Also, the processes described using flowcharts in this specification do not necessarily have to be executed in the order shown in the figures. Some process steps may be executed in parallel. Also, additional process steps may be adopted, and some process steps may be omitted.

[0084] Also, the effects described in this specification are merely illustrative or exemplary and not limiting. That is, the technology according to the present disclosure may exhibit other effects obvious to those skilled in the art from the description of this specification, together with or instead of the above effects.

[0085] Note that the following configurations also fall within the technical scope of the present disclosure. (Item 1) Receiving processing for receiving detection target information including position information of a predetermined detection target detected by a detection unit of each of a plurality of first ships that are sailing, Storage processing for aggregating and storing the received detection target information, An information processing apparatus including a control unit that executes transmission processing for transmitting at least any one of the received detection target information and predetermined output data generated under generation conditions predetermined using the detection target information to a predetermined second ship. (Item 2) The information processing apparatus according to Item 1, wherein the control unit transmits at least any one of the detection target information and the output data in response to transmission request information received from the second ship. (Item 3) The transmission request information includes own ship position information of the second ship, The control unit, transmits information about the detection target located within a predetermined distance range from the own ship position of the second ship to the second ship. The information processing apparatus according to Item 2. (Item 4) The information processing apparatus according to Item 1, wherein the control unit generates output data in which the detection target information is reflected on a map stored in advance, and transmits the output data to the second ship in the transmission processing. (Item 5) The detection target information includes speed information and traveling direction information of the detection target, The information processing apparatus according to Item 1, wherein the control unit generates the output data including predicted position information of the detection target predicted based on the speed information and the traveling direction information. (Item 6) The control unit further, executes matching processing for integrating overlapping detection target information by comparing a plurality of the received detection target information, The information processing apparatus according to Item 1, wherein output data based on the information after the matching processing is transmitted to the second ship. (Item 7) The control unit In the matching process, when the position information of two detection targets is within a predetermined distance determined in advance, it is determined as the same detection target. The information processing apparatus according to item 6 (Item 8) The control unit In the matching process, when the position information of two detection targets and the information on the type of the detection target satisfy a predetermined condition determined in advance, it is determined as the same detection target. The information processing apparatus according to item 6 (Item 9) In the storage process, the control unit stores the received position information of the detection target in association with time information. The information processing apparatus according to item 1 (Item 10) The detection target includes other ships. The information processing apparatus according to item 1 (Item 11) The detection target includes floating objects. The information processing apparatus according to item 1 (Item 12) The detection target includes people. The information processing apparatus according to item 1 (Item 13) A reception process of receiving detection target information including the position information of a predetermined detection target detected by a detection unit of each first ship from a plurality of first ships sailing; A storage process of aggregating and storing the received detection target information; A transmission process of transmitting at least any one of the received detection target information and predetermined output data generated under predetermined generation conditions using the detection target information to a predetermined second ship. An information processing method executed by an information processing apparatus (Item 14) A reception process of receiving detection target information including the position information of a predetermined detection target detected by a detection unit of each first ship from a plurality of first ships sailing; A storage process of aggregating and storing the received detection target information; A program that causes a control unit to execute a transmission process of transmitting at least one of the detected target information that has been examined and predetermined output data generated under generation conditions predetermined using the detected target information to a predetermined second ship.

Explanation of Signs

[0086] 1 Movement Route Generation System 10 Data Acquisition Unit 20 Control Unit 30 Storage Unit 40 Input Unit 50 Output Unit 60 Communication Unit 70 Ship Operation Control Unit

Claims

1. a receiving process for receiving detection target information from a plurality of sailing first ships, the detection target information including position information of a predetermined detection target other than the first ship detected by a detection unit of each of the first ships; A storage process of aggregating and storing the received detection target information; a control unit that executes a transmission process of transmitting at least one of the received detection target information and predetermined output data generated under predetermined generation conditions using the detection target information to a predetermined second ship, The control unit estimates position information for each detection target based on information regarding accuracy of the detection unit, and transmits the estimated position information for each detection target to the second ship.

2. The information processing device according to claim 1 , wherein the control unit transmits information about the detection target located within a predetermined distance range from the position of the second ship to the second ship based on the position information of the second ship.

3. The information processing apparatus according to claim 1 or 2, wherein the detection target information includes travel direction information of the detection target detected by a detection unit of the first ship.

4. The information processing device according to claim 1 , wherein the control unit transmits at least one of moving speed information and traveling direction information of the detection target to the second vessel.

5. The information processing apparatus according to claim 1 , wherein the detection target information includes information on a type of the detection target.

6. The information processing apparatus according to claim 1 , wherein the detection target information includes information about a size of the detection target.

7. 3. An information processing device as described in claim 1 or 2, wherein the control unit transmits to the second ship movement speed information of the detection target detected by the detection unit of the first ship, or movement speed information of the detection target estimated based on the detection target information.

8. The information processing device according to claim 1 or 2, wherein the control unit receives information regarding the type of the detection unit from the first ship and transmits the information regarding the type of the detection unit to the second ship.

9. The information processing apparatus according to claim 1 , wherein the detection target information includes information on a time when the detection target is detected.

10. a receiving process for receiving detection target information from a plurality of sailing first ships, the detection target information including position information of a predetermined detection target other than the first ship detected by a detection unit of each of the first ships; A storage process of aggregating and storing the received detection target information; a transmission process of transmitting at least one of the received detection target information and predetermined output data generated under predetermined generation conditions using the detection target information to a predetermined second ship; An information processing method in which the information processing device estimates position information for each detection target based on information regarding accuracy of the detection unit, and transmits the estimated position information for each detection target to the second ship.

11. a receiving process for receiving detection target information from a plurality of sailing first ships, the detection target information including position information of a predetermined detection target other than the first ship detected by a detection unit of each of the first ships; A storage process of aggregating and storing the received detection target information; a transmission process of transmitting at least one of the received detection target information and predetermined output data generated under predetermined generation conditions using the detection target information to a predetermined second vessel; The control unit estimates position information for each detection target based on information regarding accuracy of the detection unit, and transmits the estimated position information for each detection target to the second ship.

Images