Information processing device, information processing method, and program

Abstract

An information processing device, an information processing method, and a program are provided that are capable of improving safety during ship navigation in a new way. [Solution] An information processing device according to the present disclosure includes a control unit that executes a receiving process for receiving detection target information from a plurality of sailing first ships, the detection target information including position information of a specified detection target detected by a detection unit of each first ship, a storage process for consolidating and storing the received detection target information, and a transmission process for transmitting at least one of the received detection target information and specified output data generated using the detection target information under predetermined generation conditions to a specified second ship.

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Conventionally, systems such as AIS (Automatic Identification System) have been known as a means of sharing a ship's position information with other ships to improve the safety of ship navigation. However, many small ships, such as fishing boats, are not equipped with AIS, and there have been issues with information sharing.

[0003] In response to this, for example, Patent Document 1 proposes a system that uses a device equipped with GPS functionality to transmit a ship's position information to other ships, so that even ships not equipped with AIS can share their own position information. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2006-163765 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, it is practically difficult to equip all ships with AIS or GPS-equipped devices.

[0006] Therefore, the present disclosure has been made in consideration of the above-mentioned problems, and its purpose is to provide an information processing device, an information processing method, and a program that can improve safety during ship navigation in a new way. [Means for solving the problem]

[0007] According to the present disclosure, a receiving process receives detection target information from a plurality of sailing first ships, the detection target information including position information of a predetermined detection target detected by a detection unit of each of the first ships; A storage process of aggregating and storing the received detection target information; An information processing device is provided that includes a control unit that executes a transmission process to transmit at least one of the received detection target information and specified output data generated under predetermined generation conditions using the detection target information to a specified second ship. Effect of the Invention

[0008] According to the present disclosure, it is possible to provide an information processing device, an information processing method, and a program that are capable of improving safety during ship navigation in a new way. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 illustrates an example of a system according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a diagram illustrating an example of an information processing device according to the embodiment. [Diagram 3] FIG. 4 is a flowchart illustrating a series of controls in the system according to the embodiment. [Figure 4] FIG. 11 is a diagram showing an example of detection target information according to the embodiment. [Diagram 5] 2 shows an example of a ship detection system according to the embodiment. [Figure 6] 2 shows an example of a vessel according to the embodiment. [Figure 7] 4 is a flowchart illustrating a series of controls in the detection system according to the embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are denoted by the same reference numerals, and redundant description will be omitted.

[0011] Fig. 1 is a conceptual diagram of a system using an information processing device according to the present embodiment. Fig. 1 illustrates ships A1 and A2 that are sailing and can communicate with the information processing device 100, and a ship B, a floating object C, and a person D that cannot communicate with the information processing device 100.

[0012] The information processing device 100 shown in Fig. 1 functions as a management server (for example, a cloud server). As shown in Fig. 2, the information processing device 100 includes a control unit 120, a storage unit 130, an input unit 140, an output unit 150, and a communication unit 160. In this example, the control unit 120, the storage unit 130, the input unit 140, the output unit 150, and the communication unit 160 are implemented in an information processing device (computer), but this is not limiting, and the system may be configured by being divided into a plurality of devices, or each unit (at least a part) may be provided independently.

[0013] The control unit 120 includes a processor (arithmetic unit) such as a CPU, and is configured to be able to execute various types of information processing based on programs stored in the storage unit .

[0014] The storage unit 130 can store various types of information stored in advance, information generated by the control unit 120, user operation information input via the input unit 140, information received from the ship or an external device via the communication unit 160, etc. The storage unit 130 includes a main storage device configured as a volatile storage device such as a DRAM (Dynamic Random Access Memory), and an auxiliary storage device configured as a non-volatile storage device such as a flash memory or an HDD (Hard Disc Drive).

[0015] The storage unit 130 can store two-dimensional or three-dimensional map data (cartographic information) corresponding to real space, received information, various information generated based on the received information, various information related to the detection target, condition information for various processes executed by the control unit, etc. The various processes can store information related to various conditions such as a matching process, a position prediction process, and a target estimation process, which will be described later. The storage unit 130 can store information related to various parameters indicating the shapes (three-dimensional model data, etc.) and characteristics (characteristics related to output, speed, and turning amount) of various ships, navigation history information, navigation route information, tidal information, etc.

[0016] The input unit 140 receives input information based on operations from a user, etc. The input unit 140 is composed of mechanical buttons, switches, operation levers, a touch panel, a keyboard, a mouse, etc. The input unit 140 may also include a microphone that allows voice input.

[0017] The output unit 150 outputs various information as images (video), sounds, etc. The output unit 150 may include, for example, a display unit such as a liquid crystal monitor or a touch panel that displays images, an audio output unit such as a speaker that displays sounds, a vibration generating unit (vibration device) that generates vibrations, etc.

[0018] The communication unit 160 is connected to a network such as the Internet or wireless communication, and receives and transmits data from the ship, external devices (such as a management server, a control device, and a terminal device used by a user), and the like.

[0019] The control unit 120 can estimate the position, type, size, posture, etc. of an obstacle by, for example, performing image analysis on image data acquired by a camera (target estimation process). The control unit 120 may estimate the type of an obstacle by selecting one of the candidates of the type of obstacle stored in advance in the storage unit 130. The information on the type of obstacle can include type information such as ships, specific ship classifications (fishing boats, water taxis, small multipurpose boats, recreational fishing boats, passenger boats, transportation boats, work boats, fireboats / patrol boats, pleasure yachts, pleasure motorboats, special work boats, tankers, cargo ships, etc.), marine structures, people (including swimming, surfing, yachts, paddle boards, etc.), rocks, shores, driftwood, seaweed, marine life, etc. The information on the type of obstacle is stored in association with size (area, height) information.

[0020] In the obstacle information estimation process, the control unit 120 may estimate the position, type, size, attitude, and the like of the obstacle using a machine learning technique using a learning model that has been trained in advance. When the control unit 120 estimates the position of the obstacle based on image data acquired by the camera, after estimating the three-dimensional position (coordinates) of the obstacle in the camera coordinate system, the control unit 120 can convert the camera coordinate system to a world coordinate system by a coordinate conversion process based on the information on the camera position (coordinates) and shooting direction, and calculate the position (coordinates) of the obstacle on the world coordinate system. The control unit 120 can estimate the bow direction of a ship as an obstacle, the direction of a person's face (front direction), and the direction (traveling direction) of an object used such as a surfboard by image analysis of the image data from the camera or analysis of three-dimensional data (point cloud data, model data, etc.).

[0021] The control unit 120 of the information processing device can execute a receiving process of receiving detection target information including position information of a predetermined detection target detected by the detection unit of each first ship from a plurality of first ships sailing, a storage process of aggregating and storing the received detection target information, and a transmission process of transmitting at least one of the received detection target information and the predetermined output data generated under a predetermined generation condition using the detection target information to a predetermined second ship. According to this embodiment, the information detected by the first ship can be aggregated and stored and transmitted to another second ship. As a result, for example, the position information of a ship not having an AIS or GPS can be shared with the second ship via the information processing device from the first ship, thereby improving safety during navigation. Note that the ship A1 in FIG. 1 may function as the first ship and the ship A2 may function as the second ship, or vice versa. Also, the ships A1 and A2 may function as the first ships, and the ships A1 and A2 may function as the second ships. Also, at least one of the vessels A1 and A2 may function as the first vessel, and another vessel (not shown) may function as the second vessel. In either case, if any of the first vessels can detect detection targets such as the vessel B, floating object C, or person D, the detection targets can be shared with the other second vessels that cannot detect them, thereby improving safety during navigation.

[0022] For example, as shown in Fig. 3, the control unit 120 executes a reception process to acquire detection target information from a first vessel (S101). The reception process only needs to be able to receive detection target information from at least one first vessel. Furthermore, the information processing device maintains a continuous reception-enabled state unless the control unit 120 stops the reception function.

[0023] Next, the control unit 120 aggregates the detection target information and executes a storage process (S102). Aggregating and storing means consolidating and managing information acquired from a number of different ships in a centralized manner. The storage process is performed each time a reception process is performed. The control unit 120 may assign unique identification information (such as an ID including a number or letters) to each detection target and accumulate the information in a detection target information table. Also, map information that reflects the position (coordinates) of the detection target in pre-stored map information may be stored.

[0024] FIG. 4 shows an example of a detection target information table. In the example of FIG. 4, the detection target information includes information on the type, position, attitude, moving speed, moving direction, size, detection time (time when the first vessel detected it), reception time (time when the information processing device received it), and transmitting vessel (the vessel that detected the information and transmitted it to the information processing device), which are associated with the identification information and managed in an aggregated manner. The detection target information may include other information, for example, image information of each detection target photographed by a camera of the first vessel may be stored in association with the detection target. The detection target information received by the information processing device may include at least position information, and for example, the attitude, moving speed, etc. may be unknown. In the example of FIG. 4, for example, for the "driftwood" with the identification information "B0003" transmitted from the vessel "A1", information on the attitude and movement cannot be detected and is unknown (indicated by "-" in the figure). The information on the attitude and moving direction is expressed as a numerical value up to 360, with any of the east-west-north-south directions being 0°, but is not limited to this. In addition, in the illustrated example, the size is expressed in three levels: "large," "medium," and "small," but is not limited to this and may be two levels or less or four levels or more, the levels may be expressed numerically, or the width (maximum width, diameter, etc.), height, area, volume, etc. may be expressed numerically. This information can be estimated by known methods by analyzing image data acquired by the first ship. Such analysis may be performed by the first ship, or by an information processing device that receives the image data from the first ship.

[0025] The control unit 120 may repeatedly update the stored information at predetermined intervals regardless of whether or not a reception process is being performed. For example, if the detection target is moving, the control unit 120 may update the location information by predicting and storing the current location based on the elapsed time from the past time.

[0026] Then, the control unit 120 executes a process for determining the target ships (S103). The target ships may be all ships within the communication range, or only some of the ships. The "some ships" may be, for example, ships that have transmitted a transmission request, but are not limited to this, and may be only ships that satisfy other predetermined conditions. The other conditions may be, for example, ships that have transmitted information about their own ship (ship type, position, shipowner information, destination information, etc.), or only ships that have been registered with the system's prior approval for communication. In this case, the transmitted information may be assigned an identifier or encrypted so that only specific ships can obtain the information. Such conditions for determining the target ships are stored in the memory unit in advance.

[0027] Furthermore, the control unit 120 executes a process of transmitting output information to a specified second ship (S104). The control unit 120 can transmit the detection target information received so far to, for example, the ship (second ship) determined in the process of determining the transmission target ship (S103). As the output information to be transmitted, the detection target information received from all first ships may be transmitted in its original data format, or may be transmitted in a predetermined data format. In addition, various data generated based on the detection target information received from the first ship may be transmitted.

[0028] In addition, the transmission process may be repeated each time the reception process or storage process is performed, or each time the memory unit is updated, or may be repeated at regular intervals, or may be performed in response to a reception request from the second vessel.

[0029] The second ships as targets for transmission processing may be all ships with which communication is possible, or only some of the ships.

[0030] Here, the types of detection targets may include other ships, floating objects, people, and fixed objects such as rocks and shorelines. Examples of other ships include fishing boats, water taxis, small multipurpose boats, recreational fishing boats, passenger boats, transportation boats, work boats, fireboats / patrol boats, pleasure yachts, pleasure motorboats, special work boats, tankers, cargo ships, etc. Examples of floating objects include driftwood, seaweed, marine life, etc. Examples of people include people swimming, surfing, yachting, paddle boarding, etc.

[0031] The detection target information is not limited to position information, but may include posture (direction), presence or absence of movement, movement speed, movement route, type (other ships, floating objects, people, etc.), size information, etc., and various information is stored in association with identification information (such as a unique ID) assigned to each detection target. In addition, the position information and posture information may be stored in association with time information. Furthermore, various information such as position information may include not only the time when the information was acquired (such as the current time at the time of detection), but also predicted position information after a predetermined time. The size information may include at least one of the maximum diameter (m) in a planar view, area (m^2), volume (m^3), and height from the water surface (m).

[0032] The control unit 120 may transmit at least one of the detection target information and the output data in response to the transmission request information received from the second vessel. For example, the second vessel transmits transmission request information (signal) via wireless communication, satellite communication, the Internet, etc. to request information about the detection target from the information processing device. When the information processing device receives the transmission request information, it can transmit the detection target information in a similar manner to the second vessel that transmitted the transmission request information.

[0033] In this embodiment, the transmission request information may include the second ship's own ship position information, and the control unit 120 may transmit information about the detection target located within a predetermined distance range from the second ship's own ship position to the second ship. Specifically, the control unit 120 may extract the detection target located within a predetermined radius (100 m, 200 m, 500 m, 1 km, etc.) centered on the position information (coordinate information) of the second ship from the information in the storage unit to generate output data. In addition, the output data may include all information about the detection target included in the range, or may be composed of only a part of the information. For example, the detection target information shown in FIG. 4 may be composed of only the position information, or may be composed of only the type and position information, or may include other information such as image data. The transmission request information may include time information (for example, the current time when the second ship transmitted the transmission request signal, etc.). In that case, the control unit of the information processing device may generate the output data based on the time information. For example, only information detected within a specified range from the relevant time (within 1 minute, within 10 minutes, within 30 minutes, within 1 hour, etc.) may be extracted and transmitted to the second vessel. Alternatively, the position of each detection target shown in Figure 4 may be predicted based on the moving speed and moving direction according to the elapsed time from the detection time of each detection target to the transmission request time of the second vessel, and output data including the predicted position may be generated and transmitted (position prediction processing). In this way, the control unit of the information processing device can generate information corresponding to the time information included in the transmission request information and transmit it to the second vessel.

[0034] The control unit 120 may generate output data that reflects the detection target information on a pre-stored map, and transmit the output data to the second vessel in the transmission process. In this case, the map data may be displayed on a display unit such as a monitor installed on the second vessel, and the position of the detection target may be easily grasped. Note that the map data may display not only the position information of the detection target, but also various information such as type, size, moving direction, moving speed, etc., as text or figures (icons, etc.). This makes it possible to confirm at a glance where and what type of detection target is present. Also, map data similar to the output data may be generated by reflecting the position information, etc. included in the detection target information received from the information processing device on the map data pre-stored in the second vessel.

[0035] In this embodiment, the detection target information includes speed information (moving speed) and heading information (moving heading) of the detection target, and the control unit may generate output data including predicted position information of the detection target predicted based on the speed information and heading information. This makes it possible to transmit the movement of the detection target to the second ship, thereby improving safety and increasing the navigation efficiency of the second ship. That is, for example, if the movement direction of the detection target is a direction away from the second ship, there is no need to forcibly avoid the detection target, and there is no need to make an unnecessary detour, so the movement efficiency of the second ship can be improved.

[0036] In this embodiment, the control unit may further execute a matching process to integrate overlapping detection target information by comparing the multiple received detection target information, and transmit output data based on the information after the matching process to the second ship. With this configuration, for example, when the same detection target is detected by multiple first ships, it is possible to avoid the risk of misidentifying that there are multiple detection targets. Information regarding the conditions of the matching process is stored in advance in the storage unit.

[0037] In this embodiment, the control unit may determine that two detection targets are the same detection target when the position information of the two detection targets is within a predetermined distance in the matching process. For example, the control unit may determine that the detection targets are the same detection target when the position information of "B0001" and "B0005" in Fig. 4 is within a predetermined distance range. In this case, the control unit may preferentially leave "B0005" having the latest time information, and delete and update "B0001".

[0038] In this embodiment, the control unit may determine that the two detection targets are the same detection target when the position information and type information of the detection targets satisfy a predetermined condition in the matching process. For example, the control unit may determine that the detection targets are the same detection target when the types of "B0001" and "B0005" in FIG. 4 are the same and the position information is within a predetermined distance range. In this case, the control unit may preferentially leave "B0005" with the latest time information and delete "B0001" to update it. By adding not only the position information but also the type information to the matching conditions, a more accurate matching process is possible. It is possible to add any information such as size information, moving speed, moving direction, etc. to the conditions, and the combinations thereof can be changed arbitrarily.

[0039] In the present embodiment, the control unit may store the received position information of the detection target in association with time information in the storage process. The time information may include, for example, as shown in Fig. 4, either or both of the detection time when the first vessel detected the detection target and the reception time when the information processing device received the detection target information.

[0040] In this embodiment, the detection target may include floating objects, people, fixed objects, and other objects. In this way, it is possible to obtain information about obstacles that cannot be obtained by AIS, etc.

[0041] In this embodiment, an information processing method is provided in which an information processing device performs a receiving process to receive detection target information from multiple sailing first ships, the detection target information including position information of a specified detection target detected by a detection unit of each first ship, a storage process to consolidate and store the received detection target information, and a transmission process to transmit at least one of the received detection target information and specified output data generated using the detection target information under predetermined generation conditions to a specified second ship.

[0042] In this embodiment, a program is provided that causes a control unit to execute a receiving process for receiving detection target information from multiple sailing first ships, including position information of a specified detection target detected by the detection unit of each first ship, a storage process for aggregating and storing the received detection target information, and a transmission process for transmitting at least one of the received detection target information and specified output data generated using the detection target information under predetermined generation conditions to a specified second ship.

[0043] In this embodiment, the detection target information may include information (including error information) on the accuracy of the detection unit. For example, the accuracy information is expressed as 0 to 100% (or 0 to 1), and the control unit of the information processing device can store the accuracy information and generate output information including the accuracy information. For example, when detection target information with different accuracy information is obtained for the position of the same detection target, the position information with the higher accuracy value may be adopted and stored, or weighting corresponding to the accuracy value may be performed and a position between multiple detection target information may be calculated as the position information of the detection target. For example, based on first position (x1, y1) information with a sensor detection accuracy of 0.6 and second position (x2, y2) information with a sensor detection accuracy of 0.3, the control unit may estimate, as the position of the detection target, position coordinates that are closer to the first position than the midpoint between the first position and the second position by the ratio of the difference in accuracy. Such estimation conditions are stored in the storage unit in advance.

[0044] In this embodiment, the detection target information may include information on the type (category) of the detection unit such as a sensor or a camera. The control unit of the information processing device can estimate the accuracy from the information on the type of the detection unit. Then, output information can be generated based on the accuracy in the same manner as described above.

[0045] In this embodiment, the information transmitted from the first vessel may include wind speed and direction, estimated wave height, wavelength, tidal current, etc., obtained by the first vessel (own vessel), and this information can also be shared by transmitting it to the second vessel. The second vessel can obtain various detection information that cannot be obtained by the own vessel, enabling safer and more efficient navigation planning.

[0046] In this embodiment, the detection target information is not limited to the first ship, and may be information detected by a sensor installed on land such as a harbor or a fixed buoy, or collected from the air by a mobile object such as a drone. In other words, the information can be received from any device equipped with a detection unit and a transmission unit, and shared (transmitted) to the second ship, etc.

[0047] In this embodiment, the second vessel may be only a vessel that does not have or does not function as a detection unit for other vessels, etc. This can improve safety, particularly for vessels that cannot detect the positions of other vessels, etc.

[0048] In this embodiment, information such as the current position, destination position, intermediate positions, the estimated time of arrival at each position, and the navigation route may be received from the first ship and transmitted to the second ship.

[0049] Here, an example of a first ship (e.g., ships A1 and A2) capable of communicating with an information processing device will be described. The second ship may have the same functions as the first ship. That is, it may be possible to display information on the object to be detected received from the information processing device, and generate a travel route based on the object to be detected.

[0050] The first vessel is equipped with a detection unit such as a sensor and a camera, and can detect the position of a surrounding object (detection target). For example, the vessel is equipped with a camera, and by analyzing an image captured by the camera, the vessel can estimate the relative position (three-dimensional coordinates in a camera coordinate system with the camera as the origin) of the vessel, floating objects, people, etc. included in the image relative to the vessel (e.g., the camera) using a known technique. The vessel also has a vessel position estimation function, and can estimate the position of the detection target in real space (world coordinate system) based on the vessel's position information and the position information of the target relative to the vessel. The detection target position estimation may be performed using other known methods. The vessel's position information may be estimated as the current position using coordinate information as positioning data acquired by a GNSS module, for example, or may be estimated by a self-position estimation process based on a comparison between map data (three-dimensional point cloud data) acquired from Lidar and map information stored in a storage unit.

[0051] The vessel can store the information detected by the detection unit in association with information on the time of detection, and can transmit the detection information and the detection time information to the information processing device.

[0052] The ship may receive and acquire various information including the ship's own position information (ship type, size, attitude, movement speed, movement direction, route information, etc.) of other ships, and may transmit the position information of the other ships to the information processing device. In this way, not only the detection information but also the received position information of the other ships can be transmitted to the information processing device.

[0053] FIG. 5 shows an example of an obstacle estimation system (hereinafter also simply referred to as the "system") provided on a first vessel according to this embodiment. The system of this embodiment can be installed on any type of vessel, and is particularly suitable for relatively small vessels. Specifically, the system of this embodiment can be adopted in fishing vessels, water taxis, small multipurpose vessels, recreational fishing boats, passenger vessels, transportation vessels, work vessels, fireboats / patrol boats, pleasure yachts, pleasure motorboats, special work vessels, etc., and can also be adopted in large passenger vessels, tankers, cargo ships, etc.

[0054] The system of this embodiment is an obstacle estimation system 1 for detecting obstacles (detection targets) when a ship navigates. The system 1 includes, for example, a data acquisition unit 10, a control unit 20, a memory unit 30, an input unit 40, an output unit 50, a communication unit 60, and a ship operation control unit 70. Each component is connected by wire or wirelessly and can communicate with each other. In this example, the control unit 20, the memory unit 30, the input unit 40, the output unit 50, and the communication unit 60 are implemented in an information processing device (computer), but this is not limited to the above and each component may be independent.

[0055] The data acquisition unit 10 includes a plurality of data acquisition means such as various sensors and cameras, and acquires various sensor data and image data. The data acquisition unit may include, for example, a camera, a radar device such as Lidar (Light detection and ranging), a marine radar, or a millimeter wave radar, an ultrasonic distance measurement sensor, a GNSS module, a QZSS module, or other positioning device, an AIS, an inertial measurement unit (IMU), an inertial sensor, a wind direction sensor, a wind speed sensor, a speed sensor (ground speed sensor, water speed sensor), a direction sensor, an acceleration sensor, a gyro sensor, a magnetic compass, a satellite compass, a temperature sensor, a humidity sensor, a pressure sensor, an altitude sensor, an infrared sensor, or the like. The data acquisition unit 10 can acquire information on the current position (coordinates), attitude (heading such as bow direction), moving speed, moving direction, bow direction, and turning amount of the ship, information on the surrounding environment, and information on obstacles such as other ships and people. The AIS is a system for transmitting and receiving various information between ships or between a ship and a land facility, and can exchange ship information such as the position, attitude (course), moving speed, and destination of each ship by wireless communication. The direction sensor may be a magnetic direction sensor that calculates the heading direction using geomagnetism, a gyrocompass, a GPS compass, or the like.

[0056] The control unit 20 includes a processor (arithmetic device) such as a CPU, and is configured to be able to execute various information processes based on programs stored in the storage unit 30. The control unit 20 can execute ship information estimation processing to estimate ship position information and ship attitude information based on data (first data) from the data acquisition unit 10. The ship position information can be estimated using coordinate information as positioning data acquired by the GNSS module as the current position. The ship position information may be estimated by a self-position estimation processing based on a comparison between map data (three-dimensional point cloud data) acquired from Lidar and map information stored in the storage unit. The ship position information may be expressed by two-dimensional or three-dimensional coordinate information on a specific coordinate system, and may be expressed by latitude and longitude. The attitude of the ship is expressed by orientation (angle) information in a specific two-dimensional or three-dimensional coordinate system. The attitude information may be expressed by east-west-north-south orientation.

[0057] The control unit 20 can execute a ship information correction process to correct the ship position information and the ship attitude information based on the data (second data) from the data acquisition unit 10. The control unit 20 can correct the ship position information and the ship attitude information based on speed data acquired from, for example, Lidar or a camera.

[0058] The control unit 20 can execute a speed estimation process to estimate the ship's speed based on data from the data acquisition unit 10. For example, the control unit 20 can calculate the distance traveled in a given time from position information from a given time ago (0.01 seconds, 0.1 seconds, 1 second, etc.) and current position information, and estimate the speed from the information on the distance traveled in the given time.

[0059] The control unit 20 can execute an obstacle information estimation process to estimate the position and size of an obstacle based on the ship's position information and ship's attitude information corrected by the ship's information correction process, and the data (third data) from the data acquisition unit 10. The control unit 20 can also estimate the type of obstacle based on the ship's position information and ship's attitude information corrected by the ship's information correction process, and the data (third data) from the data acquisition unit 10.

[0060] The control unit 20 can estimate the type of obstacle by, for example, selecting one of candidates of obstacle types stored in advance in the storage unit 30. The information on the type of obstacle can include type information such as ships, specific ship classifications (fishing boats, water taxis, small multipurpose boats, recreational fishing boats, passenger boats, transportation boats, work boats, fireboats / patrol boats, pleasure yachts, pleasure motorboats, special work boats, etc.), marine structures, people (including swimming, surfing, yachts, paddle boards, etc.), rocks, shores, driftwood, seaweed, marine life, etc. The information on the type of obstacle is stored in association with size information (area, height).

[0061] The control unit 20 can estimate the position, posture, type, and size of the obstacle by, for example, image analysis of image data acquired by the camera. The control unit 20 may estimate the position, posture, type, and size of the obstacle by using a machine learning technique using a learning model that has been trained in advance in the obstacle information estimation process. When the control unit 20 estimates the position of the obstacle based on the image data acquired by the camera, after estimating the three-dimensional position (coordinates) of the obstacle in the camera coordinate system, the control unit 20 can convert the camera coordinate system into a world coordinate system by a coordinate conversion process based on the position (coordinates) of the camera and the information on the shooting direction, and calculate the position (coordinates) of the obstacle on the world coordinate system. The control unit 20 can estimate the bow direction of the ship as an obstacle, the direction of a person's face (front direction), and the direction (traveling direction) of an object used such as a surfboard by image analysis of the image data from the camera or analysis of three-dimensional data (point cloud data, model data, etc.). The control unit 20 can estimate the three-dimensional position and attitude of an obstacle on or underwater based on sensor data from any one or a combination of sensors such as a radar device, Lidar, etc., not limited to image data from a camera as described above.

[0062] The storage unit 30 may store candidates for types of obstacles and values ​​of sizes (information on area and height in plan view) corresponding to each type, and size ranges (upper limit and lower limit) in association with each other in advance. For example, the range of the area of ​​a ship may be 3 m^2 or more and 9200 m^2 or less, and the range of the height of the ship (height above the water surface) may be 0.5 m or more and 100 m or less. The range of the area of ​​a person may be 0.2 m^2 or more and 5 m^2 or less, and the range of the height of a person (height above the water surface) may be 0.1 m or more and 3 m or less. The storage unit 30 stores information on the appropriate avoidance distance associated with obstacle information such as the position, type, and size of the obstacle. The storage unit 30 may also store information on a calculation formula for calculating the appropriate avoidance distance. For example, the appropriate avoidance distance may be calculated according to the size (area in plan view) of the ship as an obstacle. The parameters used in the calculation may include information on the type of obstacle, the moving speed, the moving direction, and the like. The memory unit 30 may contain information about a program that simulates movement of the ship along its navigation route (planned route) and whether or not it will come into contact with an obstacle. The memory unit 30 stores information about each data acquisition means. The information about the data acquisition means may be, for example, the position of the data acquisition means on the ship (e.g., a relative position with respect to a specific point such as the center or center of gravity of the ship), a relative attitude (e.g., a direction relative to the bow direction of the ship), etc., but is not limited to this.

[0063] The control unit 20 may estimate the moving speed, moving direction, and attitude of an obstacle based on the ship's position information and ship's attitude information corrected in the ship's information correction process, the ship's moving speed information, moving direction information, and data (third data) from the data acquisition unit 10. The control unit 20 can acquire moving speed information and moving direction information of obstacles such as other ships relative to the ship, based on data from Lidar. The control unit 20 can also acquire moving speed information, moving direction information, etc. of obstacles using data from a radar device.

[0064] In the obstacle information estimation process, the control unit 20 can analyze image data acquired by the camera to estimate information such as the type, size, position, shape, and color of the obstacle. The control unit 20 may estimate the information of the obstacle based on any information of the obstacle estimated from the image data and the obstacle-related information stored in advance in the storage unit. For example, the control unit 20 may estimate the size of the obstacle from the image data, and determine (estimate) an obstacle candidate associated with a size range that matches the size as the type of the obstacle by referring to the storage unit. The control unit 20 may also analyze image data acquired by the camera to estimate the type of the obstacle, and determine (estimate) a size candidate associated with the type as the size of the obstacle by referring to the storage unit. The control unit 20 may also estimate any one of the type, size, and position of the obstacle based on any one or a combination of detection data acquired from a radar device and detection data from Lidar, without being limited to image data acquired by the camera. For example, three-dimensional data of an obstacle can be generated from distance data to multiple points of the obstacle by Lidar, and the three-dimensional shape, type, size, position, etc. of the obstacle can be estimated based on the three-dimensional data of the obstacle. In this way, the control unit 20 can select from among the obstacle candidates by comparing the information of the obstacle candidates stored in the storage unit 30 with the data on the obstacle acquired from the data acquisition unit 10, and estimate the type, size, etc. of the detected obstacle. The control unit 20 can also estimate the position and size of the obstacle based only on the data from the data acquisition unit 10.

[0065] The control unit 20 can store various information such as generated information in the storage unit 30, update the information in the storage unit 30, output the information from the output unit 50, and transmit the information to an external device, another ship, etc. via the communication unit 60. The control unit 20 can also accept information acquired by the data acquisition unit 10, user operation information input via the input unit 40, and information received by the communication unit 60, and execute information processing based on the information.

[0066] The memory unit 30 can store various types of information stored in advance, information acquired from the data acquisition unit 10, information received from external devices or other ships via the communication unit 60, information generated by the control unit 20, information input via the input unit 40, etc., and is composed of a non-volatile memory, a hard disk, etc. The memory unit 30 can store information on various parameters indicating the shape of the ship (three-dimensional model data, etc.), characteristics (characteristics related to output, speed, and turning amount), information on obstacles, two-dimensional or three-dimensional map information, navigation history information, navigation plan information, tidal information, etc.

[0067] The input unit 40 receives input information based on operations from a user, etc. The input unit 40 is composed of mechanical buttons, switches, operation levers, touch panels, etc. The input unit 40 may also include a microphone that allows voice input.

[0068] The output unit 50 outputs various information as images (video), sounds, etc. The output unit 50 may include, for example, a display unit such as a liquid crystal monitor or a touch panel that displays images, an audio output unit such as a speaker that displays sounds, a vibration generating unit (vibration device) that generates vibrations, etc.

[0069] The communication unit 60 is connected to a network such as the Internet or wireless communication, and can transmit data to external devices (management servers, smartphones and tablet terminals used by users, etc.) and other ships, and receive data from external devices.

[0070] The vessel operation control unit 70 may include an auto-rudder that controls steering, an auto-throttle that controls a power unit (engine, motor, etc.) for rotating a propeller or thruster, etc. The vessel operation control unit 70 controls the operation of the vessel 70 (movements such as forward movement, backward movement, left / right movement, turning, etc.) based on instruction information from at least one of the control unit 20, the memory unit 30, the input unit 40, and the communication unit 60, and can navigate the vessel along a predetermined route.

[0071] Fig. 6 shows an example of the present system installed on a ship A. In the example of Fig. 6, the system includes a Lidar, a camera, a GNSS antenna, an AIS antenna, and a wind speed and direction meter as the data acquisition unit 10, a control unit as the control unit 20, a monitor as the output unit 50, and an auto-rudder and an auto-throttle as the ship operation control unit 70. In the example of Fig. 6, the Lidar and the camera are installed on the front side (bow side) and the rear side (stern side) of the ship 100, respectively, so that data around the ship 100, particularly the front and rear sides, can be detected with high accuracy. The configuration of the present system installed on the ship is not limited to the illustrated example, and other sensors and the like may be included.

[0072] FIG. 7 shows an example of a process flow relating to a symmetry detection method in the system of this embodiment.

[0073] The control unit 20 executes an own ship information estimation process (S201) for estimating own ship position information and own ship attitude information based on first data acquired from at least one data acquisition means, an own ship information correction process (S202) for correcting own ship position information and own ship attitude information based on second data acquired from at least one data acquisition means, an obstacle information estimation process (S203) for estimating the position and size of an obstacle based on the own ship position information and own ship attitude information corrected in the own ship information correction process and third data acquired from at least one data acquisition means, and an output information generation process (S204) for generating output information based on the information on the position and size of the obstacle estimated in the obstacle information estimation process. The first data, the second data, and the third data may each include data acquired from a common data acquisition means. Also, the first data, the second data, and the third data may each include data acquired from one data acquisition means or from multiple data acquisition means. The ship information correction process is not essential, and the obstacle information estimation process may be performed after the ship information estimation process.

[0074] In the ship information estimation process (S201), for example, the ship's three-dimensional position is estimated based on coordinate information from a GNSS antenna as a data acquisition means, and the ship's three-dimensional attitude information is estimated based on orientation data acquired from an inertial measurement unit. The ship information estimation process (S201) may be performed based on data acquired from only one data acquisition means, but from the viewpoint of improving estimation accuracy, it is preferable to perform the estimation process based on data acquired from multiple data acquisition means.

[0075] In the ship information correction process (S202), the ship position information and ship attitude information estimated in S101 are corrected based on, for example, speed information acquired from at least one of Lidar and a camera. The control unit 20 estimates the current position and attitude (a predetermined time after the time) using, for example, the position (coordinates), movement direction information, and movement speed information at a time point a predetermined time ago (0.01 seconds, 0.1 seconds, 1 second, etc.). The estimated position and attitude information is then compared with the ship position information and ship attitude information estimated in the ship information estimation process (S201). If they do not match, the ship position information and ship attitude information can be corrected by adopting the numerical value between the two as accurate current position and current attitude information. Note that the ship information correction process may be performed by another method, and for example, the ship position information and ship attitude information may be corrected using the position information and attitude information calculated by the ship position and attitude estimation process based on image data from a camera, or a combination of these. Moreover, from the viewpoint of increasing the accuracy of the correction process, it is preferable to execute the correction process based on data acquired from a plurality of data acquisition means.

[0076] Next, in the obstacle information estimation process (S203), for example, the position and attitude of the camera are estimated (the position and attitude of the camera in the ship's coordinate system) based on the ship's position information and ship's attitude information corrected in the ship information correction process, and the position of the obstacle in the world coordinate system is estimated by coordinate conversion using the relative position data of the obstacle in the camera coordinate system. In addition, the type and size of the obstacle are estimated by analyzing the image data of the camera. The position, type and size of the obstacle may be estimated using detection data acquired from the radar device alone or in combination.

[0077] Then, in the output information generation process (S204), output information including various information such as the position, type, moving speed, moving direction, and size of the obstacle can be generated and transmitted to the information processing device 100. The transmission process may be repeatedly executed at pre-stored intervals, or may be transmitted each time a new object is detected. The output information may include image data or audio data, and may display the position of the obstacle on a map on the ship's monitor, display the type and size of the obstacle in text or image, or output audio from a speaker to alert the ship. The content of the output information is not particularly limited, and may be, for example, navigation route information for avoiding obstacles and navigating the ship, and may be generated and transmitted to the information processing device 100 or the like, displayed on a monitor, or applied to an autopilot system for automatic navigation. The output information may include the current position, the destination position, the waypoint, and the scheduled time of arrival at the destination position and the waypoint.

[0078] The output information may include information on the distance (appropriate avoidance distance) to be kept away from the obstacle. For example, if the size of the obstacle is small (such as 1 m^2 or less), the appropriate avoidance distance may be set small (5 m, 10 m, etc.), and the larger the size of the obstacle (such as 10 m^2 or more), the larger the appropriate avoidance distance may be set (20 m, 50 m, etc.). For example, if the obstacle is a person, the appropriate avoidance distance may be set large (20 m, 50 m, etc.), and if the obstacle is a floating object such as a buoy, driftwood, or seaweed, the appropriate avoidance distance may be set small (5 m, 10 m, etc.). The appropriate avoidance distance may be determined based on appropriate avoidance distance information pre-stored in the storage unit according to at least one of the type and size of the obstacle. In this case, information on the type and size of the obstacle and information on the appropriate avoidance distance are pre-stored in association with each other. If the obstacle is another ship, the appropriate avoidance distance may be calculated based on the moving speed and moving direction of the other ship.

[0079] In this embodiment, the control unit 20 may perform various processes based on information (including error information) on the accuracy (precision) of the detection unit (sensor, camera, etc.) stored in advance in the storage unit. For example, in the ship position estimation process, only information from a highly accurate sensor with an accuracy value equal to or greater than a predetermined threshold may be used preferentially (information from a low-accuracy sensor with an accuracy value less than the threshold is not used), or detection information may be integrated and used based on weighting previously stored in the storage unit. For example, the accuracy information is expressed as 0 to 100% (may be expressed as a decimal between 0 and 1), and the control unit of the information processing device can store or update the accuracy information based on information input by the user. The various processes may include at least one of a process of estimating ship position information, ship attitude information, a process of correcting information, and a process of estimating obstacles. In this way, by using a detection unit with high accuracy preferentially for information processing, the accuracy of various processes such as ship position, ship attitude, and obstacle detection can be improved. For example, when the control unit 20 has a plurality of different sensors capable of estimating the ship's position information, the control unit 20 can estimate a position determined based on the weighting of the accuracy information between positions detected by the plurality of sensors. When detection target information with different accuracy information is obtained for the position of the same detection target among the plurality of detected (estimated) ship's position information, the position information with the higher accuracy value may be adopted and stored, or a weighting corresponding to the accuracy value may be performed and a position between the plurality of detection target information may be calculated as the position information of the detection target. For example, based on first position (x1, y1) information with a sensor detection accuracy of 0.6 and second position (x2, y2) information with a sensor detection accuracy of 0.3, the control unit may estimate the position coordinates of the detection target as the position coordinates that are closer to the first position than the midpoint between the first position and the second position by the ratio of the difference in accuracy. Such estimation conditions are stored in the storage unit in advance.

[0080] Although the preferred embodiment of the present disclosure has been described in detail above with reference to the attached drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person having ordinary knowledge in the technical field of the present disclosure can conceive of various modified or amended examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally belong to the technical scope of the present disclosure.

[0081] The device described in this specification may be realized as a single device, or may be realized by a plurality of devices (e.g., cloud servers) partially or entirely connected via a network. For example, the control unit and the storage unit may be realized by different servers connected to each other via a network.

[0082] The series of processes performed by the device described in this specification may be realized using any of software, hardware, and a combination of software and hardware. A computer program for realizing each function of the control unit according to this embodiment may be created and implemented in a PC or the like. A computer-readable recording medium in which such a computer program is stored may also be provided. The recording medium may be, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, or the like. The above computer program may also be distributed, for example, via a network, without using a recording medium.

[0083] In addition, the processes described herein using flowchart diagrams do not necessarily have to be performed in the order shown. Some process steps may be performed in parallel. Additional process steps may be employed, and some process steps may be omitted.

[0084] In addition, the effects described in this specification are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that are apparent to a person skilled in the art from the description of this specification, in addition to or in place of the above effects.

[0085] Note that the following configurations also fall within the technical scope of the present disclosure. (Item 1) A receiving process for receiving detection target information from a plurality of sailing first ships, the detection target information including position information of a predetermined detection target detected by a detection unit of each of the first ships; A storage process of aggregating and storing the received detection target information; a control unit that executes a transmission process to transmit, to a specified second ship, at least one of the received detection target information and specified output data generated under predetermined generation conditions using the detection target information. (Item 2) Item 2. The information processing device described in item 1, wherein the control unit transmits at least one of the detection target information and the output data in response to transmission request information received from the second ship. (Item 3) the transmission request information includes the second vessel's own ship position information, The control unit is An information processing device as described in item 2, which transmits information about the detection target located within a predetermined distance range from the second ship's own position to the second ship. (Item 4) The control unit generates output data that reflects the detection target information on a pre-stored map, and transmits the output data to the second ship in the transmission process. (Item 5) The detection target information includes speed information and traveling direction information of the detection target, 2. The information processing device according to item 1, wherein the control unit generates the output data including predicted position information of the detection target predicted based on the speed information and traveling direction information. (Item 6) The control unit further includes: A matching process is performed to integrate overlapping pieces of the detection target information by comparing the received pieces of detection target information; An information processing device according to item 1, which transmits output data based on the information after the matching process to a second vessel. (Item 7) The control unit is 7. The information processing device according to item 6, wherein in the matching process, when the position information of two detection targets is within a predetermined distance, the detection targets are determined to be the same detection target. (Item 8) The control unit is 7. The information processing device according to item 6, wherein in the matching process, when position information of two detection targets and information on the type of the detection targets satisfy a predetermined condition, the detection targets are determined to be the same detection target. (Item 9) 2. The information processing device according to item 1, wherein the control unit, in the storage process, stores the received position information of the detection target in association with time information. (Item 10) 2. The information processing device according to item 1, wherein the detection target includes other ships. (Item 11) 2. The information processing device according to item 1, wherein the detection target includes floating matter. (Item 12) 2. The information processing device according to item 1, wherein the detection target includes a person. (Item 13) A receiving process for receiving detection target information from a plurality of sailing first ships, the detection target information including position information of a predetermined detection target detected by a detection unit of each of the first ships; A storage process of aggregating and storing the received detection target information; An information processing method in which an information processing device executes a transmission process to transmit at least one of the received detection target information and specified output data generated using the detection target information under predetermined generation conditions to a specified second ship. (Item 14) A receiving process for receiving detection target information from a plurality of sailing first ships, the detection target information including position information of a predetermined detection target detected by a detection unit of each of the first ships; A storage process of aggregating and storing the received detection target information; A program that causes the control unit to execute a transmission process to transmit at least one of the received detection target information and specified output data generated using the detection target information under predetermined generation conditions to a specified second ship. [Explanation of symbols]

[0086] 1. Travel Route Generation System 10 Data Acquisition Section 20 Control section 30 Storage section 40 Input section 50 Output section 60 Communications Department 70 Ship motion control unit

Claims

1. a receiving process for receiving detection target information from a plurality of sailing first ships, the detection target information including position information of a predetermined detection target detected by a detection unit of each of the first ships; A storage process of aggregating and storing the received detection target information; a control unit that executes a transmission process of transmitting at least one of the received detection target information and predetermined output data generated under predetermined generation conditions using the detection target information to a predetermined second ship, The control unit transmits at least one of the detection target information and the output data in response to transmission request information received from the second vessel, the transmission request information includes ship position information of the second ship, The control unit is An information processing device that transmits information about the detection target located within a predetermined distance range from the second ship's own position to the second ship.

2. The information processing device according to claim 1 , wherein the control unit generates output data that reflects the detection target information on a pre-stored map, and transmits the output data to the second ship in the transmission process.

3. The detection target information includes speed information and traveling direction information of the detection target, The information processing device according to claim 1 , wherein the control unit generates the output data including predicted position information of the detection target predicted based on the speed information and the traveling direction information.

4. The control unit further includes: A matching process is performed to integrate overlapping pieces of the detection target information by comparing the received pieces of detection target information; The information processing device according to claim 1 , further comprising: a first information processing unit configured to transmit output data based on the information after the matching process to the second vessel.

5. The control unit is The information processing apparatus according to claim 4 , wherein, in the matching process, when the position information of two detection targets is within a predetermined distance, the detection targets are determined to be the same detection target.

6. A receiving process for receiving detection target information including position information of a predetermined detection target detected by a detection unit of each of a plurality of first ships sailing along the sea; A storage process of aggregating and storing the received detection target information; a control unit that executes a transmission process of transmitting at least one of the received detection target information and predetermined output data generated under predetermined generation conditions using the detection target information to a predetermined second ship, The control unit further includes: A matching process is performed to integrate overlapping pieces of the detection target information by comparing the received pieces of detection target information; Transmitting output data based on the information after the matching process to the second vessel; The control unit is The information processing device determines that two detection targets are the same detection target when position information of the two detection targets and information on types of the detection targets satisfy a predetermined condition in the matching process.

7. The information processing device according to claim 1 , wherein the control unit, in the storage process, stores the received position information of the detection target in association with time information.

8. The information processing device according to claim 1 , wherein the detection target includes another ship.

9. The information processing device according to claim 1 , wherein the detection target includes a floating object.

10. The information processing device according to claim 1 , wherein the detection target includes a person.

11. a receiving process for receiving detection target information from a plurality of sailing first ships, the detection target information including position information of a predetermined detection target detected by a detection unit of each of the first ships; A storage process of aggregating and storing the received detection target information; a transmission process of transmitting at least one of the received detection target information and predetermined output data generated under predetermined generation conditions using the detection target information to a predetermined second ship; the control unit of the information processing device transmits at least one of the detection target information and the output data in response to transmission request information received from the second vessel, the transmission request information includes ship position information of the second ship, The control unit is An information processing method for transmitting information about the detection target located within a predetermined distance range from the second ship's own position to the second ship.

12. a receiving process for receiving detection target information from a plurality of sailing first ships, the detection target information including position information of a predetermined detection target detected by a detection unit of each of the first ships; A storage process of aggregating and storing the received detection target information; a transmission process of transmitting at least one of the received detection target information and predetermined output data generated under predetermined generation conditions using the detection target information to a predetermined second vessel; The control unit transmits at least one of the detection target information and the output data in response to transmission request information received from the second vessel, the transmission request information includes ship position information of the second ship, The control unit is A program that transmits information about the detection target located within a predetermined distance range from the second ship's own position to the second ship.

Images