An intelligent recognition system and method based on remote sensing image features

Abstract

The present invention discloses an intelligent recognition system and method based on remote sensing image features, which belongs to the field of image recognition and processing technology, and includes an image acquisition module, a ship size feature recognition module, a ship related information acquisition module, an alarm module and an early warning module. The present invention can use the optical camera and the depth camera set on the drone to simultaneously shoot the ship below, and obtain the optical image of the ship taken from above and the depth image of the ship taken from above; and can conveniently judge whether the ship is in an overloaded state based on the above two images, and then alarm the ship in the overloaded state, and can conveniently judge whether the ship has the risk of leaving the navigation area, and then warn the ship that is judged to have the risk of leaving the navigation area; realize the intelligent management of the ship's operating status based on remote sensing images.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition and processing, and in particular to an intelligent recognition system and method based on remote sensing image features. Background Art

[0002] Remote sensing images are data on the earth's surface or in the atmosphere obtained through remote sensing technology and presented on a computer screen in the form of digital images. They are mainly photographed or recorded by satellites, aircraft, drones or other remote sensing platforms, capturing different features of the earth's surface and its reflection or radiation information in different spectral bands. With the continuous advancement of remote sensing technology and the widespread application of remote sensing platforms such as satellites and drones, the acquisition of remote sensing images will become more convenient and efficient. At the same time, with the integration of technologies such as big data and artificial intelligence, the processing and analysis capabilities of remote sensing images will be further enhanced, providing strong support for research and applications in more fields.

[0003] Remote sensing images can be used to identify a variety of objects and phenomena on the earth's surface. For example, through deep learning and machine learning algorithms, the system can identify specific targets in remote sensing images, such as ships, vehicles, oil and gas pipelines, etc. This has important application value in fields such as ocean monitoring, traffic management, and energy security. With the development of deep learning technology, it can automatically learn features in images and perform classification and recognition, which improves the accuracy and efficiency of recognition. In summary, remote sensing images have broad application prospects and important social value. Through continuous technological innovation and optimization of algorithms, it can better serve the research and application needs in various fields.

[0004] How to better manage the operation status of ships through remote sensing images is an urgent problem to be solved. To this end, an intelligent recognition system and method based on remote sensing image features are proposed. Summary of the invention

[0005] The technical problem to be solved by the present invention is how to better manage the operation status of a ship through remote sensing images, and provides an intelligent recognition system based on remote sensing image features.

[0006] The present invention solves the above technical problems through the following technical solutions, which include an image acquisition module, a ship size feature recognition module, a ship related information acquisition module, an alarm module and an early warning module;

[0007] The image acquisition module is used to use the optical camera and the depth camera provided on the drone to simultaneously photograph the vessel below, obtain the optical image and the depth image of the vessel taken from above, and pre-process the two images;

[0008] The ship size feature recognition module is used to recognize the ship and the identification plate according to the pre-processed overhead optical image of the ship, and obtain the size feature information of the current ship;

[0009] The ship-related information acquisition module is used to acquire the relevant information of the current ship according to the size characteristic information of the current ship, wherein the relevant information includes the ship specification grade Gd and the minimum height Hmin above the water surface;

[0010] The alarm module is used to obtain the depth information of the pixel points at the corresponding positions in the overhead depth image of the ship according to the center point position information of the detection frames of the identification plates on the current ship in the overhead optical image of the ship, and then calculate according to the depth information and the known height information between the drone and the water surface of the river, and judge whether the current ship is in an overloaded state based on the obtained minimum height Hmin of the current ship above the water surface, and alarm the ship that is judged to be in an overloaded state;

[0011] The early warning module is used to determine whether the current ship has the risk of leaving the navigable area based on the center point position information of each identification plate detection frame on the current ship in the overhead optical image of the ship, and to issue an early warning to the ship that is judged to have the risk of leaving the navigable area.

[0012] Furthermore, the image acquisition module includes an optical image acquisition unit, a depth image acquisition unit and an image preprocessing unit; the optical image acquisition unit is used to use the optical camera set on the drone to photograph the ship below, and obtain an overhead optical image of the ship; the depth image acquisition unit is used to use the depth camera set on the drone to photograph the ship below, and obtain an overhead depth image of the ship; the image preprocessing unit is used to preprocess the overhead optical image of the ship and the overhead depth image of the ship, and the preprocessing includes noise reduction and enhancement processing.

[0013] Furthermore, the ship size feature recognition module includes a ship and identification plate detection unit and a size feature information calculation unit; the ship and identification plate detection unit is used to use the trained target detection model to sequentially identify the ship in the preprocessed overhead optical image of the ship and the four preset identification plates on the edge of the ship, and obtain the ship detection frame position information and the identification frame position information of each identification plate on the ship, wherein the four preset identification plates on the edge of the ship are on the same plane, and are respectively located at the bow position, the stern position and the two sides of the hull position of the upper edge of the ship; the size feature information calculation unit is used to calculate and obtain the size feature information of the current ship according to the detection frame position information of each identification plate on the ship.

[0014] Furthermore, the specific processing process of the ship and identification plate detection unit is as follows:

[0015] Step S11: firstly use the trained target detection model to detect the ship in the preprocessed overhead optical image of the ship, and obtain the coordinates of the upper left corner point and the lower right corner point of the ship detection frame in the image, that is, the position information of the ship detection frame;

[0016] Step S12: Then use the trained target detection model to detect the four preset identification plates on the edge of the ship in the internal area of ​​the ship detection frame, and obtain the coordinates of the upper left corner and lower right corner of each identification plate detection frame on the ship in the image, that is, the position information of each identification plate detection frame on the ship.

[0017] Furthermore, the specific processing process of the size feature information calculation unit is as follows:

[0018] Step S21: Calculate the coordinates of the center points of the four identification plate detection frames according to the coordinates of the upper left corner point and the lower right corner point of each identification plate detection frame in the image. The four center points are respectively recorded as A1, A2, B1, and B2;

[0019] Step S22: Calculate the lengths of line segment A1A2 and line segment B1B2 in the image, denoted as d1 and d2 respectively;

[0020] Step S23: Calculate the ratio of length d1 to d2, denoted as D L , the calculation formula is as follows:

[0021] D L =d1 / d2

[0022] Ratio D L That is, the size characteristic information of the current ship.

[0023] Furthermore, the ship-related information acquisition module includes a ship specification level information acquisition unit and a minimum height information acquisition unit on the water surface; the ship specification level information acquisition unit is used to search in the corresponding ratio-ship specification level database according to the size characteristic information of the current ship, and obtain the ratio D of the current ship. L The corresponding ship specification level Gd, where the ratio-ship specification level database has the current ship's ratio D preset L The correspondence with the ship specification grade Gd; the minimum height information acquisition unit on the water surface is used to obtain the corresponding minimum height Hmin on the water surface according to the ship specification grade Gd, wherein the minimum height Hmin on the water surface is within the safe load range, and the minimum vertical distance between the plane where the four preset identification plates on the edge of the ship are located and the water surface of the river is associated with the ship specification grade Gd.

[0024] Furthermore, in the alarm module, the specific processing process is as follows:

[0025] Step S31: obtaining the coordinates of the center points of the four identification plate detection frames in the overhead optical image of the ship, where the four center points are respectively denoted as A1, A2, B1, and B2;

[0026] Step S32: Read the pixel values ​​of the corresponding coordinate points in the aerial depth image of the ship according to the coordinates of the four center points, which are recorded as Xa1, Xa2, Xb1, and Xb2 respectively; the pixel values ​​Xa1, Xa2, Xb1, and Xb2 are the depth information of the pixel points at the corresponding positions;

[0027] Step S33: Calculate the average value of Xa1, Xa2, Xb1, and Xb2, denoted as Xavg, and the calculation formula is as follows:

[0028] Xavg=(Xa1+Xa2+Xb1+Xb2) / 4;

[0029] Step S34: Obtain the known height information between the drone and the river surface as the vertical distance between the depth camera and the river surface, recorded as C1; transform the average value Xavg into the world coordinates to obtain the corresponding distance value as the vertical distance C2 between the depth camera and the plane where the four preset identification plates on the edge of the ship are located;

[0030] Step S35: Calculate the difference between the vertical distances C1 and C2, recorded as Cd, and the calculation formula is as follows:

[0031] Cd=C1-C2

[0032] Step S36: Compare Cd with the minimum height Hmin above the water surface. When Cd is greater than the minimum height Hmin above the water surface, it indicates that the current vessel is not overloaded. When Cd is less than or equal to the minimum height Hmin above the water surface, it indicates that the current vessel is overloaded, and an alarm is issued to the person driving the vessel by playing voice.

[0033] Furthermore, in step S34, the known height information between the UAV and the water surface of the river is pre-detected and obtained by a laser ranging sensor provided on the UAV.

[0034] Furthermore, in the early warning module, the specific processing process is as follows:

[0035] Step S41: determining the sailing direction of the vessel;

[0036] Step S42: according to the coordinates of the shooting position of the drone in the world coordinate system, obtain the position and length information of the navigable area boundary line corresponding to the shooting position below the shooting position, where the navigable area boundary line is two parallel lines, one located on both sides of the ship;

[0037] Step S43: adding the navigable area boundary line to the ship overhead optical image processed in step S21 according to the position and length information of the navigable area boundary line and the conversion relationship between the world coordinate system and the image coordinate system of the ship overhead optical image;

[0038] Step S44: determining the coordinates of the center point of the identification plate detection frame located at the bow position according to the sailing direction of the ship, the center point being A1 or A2;

[0039] Step S45: draw perpendicular lines from the center point of the identification plate detection frame located at the bow position to the two navigation area boundary lines respectively, and obtain the length of the shorter of the two perpendicular lines, which is recorded as the boundary distance T;

[0040] Step S46: Calculate the angle between the line segment A1A2 and any boundary line of the navigation area, which is an acute angle and is recorded as angle θ;

[0041] Step S47: Compare the boundary distance T with the set boundary distance threshold T1, and compare the angle θ with the set angle threshold θ1. When the boundary distance T is less than the set boundary distance threshold T1 and the angle θ is greater than the set angle threshold θ1, it indicates that the current vessel is at risk of leaving the navigable area, and a voice warning is given to the person driving the vessel. Otherwise, it indicates that the current vessel is not at risk of leaving the navigable area.

[0042] The present invention also provides an intelligent recognition method based on remote sensing image features, which is used to better manage the operation status of a ship using the above-mentioned intelligent recognition system, and includes the following steps:

[0043] Step S1: using the optical camera and the depth camera set on the drone to simultaneously shoot the ship below, obtaining the optical image and the depth image of the ship taken from above, and pre-processing the two images;

[0044] Step S2: Identify the ship and the identification plate according to the pre-processed overhead optical image of the ship to obtain the size feature information of the current ship;

[0045] Step S3: acquiring relevant information of the current ship according to the size characteristic information of the current ship, wherein the relevant information includes the ship specification grade Gd and the minimum height above the water surface Hmin;

[0046] Step S4: according to the center point position information of each identification plate detection frame on the current ship in the overhead optical image of the ship, the depth information of the pixel point at the corresponding position in the overhead depth image of the ship is obtained, and then the depth information and the known height information between the UAV and the water surface of the river are calculated, and based on the obtained minimum height Hmin of the current ship above the water surface, whether the current ship is in an overloaded state is judged, and an alarm is issued for the ship judged to be in an overloaded state;

[0047] Step S5: judging whether the current ship is at risk of leaving the navigable area based on the center point position information of each identification plate detection frame on the current ship in the overhead optical image of the ship, and issuing an early warning to the ship judged to be at risk of leaving the navigable area.

[0048] Compared with the prior art, the present invention has the following advantages: the intelligent recognition system and method based on remote sensing image features can use the optical camera and the depth camera arranged on the unmanned aerial vehicle to simultaneously shoot the ship below, and obtain the overhead optical image of the ship and the overhead depth image of the ship; and can conveniently judge whether the ship is in an overloaded state based on the above two images, and then alarm the ship in the overloaded state, and can conveniently judge whether the ship has the risk of leaving the navigable area, and then warn the ship that is judged to have the risk of leaving the navigable area; realize the intelligent management of the ship operation status based on remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a schematic diagram of the structure of an intelligent recognition system based on remote sensing image features in an embodiment of the present invention;

[0050] Figure 2 is an example diagram of an overhead optical image of a ship in an embodiment of the present invention;

[0051] Figure 3 Schematic diagram of the relationship between the vertical distances C1 and C2 in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given, but the protection scope of the present invention is not limited to the following embodiment.

[0053] like Figure 1 As shown, this embodiment provides a technical solution: an intelligent recognition system based on remote sensing image features, comprising the following modules: an image acquisition module, a ship size feature recognition module, a ship related information acquisition module, an alarm module and an early warning module;

[0054] In this embodiment, the image acquisition module is used to use the optical camera and the depth camera provided on the drone to simultaneously photograph the vessel below, obtain the optical image of the vessel taken from above and the depth image of the vessel taken from above, and pre-process the two images;

[0055] To be more specific, the image acquisition module includes an optical image acquisition unit, a depth image acquisition unit and an image preprocessing unit; the optical image acquisition unit is used to use the optical camera set on the drone to photograph the ship below, and obtain an overhead optical image of the ship; the depth image acquisition unit is used to use the depth camera set on the drone to photograph the ship below, and obtain an overhead depth image of the ship; the image preprocessing unit is used to preprocess the overhead optical image of the ship and the overhead depth image of the ship, and the preprocessing includes noise reduction and enhancement processing.

[0056] To be more specific, in the image preprocessing unit, the noise reduction processing uses a Gaussian filter to process the image, which effectively reduces the noise in the image, and the enhancement processing uses a histogram equalization method to process the image, which effectively improves the quality of the image. After the noise reduction and enhancement processing, the image quality is higher, which is conducive to subsequent recognition work.

[0057] It should be noted that in this embodiment, the drone (remote sensing platform) uses a horizontal posture for fixed-point shooting, and uses an optical camera and a depth camera to simultaneously shoot the ship below. During the flight of the drone, the optical axes of the optical camera and the depth camera are always vertically downward, and the distance between the optical axes of the optical camera and the depth camera is small, and the impact caused in the subsequent recognition and other processing processes can be ignored. The size, resolution and other specifications of the optical image of the ship taken from above and the depth image of the ship taken from above are consistent.

[0058] It should be noted that, in the overhead optical image and the overhead depth image of the ship in the present embodiment, the ship is located in the middle area of ​​the image. During actual shooting, the drone uses the optical camera and the depth camera to take multiple shots at a fixed position to obtain multiple pairs of overhead optical images and overhead depth images of the ship. The position of the center point of the ship detection frame can be used to determine whether the ship is in the middle area of ​​the image. When the center point of the ship detection frame is within the preset coordinate range, it indicates that the ship is in the middle area of ​​the image. Then, a pair of overhead optical images and overhead depth images of the ship in which the ship is in the middle area of ​​the image are screened out as basic data for the processing process in the present embodiment.

[0059] In this embodiment, the ship size feature recognition module is used to recognize the ship and the identification plate according to the pre-processed overhead optical image of the ship to obtain the size feature information of the current ship;

[0060] More specifically, the ship size feature recognition module includes a ship and identification plate detection unit and a size feature information calculation unit; the ship and identification plate detection unit is used to use the trained target detection model to sequentially identify the ship in the preprocessed overhead optical image of the ship and four preset identification plates on the edge of the ship (circular plates with set colors, which can also be replaced by circular patterns of set colors) to obtain the ship detection frame position information and the detection frame position information of each identification plate on the ship; the size feature information calculation unit is used to calculate and obtain the size feature information of the current ship based on the detection frame position information of each identification plate on the ship.

[0061] More specifically, the target detection model is trained based on the YOLO V3 network, which can detect targets of various categories.

[0062] As more specific, the specific processing process of the ship and identification plate detection unit is as follows:

[0063] Step S11: firstly use the trained target detection model to detect the ship in the preprocessed overhead optical image of the ship, and obtain the coordinates of the upper left corner point and the lower right corner point of the ship detection frame in the image, that is, the position information of the ship detection frame;

[0064] Step S12: Then use the trained target detection model to detect the four preset identification plates on the edge of the ship in the internal area of ​​the ship detection frame, and obtain the coordinates of the upper left corner and lower right corner of each identification plate detection frame on the ship in the image, that is, the position information of each identification plate detection frame on the ship.

[0065] It should be noted that if Figure 2 As shown, four preset identification plates on the upper edge of the vessel are in the same plane, and are respectively located at the bow position, the stern position and the two sides of the hull position of the upper edge of the vessel.

[0066] As more specific, the specific processing process of the size feature information calculation unit is as follows:

[0067] Step S21: According to the coordinates of the upper left corner and the lower right corner of each sign plate detection frame in the image, the coordinates of the center points of the four sign plate detection frames are calculated. The four center points are respectively recorded as A1, A2, B1, and B2 (see Figure 2 );

[0068] Step S22: Calculate the lengths of line segment A1A2 and line segment B1B2 in the image, denoted as d1 and d2 respectively;

[0069] Step S23: Calculate the ratio of length d1 to d2, denoted as D L , the calculation formula is as follows:

[0070] D L =d1 / d2

[0071] Ratio D L That is, the size characteristic information of the current ship.

[0072] It should be noted that, in step S21, A1 and A2 are two center points close to the short side of the identification plate detection frame, and B1 and B2 are two center points close to the long side of the identification plate detection frame.

[0073] In this embodiment, the ship-related information acquisition module is used to acquire the relevant information of the current ship according to the size characteristic information of the current ship, wherein the relevant information includes the ship specification grade Gd and the minimum height Hmin above the water surface;

[0074] More specifically, the ship-related information acquisition module includes a ship specification level information acquisition unit and a minimum height information acquisition unit on the water surface; the ship specification level information acquisition unit is used to search in the corresponding ratio-ship specification level database according to the size feature information of the current ship, and obtain the ratio D of the current ship. L The corresponding ship specification level Gd; the minimum height information acquisition unit on the water surface is used to obtain the corresponding minimum height Hmin on the water surface according to the ship specification level Gd, wherein the minimum height Hmin on the water surface is the minimum vertical distance between the plane where the four preset identification plates on the edge of the ship are located and the river surface within the safe load range. When it is greater than the minimum height Hmin on the water surface, it means that the load of the ship is within the safe range, otherwise it means that the ship is overloaded.

[0075] It should be noted that the ratio value-vessel specification level database has a preset ratio value D of the current vessel. L Correspondence with ship specification level Gd. Different ship specification levels Gd are associated with the corresponding minimum height above the water surface Hmin.

[0076] In this embodiment, the alarm module is used to obtain the depth information of the pixel points at the corresponding positions in the overhead depth image of the ship according to the center point position information of the detection frames of the identification plates on the current ship in the overhead optical image of the ship, and then calculate according to the depth information and the known height information between the drone and the water surface of the river, and judge whether the current ship is in an overloaded state based on the obtained minimum height Hmin of the current ship above the water surface, and alarm the ship that is judged to be in an overloaded state;

[0077] As more specific, in the alarm module, the specific processing process is as follows:

[0078] S31: Obtain the center point coordinates of the four identification plate detection frames in the aerial optical image of the ship in step S21, where the four center points are respectively denoted as A1, A2, B1, and B2;

[0079] S32: Read the pixel values ​​of the corresponding coordinate points in the aerial depth image of the ship according to the coordinates of the four center points, and record them as Xa1, Xa2, Xb1, and Xb2 respectively; the pixel values ​​Xa1, Xa2, Xb1, and Xb2 are the depth information of the pixel points at the corresponding positions;

[0080] S33: Calculate the average value of Xa1, Xa2, Xb1, and Xb2, recorded as Xavg, and the calculation formula is as follows:

[0081] Xavg=(Xa1+Xa2+Xb1+Xb2) / 4;

[0082] S34: Obtain the known height information between the drone and the river surface as the vertical distance between the depth camera and the river surface, recorded as C1; transform the average value Xavg into the world coordinates to obtain the corresponding distance value as the vertical distance C2 between the depth camera and the plane where the four preset identification plates on the edge of the ship are located;

[0083] S35: Calculate the difference between the vertical distances C1 and C2, recorded as Cd (see Figure 3 ), the calculation formula is as follows:

[0084] Cd=C1-C2

[0085] S36: Compare Cd with the minimum height Hmin above the water surface. When Cd is greater than the minimum height Hmin above the water surface, it indicates that the current vessel is not overloaded. When Cd is less than or equal to the minimum height Hmin above the water surface, it indicates that the current vessel is overloaded, and an alarm is issued to the person driving the vessel by playing voice.

[0086] It should be noted that the height information between the known UAV and the river water surface is obtained through pre-detection by a laser ranging sensor installed on the UAV (detection is performed when there is no ship below the shooting position).

[0087] In this embodiment, the warning module is used to determine whether the current ship has a risk of leaving the navigation area based on the center point position information of each identification plate detection frame on the current ship in the overhead optical image of the ship, and to issue a warning to the ship that is judged to have a risk of leaving the navigation area;

[0088] As more specific, in the early warning module, the specific processing process is as follows:

[0089] S41: Determine the sailing direction of the vessel;

[0090] S42: according to the coordinates of the shooting position of the drone in the world coordinate system, obtaining the position and length information of the navigable area boundary line corresponding to the shooting position below the shooting position, where the navigable area boundary line is two parallel lines, respectively located on both sides of the ship;

[0091] S43: adding the navigable area boundary line to the ship overhead optical image processed in step S21 according to the position and length information of the navigable area boundary line and the conversion relationship between the world coordinate system and the image coordinate system of the ship overhead optical image;

[0092] S44: determining the coordinates of the center point of the identification plate detection frame located at the bow position according to the sailing direction of the ship, the center point being A1 or A2;

[0093] S45: Draw perpendicular lines from the center point of the identification plate detection frame located at the bow position to the two navigation area boundary lines, and obtain the length of the shorter of the two perpendicular lines, which is recorded as the boundary distance T;

[0094] S46: Calculate the angle between the line segment A1A2 and any boundary line of the navigation area, which is an acute angle and is recorded as angle θ;

[0095] S47: Compare the boundary distance T with the set boundary distance threshold T1, and compare the angle θ with the set angle threshold θ1. When the boundary distance T is less than the set boundary distance threshold T1 and the angle θ is greater than the set angle threshold θ1, it indicates that the current vessel is at risk of leaving the navigable area, and a voice warning is given to the person driving the vessel. Except for this situation, it indicates that the current vessel is not at risk of leaving the navigable area.

[0096] More specifically, in step S41, in any two adjacent frames of the overhead optical image of the ship, the target detection model is used to detect the ship in the overhead optical image of the ship, and the navigation direction of the ship can be obtained according to the change process of the position information of the ship detection frame.

[0097] To be more specific, in step S42, in the navigable area boundary line database, there is a preset correspondence between the coordinates of the drone's shooting position in the world coordinate system and the navigable area boundary line below it, and the navigable area boundary line is a navigable area boundary line of a set position and length. Therefore, by comparing and searching in the navigable area boundary line database based on the coordinates of the drone's shooting position in the world coordinate system, the position and length information of the navigable area boundary line corresponding to the shooting position below the shooting position can be obtained.

[0098] In this embodiment, an intelligent recognition method based on remote sensing image features is also provided, which is used to better manage the operation status of ships in a set area using the above-mentioned intelligent recognition system, including the following steps:

[0099] Step S1: using the optical camera and the depth camera set on the drone to simultaneously shoot the ship below, obtaining the optical image and the depth image of the ship taken from above, and pre-processing the two images;

[0100] Step S2: Identify the ship and the identification plate according to the pre-processed overhead optical image of the ship to obtain the size feature information of the current ship;

[0101] Step S3: acquiring relevant information of the current ship according to the size characteristic information of the current ship, wherein the relevant information includes the ship specification grade Gd and the minimum height above the water surface Hmin;

[0102] Step S4: according to the center point position information of each identification plate detection frame on the current ship in the overhead optical image of the ship, the depth information of the pixel point at the corresponding position in the overhead depth image of the ship is obtained, and then the depth information and the known height information between the UAV and the water surface of the river are calculated, and based on the obtained minimum height Hmin of the current ship above the water surface, whether the current ship is in an overloaded state is judged, and an alarm is issued for the ship judged to be in an overloaded state;

[0103] Step S5: judging whether the current ship is at risk of leaving the navigable area based on the center point position information of each identification plate detection frame on the current ship in the overhead optical image of the ship, and issuing an early warning to the ship judged to be at risk of leaving the navigable area.

[0104] To sum up, the intelligent recognition system and method based on remote sensing image features of the above-mentioned embodiments can use the optical camera and the depth camera set on the drone to simultaneously shoot the ship below, and obtain the overhead optical image of the ship and the overhead depth image of the ship; and can conveniently judge whether the ship is in an overloaded state based on the above two images, and then alarm the ship in an overloaded state, and at the same time can conveniently judge whether the ship has the risk of leaving the navigable area, and then warn the ship that is judged to have the risk of leaving the navigable area; realize the intelligent management of the ship's operating status based on remote sensing images.

[0105] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. An intelligent recognition system based on remote sensing image features, characterized in that: It includes an image acquisition module, a ship size feature recognition module, a ship related information acquisition module, an alarm module and an early warning module; The image acquisition module is used to use the optical camera and the depth camera provided on the drone to simultaneously photograph the vessel below, obtain the optical image and the depth image of the vessel taken from above, and pre-process the two images; The ship size feature recognition module is used to recognize the ship and the identification plate according to the pre-processed overhead optical image of the ship, and obtain the size feature information of the current ship; The ship-related information acquisition module is used to acquire the relevant information of the current ship according to the size characteristic information of the current ship, wherein the relevant information includes the ship specification grade Gd and the minimum height Hmin above the water surface; The alarm module is used to obtain the depth information of the pixel points at the corresponding positions in the overhead depth image of the ship according to the center point position information of the detection frames of the identification plates on the current ship in the overhead optical image of the ship, and then calculate according to the depth information and the known height information between the drone and the water surface of the river, and judge whether the current ship is in an overloaded state based on the obtained minimum height Hmin of the current ship above the water surface, and alarm the ship that is judged to be in an overloaded state; The warning module is used to determine whether the current ship has a risk of leaving the navigation area based on the center point position information of each identification plate detection frame on the current ship in the overhead optical image of the ship, and to issue a warning to the ship that is judged to have a risk of leaving the navigation area; The ship size feature recognition module includes a ship and identification plate detection unit and a size feature information calculation unit; the ship and identification plate detection unit is used to use the trained target detection model to sequentially identify the ship in the preprocessed ship overhead optical image and four identification plates preset on the edge of the ship, and obtain the ship detection frame position information and the identification plate detection frame position information on the ship, wherein the four identification plates preset on the edge of the ship are on the same plane, and are respectively located at the bow position, the stern position and the two sides of the hull position on the edge of the ship; the size feature information calculation unit is used to calculate and obtain the size feature information of the current ship according to the identification plate detection frame position information on the ship; The specific processing process of the ship and identification plate detection unit is as follows: Step S11: firstly use the trained target detection model to detect the ship in the preprocessed overhead optical image of the ship, and obtain the coordinates of the upper left corner point and the lower right corner point of the ship detection frame in the image, that is, the position information of the ship detection frame; Step S12: Then, the trained target detection model is used to detect four preset identification plates on the edge of the ship in the internal area of ​​the ship detection frame, and the coordinates of the upper left corner and the lower right corner of each identification plate detection frame on the ship in the image are obtained, that is, the position information of each identification plate detection frame on the ship; The specific processing process of the size feature information calculation unit is as follows: Step S21: Calculate the coordinates of the center points of the four identification plate detection frames according to the coordinates of the upper left corner point and the lower right corner point of each identification plate detection frame in the image. The four center points are respectively recorded as A1, A2, B1, and B2; Step S22: Calculate the lengths of line segment A1A2 and line segment B1B2 in the image, denoted as d1 and d2 respectively; Step S23: Calculate the ratio of length d1 to d2, denoted as D L , the calculation formula is as follows: D L =d1 / d2; Ratio D L That is, the size characteristic information of the current ship; The ship-related information acquisition module includes a ship specification level information acquisition unit and a minimum height information acquisition unit on the water surface; the ship specification level information acquisition unit is used to search in the corresponding ratio-ship specification level database according to the size characteristic information of the current ship, and obtain the ratio D of the current ship. L The corresponding ship specification level Gd, where the ratio-ship specification level database has the current ship's ratio D preset L The corresponding relationship with the ship specification level Gd; the minimum height information acquisition unit on the water surface is used to acquire the corresponding minimum height Hmin on the water surface according to the ship specification level Gd, wherein the minimum height Hmin on the water surface is the minimum vertical distance between the plane where the four preset identification plates on the edge of the ship are located and the river water surface within the safe load range, which is associated with the ship specification level Gd; In the alarm module, the specific processing process is as follows: Step S31: obtaining the coordinates of the center points of the four identification plate detection frames in the overhead optical image of the ship, where the four center points are respectively denoted as A1, A2, B1, and B2; Step S32: Read the pixel values ​​of the corresponding coordinate points in the aerial depth image of the ship according to the coordinates of the four center points, which are recorded as Xa1, Xa2, Xb1, and Xb2 respectively; the pixel values ​​Xa1, Xa2, Xb1, and Xb2 are the depth information of the pixel points at the corresponding positions; Step S33: Calculate the average value of Xa1, Xa2, Xb1, and Xb2, denoted as Xavg, and the calculation formula is as follows: Xavg=(Xa1+Xa2+Xb1+Xb2) / 4; Step S34: Obtain the known height information between the drone and the river surface as the vertical distance between the depth camera and the river surface, recorded as C1; transform the average value Xavg into the world coordinates to obtain the corresponding distance value as the vertical distance C2 between the depth camera and the plane where the four preset identification plates on the edge of the ship are located; Step S35: Calculate the difference between the vertical distances C1 and C2, recorded as Cd, and the calculation formula is as follows: Cd=C1-C2; Step S36: Compare Cd with the minimum height Hmin on the water surface. When Cd is greater than the minimum height Hmin on the water surface, it indicates that the current vessel is not overloaded. When Cd is less than or equal to the minimum height Hmin on the water surface, it indicates that the current vessel is overloaded, and an alarm is issued to the person driving the vessel by playing a voice. In the early warning module, the specific processing process is as follows: Step S41: determining the sailing direction of the vessel; Step S42: according to the coordinates of the shooting position of the drone in the world coordinate system, obtain the position and length information of the navigable area boundary line corresponding to the shooting position below the shooting position, where the navigable area boundary line is two parallel lines, one located on both sides of the ship; Step S43: adding the navigable area boundary line to the ship overhead optical image processed in step S21 according to the position and length information of the navigable area boundary line and the conversion relationship between the world coordinate system and the image coordinate system of the ship overhead optical image; Step S44: determining the coordinates of the center point of the identification plate detection frame located at the bow position according to the sailing direction of the ship, the center point being A1 or A2; Step S45: draw perpendicular lines from the center point of the identification plate detection frame located at the bow position to the two navigation area boundary lines respectively, and obtain the length of the shorter of the two perpendicular lines, which is recorded as the boundary distance T; Step S46: Calculate the angle between the line segment A1A2 and any navigable area boundary line, which is an acute angle and is recorded as angle θ; Step S47: Compare the boundary distance T with the set boundary distance threshold T1, and compare the angle θ with the set angle threshold θ1. When the boundary distance T is less than the set boundary distance threshold T1 and the angle θ is greater than the set angle threshold θ1, it indicates that the current vessel is at risk of leaving the navigable area, and a voice warning is given to the person driving the vessel. Otherwise, it indicates that the current vessel is not at risk of leaving the navigable area.

2. The intelligent recognition system based on remote sensing image features according to claim 1, characterized in that: The image acquisition module includes an optical image acquisition unit, a depth image acquisition unit and an image preprocessing unit; the optical image acquisition unit is used to use the optical camera set on the drone to shoot the ship below to obtain the optical image of the ship from above; the depth image acquisition unit is used to use the depth camera set on the drone to shoot the ship below to obtain the depth image of the ship from above; The image preprocessing unit is used to preprocess the optical image taken from above the ship and the depth image taken from above the ship, and the preprocessing includes noise reduction and enhancement processing.

3. The intelligent recognition system based on remote sensing image features according to claim 1, characterized in that: In step S34, the known height information between the UAV and the water surface of the river is pre-detected and obtained by a laser ranging sensor provided on the UAV.

4. An intelligent recognition method based on remote sensing image features, characterized in that: The method for better managing the operation status of a ship by using the intelligent identification system according to any one of claims 1 to 3 comprises the following steps: Step S1: using the optical camera and the depth camera set on the drone to simultaneously shoot the ship below, obtaining the optical image and the depth image of the ship taken from above, and pre-processing the two images; Step S2: Identify the ship and the identification plate according to the pre-processed overhead optical image of the ship to obtain the size feature information of the current ship; Step S3: acquiring relevant information of the current ship according to the size characteristic information of the current ship, wherein the relevant information includes the ship specification grade Gd and the minimum height above the water surface Hmin; Step S4: according to the center point position information of each identification plate detection frame on the current ship in the overhead optical image of the ship, the depth information of the pixel point at the corresponding position in the overhead depth image of the ship is obtained, and then the depth information and the known height information between the UAV and the water surface of the river are calculated, and based on the obtained minimum height Hmin of the current ship above the water surface, whether the current ship is in an overloaded state is judged, and an alarm is issued for the ship judged to be in an overloaded state; Step S5: judging whether the current ship is at risk of leaving the navigable area based on the center point position information of each identification plate detection frame on the current ship in the overhead optical image of the ship, and issuing an early warning to the ship judged to be at risk of leaving the navigable area.

Images