A ship load prediction system based on image recognition

Abstract

The present invention discloses a ship load prediction system based on image recognition, belonging to the technical field of image recognition, which solves the problem that the water surface has wave undulations and the ship itself also undulates, resulting in inaccurate results when the image recognizes the distance between the water surface and the waterline. It includes a wave-following image acquisition module, an empty ship signal acquisition module, and a data processing module arranged on the ship body. The number of wave-following image acquisition modules is several, and several wave-following image acquisition modules are evenly spaced. The wave-following image acquisition module continuously acquires the contact picture between the ship body and the water surface and feeds it back to the data processing module. The empty ship signal acquisition module includes a ship end and an empty end connected by communication. The number of ship ends is the same as the number of wave-following image acquisition modules, and the empty end can be suspended horizontally. The present invention provides a reference for the stable load of ships in waves, evaluates the inclination, ensures the reasonable distribution of goods, and avoids inaccurate image recognition caused by the water environment.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a ship load prediction system based on image recognition. Background Art

[0002] Ships are a type of water transportation that have played an important role in the development of human civilization since ancient times. They are mainly used for the transportation of goods and people, but are also used for a variety of purposes such as military, scientific research and entertainment. There are many types of modern ships, including cargo ships, passenger ships, tankers, container ships, warships, etc. They vary in size, shape and function, but all have the basic ability to sail on water. With the development of technology, the design of ships has become more and more advanced, with faster sailing speeds and heavier loads, while also having less impact on the environment.

[0003] Among ships, ship overload detection is an important part of ensuring water traffic safety. It involves the use of advanced technologies to monitor and evaluate whether the ship exceeds its approved load line. This detection usually includes technologies such as laser scanning, video analysis and image recognition to achieve accurate measurement of the actual load of the ship. Through these technologies, the freeboard height and draft of the ship can be automatically monitored, and whether the ship is overloaded or over-drafted can be intelligently judged, thereby effectively preventing safety accidents caused by overloading. In addition, the ship overload detection method based on video analysis technology uses the pan-tilt camera on the dock shore to capture the ship video and image information, and then analyzes and processes it through the video analysis server, and compares it with the safety management rules to determine whether the ship is overloaded. This detection method not only improves efficiency, but also reduces the interference of human factors, ensuring the accuracy and reliability of detection.

[0004] Current ship overloading detection technologies, such as the ship overloading determination method and device based on image recognition with the application number CN202410636646.2, have the following technical key points: Input the visible light image to be recognized of the target ship into the first ship prediction model to obtain the first recognition result output by the first ship prediction model; input the infrared image to be recognized of the target ship into the second ship prediction model to obtain the second recognition result output by the second ship prediction model; determine the ship type of the target ship according to the first recognition result and the second recognition result; input the visible light image to be recognized into the preset segmentation model to obtain the segmented image output by the preset segmentation model, and determine the actual draft line from the segmentation area between the ship and the water surface in the segmented image; determine the target overloading draft line of the ship type from the corresponding relationship between the preset ship type and the preset overloading draft line. In the case where the actual draft line is higher than the target overloading draft line, it is determined that the target ship is overloaded; the first ship prediction model is determined after being trained according to all visible light image samples and the ship type probability corresponding to each visible light image sample, the second ship prediction model is determined after being trained according to all infrared image samples and the ship type probability corresponding to each infrared image sample, the preset segmentation model is determined after being trained according to all infrared image samples and the segmented upper part of the ship and the segmented lower part of the ship corresponding to each infrared image sample, and the segmented upper part of the ship and the segmented lower part of the ship are distinguished according to the different temperatures of the upper part of the ship and the lower part of the ship.

[0005] The above solution can combine image recognition models to recognize visible light images and infrared images, and combine image segmentation models to recognize infrared images, thereby improving the applicability and accuracy of ship overloading determination. It mainly eliminates the influence of ambient light on image recognition. However, when a ship is parked in water, the water surface does not always remain stable. Under the influence of weather, large waves may appear on the water surface, causing the ship to also rise and fall when parked in water. In this way, not only is there wave undulation on the water surface, but the ship itself is also fluctuating, resulting in inaccurate results when the image recognizes the distance between the water surface and the draft line.

[0006] Therefore, a ship load prediction system based on image recognition is proposed to solve or alleviate the above problems. Summary of the Invention

[0007] The purpose of the present invention is to solve the deficiencies in the prior art and propose a ship load prediction system based on image recognition.

[0008] To achieve the above purpose, the present invention adopts the following technical solutions:

[0009] A ship load prediction system based on image recognition, comprising a wave-following image acquisition module, an empty ship signal acquisition module, and a data processing module arranged on the ship body. The number of the wave-following image acquisition modules is several, and the several wave-following image acquisition modules are evenly spaced. The wave-following image acquisition module continuously acquires the contact picture between the ship body and the water surface and feeds it back to the data processing module. The empty ship signal acquisition module includes a ship end and an empty end connected by communication. The number of the ship ends is the same as that of the wave-following image acquisition modules, and the empty end can be suspended horizontally. The empty end receives the ship end signal and outputs the vertical displacement of each ship end to the data processing module. The data processing module identifies the ship's draft line, inclination condition, and load based on the contact picture and the vertical displacement of each ship end.

[0010] Preferably, the wave-following image acquisition module includes a camera.

[0011] Preferably, the wave-following image acquisition module continuously acquires the contact picture between the ship body and the water surface and feeds it back to the data processing module, including the following steps:

[0012] Set several evenly spaced points on the ship body according to the number of the wave-following image acquisition modules;

[0013] Install a single wave-following image acquisition module at the points;

[0014] The wave-following image acquisition module continuously acquires the contact picture between the ship body and the water surface on the time axis and feeds it back to the data processing module.

[0015] Preferably, the ship end includes an infrared emitter, the empty end includes drones, the number of the drones is at least three, the drones include a microprocessor, an infrared receiver coupled to the microprocessor, and a wireless communication module coupled to the microprocessor. The drones are communicatively connected to the data processing module through the wireless communication module, and the drones are also communicatively connected to each other through the wireless communication module.

[0016] Preferably, the empty end receives the ship end signal and outputs the vertical displacement of each ship end to the data processing module, including the following steps:

[0017] The drones fly above the ship body and keep all the drones on the same horizontal plane;

[0018] Each drone receives the distance information from the ship end to the horizontal plane, and the distance information between each drone calculates the vertical displacement of each ship end on the time axis through the Pythagorean theorem as the vertical displacement of each point on the time axis.

[0019] Preferably, the data processing module includes a processor.

[0020] Preferably, the data processing module identifies the ship's draft line, inclination, and load based on the contact screen and the vertical displacement of each ship end, including the following steps:

[0021] Preprocess the contact screen on the time axis through Gaussian filtering;

[0022] Use an edge detection algorithm on the contact screen on the time axis to identify the boundary line between the water surface and the ship's hull;

[0023] Normalize the boundary line between the water surface and the ship's hull in the contact screen on the time axis and complete spectral analysis through Fourier transform to determine the amplitude and frequency of the water surface waves at each point;

[0024] Set the water surface waves as a sine waveform, , where is the water surface height at the i-th point at time t, is the amplitude of the water surface fluctuation at the i-th point, is the angular frequency of the water surface waves, , where is the water surface wave frequency at the i-th point, is the phase of the water surface waves;

[0025] The output result is the change in water surface height at each point on the time axis .

[0026] Preferably, the output result is the change in water surface height at each point on the time axis , including the following steps:

[0027] Calculate the spatial wave synchrony and phase difference , where λ is the wavelength, is the distance between two points;

[0028] Obtain the change in water surface height at different positions through the phase difference in space , where is the phase change caused by the spatial position difference;

[0029] The output result is the change in water surface height at each point on the time axis .

[0030] Preferably, the data processing module identifies the ship's draft line, inclination, and load based on the contact screen and the vertical displacement of each ship end, and further includes the following steps:

[0031] Input the change in water surface height at different positions on the time axis and the vertical displacement at each point on the time axis , calculate the draft depth at each point on the time axis by calculating the difference between the two .

[0032] Preferably, the data processing module identifies the ship's draft line, inclination, and load based on the contact screen and the vertical displacement of each ship end, and further includes the following steps

[0033] Input the draft depth at each point on the time axis , and calculate the average draft depth at each point ;

[0034] Establish a spatial coordinate, place the ship in the spatial coordinate, confirm the coordinates of each point in the spatial coordinate, and mark the average draft line of each point in the spatial coordinate based on the coordinates of each point and the average draft depth of each point, and then connect the lines;

[0035] Calculate the volume of the ship below the average draft line of each point after connecting the lines as the displacement;

[0036] Calculate the weight of water based on the displacement according to the density formula, and obtain the load based on the difference between the weight of water and the self-weight of the ship;

[0037] Calculate the average height of the draft lines at all points ;

[0038] Calculate the standard deviation of the draft lines ;

[0039] Set a standard threshold, compare the standard deviation of the draft lines with the standard threshold. If the standard deviation of the draft lines is greater than the standard threshold, it is considered that the ship is inclined; otherwise, it is considered that the ship is not inclined.

[0040] The present invention has the following beneficial effects:

[0041] When the ship encounters waves in the water and the ship rises and falls due to the waves in the water, resulting in inaccurate image recognition of the ship's draft line and other parameters to calculate the ship's load, the present invention can provide certain reference, and through the present invention, it can also evaluate whether the ship is inclined, which plays an effective reference role in reasonably distributing the cargo weight on the ship, and avoids the situation where the result obtained when the image recognizes the distance between the water surface and the draft line is inaccurate due to the water environment. Description of the Drawings

[0042] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other relevant drawings can also be obtained based on these drawings.

[0043] Figure 1 It is a structural block diagram of the present invention.

[0044] 1. Wave-following image acquisition module; 2. Empty ship signal acquisition module; 3. Data processing module. Detailed implementation manners

[0045] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Usually, the components of the embodiments of the present invention described and shown in the accompanying drawings here can be arranged and designed in various different configurations.

[0046] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed present invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0047] It should be noted that similar reference numerals and letters indicate similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0048] In the description of the present invention, it should be understood that the orientation or positional relationships indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. are based on the orientation or positional relationships shown in the accompanying drawings, or the orientation or positional relationships when the product of the present invention is usually placed, or the orientation or positional relationships commonly understood by those skilled in the art. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention.

[0049] In addition, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0050] In the description of the present invention, it should also be noted that unless otherwise clearly specified and defined, the terms "set", "installed", "connected", and "coupled" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0051] A ship load prediction system based on image recognition, as Figure 1 shown, includes a wave-following image acquisition module 1, an empty ship signal acquisition module 2, and a data processing module 3 arranged on the ship body. The number of wave-following image acquisition modules 1 is several, and several wave-following image acquisition modules 1 are evenly spaced. The wave-following image acquisition module 1 continuously acquires the contact picture between the ship body and the water surface and feeds it back to the data processing module 3. The empty ship signal acquisition module 2 includes a ship end and an empty end connected by communication. The number of ship ends is the same as the number of wave-following image acquisition modules 1, and the empty end can be suspended horizontally. The empty end receives the ship end signal and outputs the vertical displacement of each ship end to the data processing module 3. The data processing module 3 identifies the ship's draft line, inclination, and load based on the contact picture and the vertical displacement of each ship end. Among them, the wave-following image acquisition module 1 includes a camera, the ship end includes an infrared transmitter, the empty end includes an unmanned aerial vehicle (UAV), the number of UAVs is at least three, the UAV includes a microprocessor, an infrared receiver coupled to the microprocessor, and a wireless communication module coupled to the microprocessor. The UAV is communicatively connected to the data processing module 3 through the wireless communication module, and the UAVs are also communicatively connected to each other through the wireless communication module. The wireless communication module includes a WIFI module, and the data processing module 3 includes a processor.

[0052] The present invention aims to solve the problem that when a ship is parked in water and encounters waves, the undulation caused by the waves makes it difficult for image recognition technology to accurately capture the ship's draft line, thereby affecting the calculation of the ship's load. The present invention provides a new solution. By using UAVs to establish a plane to measure the vertical displacement of the ship, it helps to confirm the change at the junction of the ship and the water surface waves, thereby evaluating the draft line situation of the ship, and based on this, determining whether there is an inclination and evaluating the load of the ship at this time. This is crucial for reasonably distributing the weight of the goods on the ship. Through the application of the present invention, the image recognition error caused by water environment factors can be effectively avoided, and the accuracy of the ship's load calculation can be ensured.

[0053] Preferably, the wave-following image acquisition module 1 continuously acquires the contact picture between the ship body and the water surface and feeds it back to the data processing module 3, including the following steps:

[0054] Set a number of evenly spaced points on the ship body according to the number of wave-following image acquisition modules 1;

[0055] Install a single wave-following image acquisition module 1 at a point position;

[0056] The wave-following image acquisition module 1 continuously acquires the contact picture between the hull and the water surface on the time axis and feeds it back to the data processing module 3.

[0057] Through the above method steps, multiple point positions can be set on the hull to provide data, so that when calculating the vertical displacement of the ship later, it can also be split into multiple point positions. If the vertical displacement of the entire ship is required, it can be estimated based on the data of multiple point positions to ensure the accuracy of the final data. And if it is necessary to evaluate whether the hull is tilted, it can also be compared through the splitting of multiple point positions to determine the tilting direction of the hull. And when calculating the load, the hull can be segmented according to the waterline of multiple point positions to obtain the volume of the hull below the water surface, so as to calculate a more accurate displacement. Even if the hull has tilted, relatively accurate data results can still be obtained.

[0058] Preferably, the airborne end receives the ship end signal and outputs the vertical displacements of each ship end to the data processing module 3, including the following steps.

[0059] The unmanned aerial vehicles fly above the hull and keep all unmanned aerial vehicles at the same horizontal plane;

[0060] Each unmanned aerial vehicle receives the distance information from the ship end to the horizontal plane, as well as the distance information between each unmanned aerial vehicle and calculates the vertical displacements of each ship end on the time axis through the Pythagorean theorem as the vertical displacements of each point position on the time axis.

[0061] By keeping multiple unmanned aerial vehicles at the same horizontal plane, the horizontal plane formed by them can provide a plane reference for the ship, so as to obtain the vertical displacement of the ship and ensure that the vertical displacement data of the ship can be obtained even when there is no land or other reference around the ship.

[0062] Preferably, the data processing module 3 identifies the waterline, tilting condition, and load of the hull according to the contact picture and the vertical displacements of each ship end, including the following steps.

[0063] Preprocess the contact picture on the time axis through Gaussian filtering;

[0064] Use an edge detection algorithm for the contact picture on the time axis to identify the boundary line between the water surface and the hull;

[0065] Normalize the intersection line between the water surface and the hull in the contact screen on the time axis and complete the spectrum analysis through Fourier transform to determine the amplitude and frequency of the water waves at each point;

[0066] Set the water wave as a sine wave, , where, is the water surface height at the i-th point at time t, is the amplitude of the water surface fluctuation at the i-th point, is the angular frequency of the water wave, , where, is the water wave frequency at the i-th point, is the phase of the water wave;

[0067] The output result is the change in water surface height at each point on the time axis .

[0068] The output result is the change in water surface height at each point on the time axis , including the following steps,

[0069] Calculate the spatial wave synchrony and phase difference , where λ is the wavelength, is the distance between two points. Assuming that different points on the hull have the same water wave frequency, the water waves are synchronous but have different initial phases , the phase difference between different points is related to their spatial distance

[0070] Deduce the synchrony of the water waves at different points through the phase difference in space to obtain the change in water surface height at different positions , where, is the phase change caused by the spatial position difference;

[0071] The output result is the change in water surface height at each point on the time axis .

[0072] Through the above steps and methods, the system can obtain more accurate changes in water surface height, which can consider the influence of water waves, and then ensure the accuracy of the final data.

[0073] Preferably, the data processing module 3 identifies the draft line, tilt condition, and load of the hull according to the contact screen and the vertical displacement of each ship end, and further includes the following steps,

[0074] Input the changes in water surface height at different positions on the time axis and the vertical displacement of each point on the time axis , calculate the draft depth at each point on the time axis by calculating the difference between the two .

[0075] The data processing module 3 identifies the ship's draft line, tilt condition, and load based on the contact screen and the vertical displacement of each ship end, and also includes the following steps

[0076] Input the draft depth at each point on the time axis , and calculate the average draft depth at each point ;

[0077] Establish a spatial coordinate, place the ship in the spatial coordinate, confirm the coordinates of each point in the spatial coordinate, and mark the average draft line of each point in the spatial coordinate based on the coordinates of each point and the average draft depth of each point, and then connect the lines;

[0078] Calculate the volume of the ship below the average draft line of each point after connecting the lines as the displacement;

[0079] Calculate the weight of water according to the displacement through the density formula, and obtain the load according to the difference between the weight of water and the self-weight of the ship;

[0080] Calculate the average height of the draft line at all points ;

[0081] Calculate the standard deviation of the draft line ;

[0082] Set a standard threshold, compare the standard deviation of the draft line with the standard threshold. If the standard deviation of the draft line is greater than the standard threshold, it is considered that the ship is tilted. Otherwise, it is considered that the ship is not tilted.

[0083] Through the above steps and methods, the system can identify the ship's draft line, assist in calculating the ship's load, and evaluate the ship's tilt condition, guide the reasonable distribution of goods, and ensure navigation safety.

[0084] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A ship loading prediction system based on image recognition, characterized in that, It includes a wave-following image acquisition module (1), an empty ship signal acquisition module (2), and a data processing module (3) arranged on the hull. The number of the wave-following image acquisition modules (1) is several, and several of the wave-following image acquisition modules (1) are arranged at uniform intervals. The wave-following image acquisition module (1) continuously acquires the contact picture between the hull and the water surface and feeds it back to the data processing module (3). The empty ship signal acquisition module (2) includes a ship end and an empty end connected by communication. The number of the ship ends is the same as that of the wave-following image acquisition modules (1), and the empty end can be suspended horizontally. The empty end receives the ship end signal and outputs the vertical displacement of each ship end to the data processing module (3). The data processing module (3) identifies the draft line, inclination condition, and load of the hull based on the contact picture and the vertical displacement of each ship end; The wave-following image acquisition module (1) continuously acquires the contact picture between the hull and the water surface and feeds it back to the data processing module (3), including the following steps, Set several points at uniform intervals on the hull according to the number of the wave-following image acquisition modules (1); Install a single wave-following image acquisition module (1) at the points; The wave-following image acquisition module (1) continuously acquires the contact picture between the hull and the water surface on the time axis and feeds it back to the data processing module (3); The empty end receives the ship end signal and outputs the vertical displacement of each ship end to the data processing module (3), including the following steps, The unmanned aerial vehicles fly above the hull and keep all the unmanned aerial vehicles on the same horizontal plane; Each unmanned aerial vehicle D j (j = 1, 2, …, m) receives the distance information from the ship end P i (i = 1, 2, …, n) to the horizontal plane, and the distance information between each unmanned aerial vehicle D j The vertical displacement z of each ship end on the time axis is calculated through the Pythagorean theorem i (t) as the vertical displacement of each point position on the time axis; The data processing module (3) identifies the draft line, inclination condition, and load of the hull based on the contact picture and the vertical displacement of each ship end, including the following steps, Preprocess the contact picture on the time axis through Gaussian filtering; Use an edge detection algorithm for the contact picture on the time axis to identify the boundary line between the water surface and the hull; Normalize the boundary line between the water surface and the hull in the contact picture on the time axis and complete spectrum analysis through Fourier transform to determine the amplitude and frequency of the water waves at each point; Set the water surface wave as a sine wave, h i (t) = A i sin(ω i t + φ i ), where h i (t) is the water surface height at the i-th point at time t, A i is the amplitude of the water surface fluctuation at the i-th point, ω i is the angular frequency of the water surface wave, ω i = 2πf i , where f i is the water surface wave frequency at the i-th point, φ i is the phase of the water surface wave; The output result is the water surface height change of each point on the time axis {h1(t), h2(t), …, h n (t)}; The data processing module (3) identifies the draft line, inclination condition, and load of the hull based on the contact picture and the vertical displacement of each ship end, and also includes the following steps, The change in water surface height h i at different positions on the time axis and the vertical displacement z i at each point on the time axis are calculated by taking the difference between the two to obtain the draft depth D i (t).

2. The ship load prediction system based on image recognition according to claim 1, wherein The wave-following image acquisition module (1) includes a camera.

3. The ship load prediction system based on image recognition according to claim 1, wherein The ship end includes an infrared transmitter, the empty end includes unmanned aerial vehicles, the number of the unmanned aerial vehicles is at least three, the unmanned aerial vehicles include a microprocessor, an infrared receiver coupled to the microprocessor, and a wireless communication module coupled to the microprocessor. The unmanned aerial vehicles are communicatively connected to the data processing module (3) through the wireless communication module, and the unmanned aerial vehicles are also communicatively connected to each other through the wireless communication module.

4. The ship load prediction system based on image recognition according to claim 1, wherein, The data processing module (3) includes a processor.

5. The ship load prediction system based on image recognition according to claim 1, characterized in that, The output result is the water surface height change of each point on the time axis {h1(t), h2(t), …, h n (t)}, and the steps are as follows: Calculating the spatial wave synchrony and phase difference where λ is the wavelength and Δx is the distance between two positions; The water surface height change h at different positions is obtained through the spatial phase difference i (t) = A i sin(2πft i + φ i + Δφ i ), where Δφ i is the phase change caused by the spatial position difference; The output result is the water surface height change of each point on the time axis {h1(t), h2(t), …, h n (t)}.

6. The ship load prediction system based on image recognition according to claim 5, characterized in that, The data processing module (3) identifies the draft line, inclination condition, and load of the hull based on the contact picture and the vertical displacement of each ship end, and also includes the following steps, Input the draft depth D of each point on the timeline i (t), and calculate the average draft depth of each point Establish a space coordinate, place the ship in the space coordinate, confirm the coordinates of each point in the space coordinate, and mark the average draft line of each point in the space coordinate based on the coordinates of each point and the average draft line depth of each point, and then connect the lines; Calculate the volume of the ship below the average draft line at each point after connection as the displacement; Calculate the weight of water based on the displacement according to the density formula, and obtain the load based on the difference between the weight of water and the deadweight of the ship; Calculate the average height of the waterline at all points Calculate the standard deviation of the draft Set a standard threshold, compare the standard deviation of the draft line with the standard threshold. If the standard deviation of the draft line is greater than the standard threshold, it is considered that the ship is tilted. Otherwise, it is considered that the ship is not tilted.

Images