Marine target detection method and device, electronic equipment and storage medium

Abstract

The application discloses a kind of offshore target detection method, device, electronic equipment and storage medium, it is related to offshore target perception technical field, comprising: according to the distance information measured by navigation radar, photoelectric tracking system, laser equipment and visible light camera equipment generates motion consistent feature;According to motion consistent feature determines motion distance;According to the image information obtained by the visible light camera equipment, the apparent consistent feature vector of target object is determined, and apparent consistent feature vector includes target category, video target width-height ratio, point cloud target length-width-height and length-width-height ratio;According to apparent consistent feature determines apparent distance;According to the real-time sea state information dynamic and membership function for determining motion weight, dynamically determine motion weight and apparent weight, according to the motion distance, motion weight, apparent distance and apparent weight determine target distance.Combination of sensor ranging, machine vision ranging and real-time sea state is realized, and the accuracy of offshore target detection is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of marine target perception technology, and in particular to a marine target detection method and device, an electronic device and a storage medium. BACKGROUND

[0002] With the development of navigation technology, navigation perception technology is one of the key technologies of water surface intelligent ships and unmanned boats.

[0003] Currently, when detecting marine targets, a radar or other sensors are usually used to detect large ships on the sea. However, when detecting small targets on the sea, such as small fishing boats and water buoys, there is a target loss. How to improve the accuracy of marine small target detection has become a problem to be solved. SUMMARY

[0004] The present application provides a marine target detection method, device, electronic device and storage medium to solve the problem of inaccurate marine small target detection.

[0005] According to one aspect of the present application, a marine target detection method is provided, comprising:

[0006] A motion consistent feature is generated according to distance information measured by a navigation radar, an optical-electric tracking system, a laser device and a visible light camera device; and a motion distance is determined according to the motion consistent feature.

[0007] An apparent consistent feature vector of a target object is determined according to image information obtained by the visible light camera device, the apparent consistent feature vector including target type, video target width-height ratio, point cloud target length-width-height and length-width-height ratio; and an apparent distance is determined according to the apparent consistent feature.

[0008] A motion weight and an apparent weight are dynamically determined according to real-time sea state information and a membership function for determining the motion weight; and a target distance is determined according to the motion distance, the motion weight, the apparent distance and the apparent weight.

[0009] According to another aspect of the present application, a marine target detection device is provided, comprising:

[0010] A motion consistent feature determination module is configured to generate a motion consistent feature according to distance information measured by a navigation radar, an optical-electric tracking system, a laser device and a visible light camera device; and to determine a motion distance according to the motion consistent feature.

[0011] An apparent consistent feature determination module is configured to determine an apparent consistent feature vector of a target object according to image information obtained by the visible light camera device, the apparent consistent feature vector including target type, video target width-height ratio, point cloud target length-width-height and length-width-height ratio; and to determine an apparent distance according to the apparent consistent feature.

[0012] a distance fusion module, configured to dynamically determine a motion weight and an apparent weight according to real-time sea state information and a membership function for determining the motion weight, determine a target distance according to the motion distance, the motion weight, the apparent distance and the apparent weight.

[0013] According to another aspect of the present application, an electronic device is provided, the electronic device comprising:

[0014] at least one processor; and

[0015] a memory connected with the at least one processor; wherein,

[0016] the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the offshore target detection method according to any one of the embodiments of the present application.

[0017] According to another aspect of the present application, a computer readable storage medium is provided, the computer readable storage medium stores computer instructions for enabling a processor to perform the offshore target detection method according to any one of the embodiments of the present application when executed by the processor.

[0018] The technical solution of the embodiments of the present application generates a motion consistent feature according to distance information measured by a navigation radar, an optoelectronic tracking system, a laser device and a visible light camera device; determines a motion distance according to the motion consistent feature; determines an apparent consistent feature vector of a target object according to image information obtained by the visible light camera device, the apparent consistent feature vector including a target type, a video target width-height ratio, a point cloud target length-width-height and a length-width-height ratio; determines an apparent distance according to the apparent consistent feature; dynamically determines a motion weight and an apparent weight according to real-time sea state information and a membership function for determining the motion weight, and determines a target distance according to the motion distance, the motion weight, the apparent distance and the apparent weight. Compared with the current target detection using only sensors, the technical solution of the embodiments of the present application can determine a motion consistent feature through multiple sensors, and further determine a motion distance. At the same time, image information of a target is obtained through a visible light camera device, and an apparent consistent feature vector is generated, and an apparent distance is determined according to the apparent consistent feature vector. Then, a motion weight and an apparent weight are dynamically determined according to sea state information, and a target distance of a target is determined through weighted summation, so as to combine sensor ranging, machine vision ranging and real-time sea state, and improve the accuracy of offshore target detection.

[0019] It is to be understood that the embodiments described herein are merely exemplary of the application and that a myriad of modifications, both as to the nature and number of elements within the execution of the application and as to the modes of execution thereof, can be made by those skilled in the art, without expressly quantifying the application and without departing from the scope of the application. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without paying creative labor on the basis of these drawings.

[0021] Figure 1 A flowchart of a sea target detection method provided by an embodiment of the present application;

[0022] Figure 2 A schematic diagram of a sensor detection capability provided by an embodiment of the present application;

[0023] Figure 3 A schematic diagram of a cooperative detection architecture provided by an embodiment of the present application;

[0024] Figure 4 A structural schematic diagram of a sea target detection device provided by an embodiment of the present application;

[0025] Figure 5 A structural schematic diagram of an electronic device for implementing a sea target detection method of an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the technical personnel in the art better understand the present application scheme, the following will combine the drawings in the embodiments of the present application, and the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only some embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of the present application.

[0027] It should be noted that the terms "include" and "have" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device containing a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0028] The inventor finds that with the development of navigation technology, the navigation perception technology is one of the key technologies of the intelligent water surface ship and the unmanned ship. In the detection of the sea target, the radar and other sensors can be used to detect the large ships on the sea surface. For the detection of the large ships on the water surface, the navigation radar, AIS and photoelectric detection equipment can realize relatively stable detection, but for the detection of the typical small water surface targets such as the small fishing boats (the size is about 6 meters), the water surface buoy (the height above the water surface is about 1 meter) and the water surface swimmer (the height out of the water surface is about 30 cm), the detection is inaccurate due to the sea waves and the ship swing, thereby causing the navigation embarrassment and the marine safety accident. Therefore, the stable detection of the small sea target is of great significance to the navigation safety of the marine ship. How to improve the accuracy of the detection of the small sea target becomes a problem to be solved.

[0029] Figure 1 It is a flowchart of a sea target detection method provided by an embodiment of the present application, and the embodiment can be applicable to the detection of the small target on the sea surface. The method can be executed by a sea target detection device, which can be realized in the form of hardware and / or software, and can be configured in an electronic device such as a personal computer, a notebook computer, a smart device, a mobile device or a server. As shown in the figure, the method comprises the following steps. Figure 1

[0030] In step S101, the motion consistent feature is generated according to the distance information measured by the navigation radar, the photoelectric tracking system, the laser device and the visible light camera device, and the motion distance is determined according to the motion consistent feature.

[0031] Firstly, the characteristics of the small sea target in the sea wave are analyzed, and the requirement for the detection sensor is determined. Specifically, the small sea target can include the small ship on the sea (the fishing boat, the fishing boat and the like with the size of about 7 meters), the buoy (various navigation marks, ocean observation buoy and the like) or the water personnel (swimming personnel). The small sea target is small in size (the height above the water surface is generally not higher than 2 meters), and in the low sea state (3 levels and below, the wave height is below 1.25 meters), the target can be stably exposed to the water surface in the sensor detection visual angle; when the sea state is high (such as 4 levels, the wave height is 1.25-2.5 meters), the target is exposed to the water surface in the wave peak, and the target is submerged in the wave in the wave valley. From the perspective of sensor detection, for the navigation radar, when the sea state is large, the echo of the target appears and disappears; for the photoelectric tracking system, the video and the laser detection means, the target is lost in the video, and the laser cannot detect the target. The period of detection and loss of the target in the wave is basically consistent with the wave period (because the inertia of the small target is small, the motion period when the small target acts on the wave is basically consistent with the wave).

[0032] ​Based on the above analysis, it is virtually impossible for a single sensor to achieve continuous and stable detection of small targets at sea; a multi-sensor fusion approach is necessary. Considering the characteristics of small targets at sea, navigation radar has blind spots at close range, making target detection impossible; therefore, detection methods covering close range are required. Within the radar's detection range, targets are often obscured by waves, leading to discontinuous target echoes. Furthermore, wave and clutter suppression can easily result in the loss of small targets. Therefore, combining visual information for target fusion filtering necessitates the configuration of methods capable of detecting and identifying single-point targets. Additionally, considering that some small speedboats are equipped with AIS (Air Traffic Control System) devices, the system needs to be able to detect small targets equipped with AIS.

[0033] Secondly, a multi-sensor collaborative detection system is established. Specifically, step 1.1, based on the sensor configuration requirements for small target detection on surface vessels, a sensor collaborative detection configuration scheme is proposed. Based on the analysis of sensor configuration requirements, the system is designed with the vessel as the center, proceeding from near to far in terms of detection distance. For near-range targets (target distance less than or equal to 200m), high-precision photoelectric 3D equipment capable of detecting azimuth and distance is primarily used, incorporating near-range target feature information from photoelectric tracking equipment; for medium-range targets (… For targets requiring high accuracy in detection range, a fusion of navigation radar and electro-optical tracking equipment is used to detect and track small targets, achieving feature fusion and stable tracking. At longer distances (target distance greater than 1000m), the unmanned surface vessel (USV) does not require high accuracy in target position; target acquisition primarily relies on navigation radar detection, supplemented by electro-optical target identification. Within the entire range, stable target information reception for cooperative targets (configured with AIS equipment) is required, necessitating the configuration of AIS equipment. Specific equipment configurations are shown in Table 1. Sensor detection capabilities are as follows: Figure 2 As shown in the diagram. The system architecture diagram for collaborative detection is as follows. Figure 3 As shown.

[0034]

[0035] Step 1.2: Based on the sensor configuration scheme, provide the architecture of the collaborative detection system and clarify the software and hardware functions. Specifically, based on the requirements for small target detection, establish a three-layer perception system consisting of sensors, information processing, and coordination scheduling. The sensor layer comprises AIS, an optoelectronic tracking system, navigation radar, and optoelectronic 3D equipment. The information processing layer includes information processing of each sensor and target information output. The coordination scheduling layer consists of multi-sensor fusion and sensor coordination scheduling functional modules. The multi-sensor fusion module includes the unification of spatiotemporal information of sensor targets and small target fusion based on multiple sensors. The system architecture is shown below. Figure 2 .

[0036] Step 1.3. According to the requirement of continuous and stable detection of small targets, a system for detecting small targets on the sea is established by guiding a photoelectric tracking system with an omnidirectional detection sensor (a navigation radar and a laser and camera device) according to the relationship between the small targets on the water surface and the relative movement of the ship.

[0037] Step 1.4. A track fusion method using multi-sensor cooperation is established based on the characteristics of small targets.

[0038] Optionally, motion consistent features are generated according to the distance information measured by the navigation radar, the photoelectric tracking system, the laser device and the visible light camera device, including:

[0039] According to a multi-sensor target space-time synchronization method of inertial navigation and robust Kalman filtering, the navigation radar, the photoelectric tracking system, the laser device and the visible light camera device are spatially aligned; and according to a robust Kalman filtering algorithm, the distance information measured by the navigation radar, the photoelectric tracking system, the laser device and the visible light camera device is predicted to obtain the motion consistent features at the current time.

[0040] Specifically, in step 2.1, according to the target information output by the navigation radar, the photoelectric tracking system, the laser and the visible light camera, a multi-sensor target space-time synchronization method using inertial navigation and robust Kalman filtering is used to realize the space-time alignment of the sensor information. Specifically, the space alignment of the information is performed by using the pose information obtained by the inertial navigation of the ship platform and the spatial relationship between the installation position of the sensor and the inertial navigation reference system; based on the detection period and the detection distance of the sensor, a linear constant motion velocity model is used in the robust Kalman filter to dynamically predict the 6-dimensional state vector of each target in the tracking target list at the current time according to the time stamp in the input of the perception module, so as to complete the time synchronization between the observed targets of the perception module and the fused tracking targets; the state parameters of the Kalman filter are updated according to the observation results, and the prediction process for the next target input is prepared.

[0041] Step 2.2. Calculate the motion consistent feature vector of the multi-sensor target The distance feature vector of the position and velocity vector of the current perception target and the fused tracking target is calculated to realize:

[0042]

[0043] In the formula: Xi represents the covariance matrix of the i-th fused target at the current observation time; Xj represents the motion state vector of the j-th observed target at the current observation time. In the formula, i represents the i-th fused target, j represents the j-th observed target at the current observation time, Xi,j represents the motion consistent feature vector of the i-th fused target and the j-th observed target at the current observation time. Xi (t) represents the predicted state vector of the i-th fusion target at the current time.

[0044] Using the chi-square distribution 0.95 quantile As the maximum threshold of motion consistency, the following indicator function is defined: .

[0045] The above embodiment can predict the distance information measured by the navigation radar, the photoelectric tracking system, the laser device and the visible light camera device through Kalman filtering, and improve the accuracy of the motion consistency feature.

[0046] In step S102, the apparent consistency feature vector of the target object is determined according to the image information obtained by the visible light camera device, and the apparent consistency feature vector includes the target type, the video target width-height ratio, the point cloud target length-width-height and the length-width-height ratio; and the apparent distance is determined according to the apparent consistency feature.

[0047] Optionally, determining the apparent consistency feature vector of the target object according to the image information obtained by the visible light camera device comprises:

[0048] The apparent consistency feature vector of the target object is determined according to the Manhattan distance between the historical apparent consistency feature vector and the current apparent consistency feature vector of the target object.

[0049] Specifically, the apparent consistency feature vector of the multi-sensor target is calculated The Manhattan distance between the fusion target feature vector and the observed target feature vector is calculated by comparing and calculating the feature vector composed of the target type, the video target width-height ratio, the point cloud target length-width-height and the length-width-height ratio, which is a consistency measure. In order to reduce the influence of the change of the appearance of the target over a long period of time, the last several times of successful matching targets are selected for consistency operation:

[0050]

[0051] Wherein: Xi (t) represents the apparent feature vector of the j-th observed target at the current time; Xi (t) represents the apparent feature vector of the i-th fusion target at the last successful matching time (the k-th batch in history). In i represents the i-th fusion target, j represents the j-th observed target, Xi (t) represents the apparent consistency feature vector of the i-th fusion target and the j-th observed target at the current observation time; is the set of all successfully matched batches.

[0052] In addition, a target distance threshold is added: .

[0053] A threshold function is constructed to determine the consistency matching success:

[0054]

[0055] If 1, the matching is successful, otherwise the matching fails.

[0056] The above embodiment can compare the current appearance consistent feature vector with the historical appearance consistent feature vector through Manhattan distance, exclude unstable pictures from the machine vision aspect, and thus realize more accurate detection of the small target at sea.

[0057] In step S103, the motion weight and the appearance weight are dynamically determined according to the real-time sea state information and the membership function for determining the motion weight, and the target distance is determined according to the motion distance, the motion weight, the appearance distance and the appearance weight.

[0058] Optionally, the motion weight and the appearance weight are dynamically determined according to the real-time sea state information and the membership function for determining the motion weight, which can be implemented in the following manner:

[0059] The wave height data and the visibility data are obtained;

[0060] The wave height data and the visibility data are used as input variables to substitute into the membership function, and the output of the membership function is the motion weight;

[0061] The appearance weight is determined according to the motion weight and the weight sum threshold.

[0062] Considering that the matching of the small target at sea is affected by the feature change caused by different angles of the sea target and the target frame loss caused by the sea wave and the swing, a method of target comprehensive distance matching combined with the motion consistent feature and the appearance consistent feature is established, and the weighted distance of the observed target and the fused target is calculated :

[0063]

[0064] In the formula: The motion weight is represented by w, The appearance weight is represented by w. The weighted distance of the observed target and the fused target is represented by d.

[0065] The above embodiment can dynamically determine the motion weight based on the wave height data and the visibility data, and improve the accuracy of the motion weight.

[0066] Because small targets on the sea usually exist target frame loss caused by factors such as waves and visibility, frame loss has a greater impact on target fusion, therefore, a dynamic weight adjustment strategy based on sea conditions and visibility and other environments is adopted, when the sea condition becomes larger , the consistency weight increases, and vice versa , the consistency weight decreases, which can further improve the performance of small target fusion. Considering that the correlation between environmental factor changes and cannot be described by an accurate mathematical model, a fuzzy reasoning-based adjustment method is proposed, taking sea condition level (or wave height) and visibility data as input quantities for weight adjustment, first normalizing the input quantities, establishing fuzzy rules, performing fuzzy reasoning, and finally normalizing the output value to obtain .

[0067] Optionally, obtaining wave height data and visibility data can be implemented as follows:

[0068] Determining the wave height data according to the ratio of the current wave height value to the maximum sea wave value; determining the visibility data according to the ratio of the current visibility to the maximum visibility.

[0069] The calculation formula of the wave height data is as follows, the sea condition level can be obtained by a sea condition observation sensor, and normalization is performed.

[0070]

[0071] Wherein is the maximum sea condition value, is the wave height value at the current time i.

[0072] The calculation formula of the visibility data is as follows, the current visibility can be measured by a visibility sensor.

[0073]

[0074] Wherein is the maximum visibility, is the visibility at the current time i.

[0075] The above embodiment can represent the wave height and the visibility in the form of a ratio, and further accurately express the wave height and the visibility, thereby improving the accuracy.

[0076] Optionally, the wave height data and the visibility data are taken as input variables of a membership function, and the output of the membership function is a motion weight, including:

[0077] Determining the consistency weight according to the wave height data and the visibility data;

[0078] According to the expression of the consistency weight and the plurality of membership intervals of the membership function, a target membership corresponding to the consistency weight is determined; and a quantitative motion weight is determined according to the target membership.

[0079] The input variable of the membership function is ( , ) and the output variable is the motion weight , the sea state input range is (0.125, 1), the visibility input range is (0.1, 1), and the motion weight value range is (0.5, 1).

[0080] The expression of the trapezoidal membership function is:

[0081]

[0082]

[0083]

[0084]

[0085]

[0086] wherein u(x; a, b, c, d) is the output of the gradient membership function. x is the input. a, b, c, d are four ladder threshold values,

[0087] The input variable is divided into fuzzy subsets {low, medium, high}, the output variable is divided into fuzzy subsets {small, medium, large}, and a trapezoidal membership function is assigned to each fuzzy subset. For example, the (a, b, c, d) of the input "low" can be set to (0.125, 0.2, 0.25, 0.4), and the (a, b, c, d) of the output "large" can be set to (0.6, 0.75, 0.85, 0.95). In this way, the membership functions are assigned to all input and output fuzzy subsets.

[0088] The fuzzy rule base is constructed as shown below. Based on expert experience, 9 core fuzzy rules are designed, following the principle of "low sea state and low visibility, then large". The rule table is shown in Table 2.

[0089]

[0090] The above embodiment can determine the target membership corresponding to the wave height data and the visibility data based on the membership interval, accurately represent the current sea state through the target membership, and provide accuracy.

[0091] Optionally, determining the quantitative motion weight according to the target membership comprises:

[0092] obtaining a center value of a target membership interval to which the target membership belongs;

[0093] determining a quantitative motion weight according to the center value and a membership corresponding to the target membership interval.

[0094] The target membership is a fuzzy value. The fuzzy value can be converted into a quantitative value through the above steps. The process of defuzzification is shown as follows.

[0095] The barycenter method is used for defuzzification to convert the fuzzy output into a quantitative weight :

[0096]

[0097] wherein is a membership function of a fuzzy subset, is a fuzzy reasoning output variable.

[0098] The above embodiment can convert the fuzzy target membership into a quantitative motion weight, and further accurately represent the current sea state through the motion weight, thereby improving the target detection accuracy.

[0099] The sea target detection method provided by the embodiment of the present application generates a motion consistent feature according to the distance information measured by a navigation radar, an optical-electric tracking system, a laser device and a visible light camera device; determines a motion distance according to the motion consistent feature; determines an apparent consistent feature vector of a target object according to image information obtained by the visible light camera device, the apparent consistent feature vector including a target type, a video target width-height ratio, a point cloud target length-width-height and a length-width-height ratio; determines an apparent distance according to the apparent consistent feature; dynamically determines a motion weight and an apparent weight according to real-time sea state information and a membership function used for determining the motion weight; and determines a target distance according to the motion distance, the motion weight, the apparent distance and the apparent weight. Compared with the current target detection method using only sensors, the sea small target detection is not accurate. The sea target detection method provided by the embodiment of the present application can determine the motion consistent feature through multiple sensors, and further determine the motion distance. At the same time, the image information of the target is obtained through the visible light camera device, and the apparent consistent feature vector is generated, and the apparent distance is determined according to the apparent consistent feature vector. Then, the motion weight and the apparent weight are dynamically determined according to the sea state information, and the target distance of the target is determined through weighted summation, so as to combine the sensor ranging, machine vision ranging and real-time sea state, and improve the accuracy of sea target detection.

[0100] The sea target detection method provided by the embodiment of the present application fuses motion consistency and representation consistency according to the characteristics of small targets on the sea, determines apparent consistency elements by using target types, video target width-height ratio, point cloud target length-width-height, length-width-height ratio and other characteristics, and can greatly improve the fusion accuracy. The motion consistency and representation consistency weight values based on the fuzzy promotion method can be dynamically adjusted according to the characteristics of small targets in waves and sea conditions and visibility, and the adaptability of small target fusion to sea conditions can be improved.

[0101] Figure 4 is a structural schematic diagram of a sea target detection device provided by the embodiment of the present application, and the embodiment can be applicable to the case of detecting small targets on the sea. The sea target detection device can be realized in the form of hardware and / or software, and the sea target detection device can be configured in an electronic device such as a personal computer, a notebook computer, a smart device, a mobile device or a server. As shown in the figure, the device comprises a motion consistent feature determination module 21, an apparent consistent feature determination module 22 and a distance fusion module 23. Figure 4

[0102] The motion consistent feature determination module 21 is configured to generate motion consistent features according to distance information measured by a navigation radar, an optical tracking system, a laser device and a visible light camera device; and determine a motion distance according to the motion consistent features.

[0103] The apparent consistent feature determination module 22 is configured to determine an apparent consistent feature vector of a target object according to image information obtained by the visible light camera device, wherein the apparent consistent feature vector comprises target types, video target width-height ratio, point cloud target length-width-height and length-width-height ratio; and determine an apparent distance according to the apparent consistent features.

[0104] The distance fusion module 23 is configured to dynamically determine a motion weight and an apparent weight according to real-time sea condition information and a membership function used to determine the motion weight; and determine a target distance according to the motion distance, the motion weight, the apparent distance and the apparent weight.

[0105] On the basis of the above-mentioned embodiment, the distance fusion module 23 is configured to dynamically determine a motion weight and an apparent weight according to real-time sea condition information and a membership function used to determine the motion weight, and comprises:

[0106] acquire wave height data and visibility data;

[0107] substitute the wave height data and the visibility data as input variables into the membership function, and the output of the membership function is the motion weight;

[0108] determine the apparent weight according to the motion weight and a weight sum threshold.

[0109] ​On the basis of the above-mentioned embodiments, optionally, the distance fusion module 23 is configured to acquire the wave height data and the visibility data, comprising:

[0110] determining the wave height data according to a ratio of the current wave height value and the maximum sea wave value;

[0111] determining the visibility data according to a ratio of the current visibility and the maximum visibility.

[0112] On the basis of the above-mentioned embodiments, optionally, the distance fusion module 23 is configured to substitute the wave height data and the visibility data as input variables into a membership function, and an output of the membership function is a motion weight, comprising:

[0113] determining a consistency weight according to the wave height data and the visibility data;

[0114] determining a target membership degree corresponding to the consistency weight according to a matching between the consistency weight and expressions of a plurality of membership degree intervals of the membership function;

[0115] determining a quantitative motion weight according to the target membership degree.

[0116] On the basis of the above-mentioned embodiments, optionally, the distance fusion module 23 is configured to determine a quantitative motion weight according to the target membership degree, comprising:

[0117] acquiring a center value of a target membership degree interval to which the target membership degree belongs;

[0118] determining a quantitative motion weight according to the center value and a membership degree corresponding to the target membership degree interval.

[0119] On the basis of the above-mentioned embodiments, optionally, the apparent consistency feature determination module 22 is configured to determine an apparent consistency feature vector of a target object according to image information acquired by the visible light camera device, comprising:

[0120] determining the apparent consistency feature vector of the target object according to a Manhattan distance between a historical apparent consistency feature vector and a current apparent consistency feature vector of the target object.

[0121] On the basis of the above-mentioned embodiments, optionally, the motion consistency feature determination module 21 is configured to generate a motion consistency feature according to distance information measured by the navigation radar, the photoelectric tracking system, the laser device and the visible light camera device, comprising:

[0122] performing spatial alignment on the navigation radar, the photoelectric tracking system, the laser device and the visible light camera device according to a multi-sensor target space-time synchronization device of inertial navigation and robust Kalman filtering;

[0123] According to the robust Kalman filtering algorithm, the distance information measured by the navigation radar, the photoelectric tracking system, the laser device and the visible light camera device is predicted to obtain the motion consistent feature at the current time.

[0124] The sea target detection device provided by the embodiment of the present application comprises a motion consistent feature determination module 21, which is used for generating a motion consistent feature according to the distance information measured by the navigation radar, the photoelectric tracking system, the laser device and the visible light camera device; determining a motion distance according to the motion consistent feature; an apparent consistent feature determination module 22, which is used for determining an apparent consistent feature vector of a target object according to image information obtained by the visible light camera device, wherein the apparent consistent feature vector comprises a target type, a video target width-height ratio, a point cloud target length-width-height and a length-width-height ratio; determining an apparent distance according to the apparent consistent feature; and a distance fusion module 23, which is used for dynamically determining a motion weight and an apparent weight according to real-time sea state information and a membership function used for determining the motion weight, and determining a target distance according to the motion distance, the motion weight, the apparent distance and the apparent weight. Compared with the current target detection using only sensors, the sea target detection device provided by the embodiment of the present application can determine the motion consistent feature through multiple sensors, and then determine the motion distance. At the same time, the image information of the target is obtained by the visible light camera device, and the apparent consistent feature vector is generated, and the apparent distance is determined according to the apparent consistent feature vector. Then, the motion weight and the apparent weight are dynamically determined according to the sea state information, and the target distance of the target is determined through weighted summation, so as to combine the sensor ranging, the machine vision ranging and the real-time sea state, and improve the accuracy of the sea target detection.

[0125] The sea target detection device provided by the embodiment of the present application can execute the sea target detection method provided by any embodiment of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0126] Figure 5 Figure 1 is a structural schematic diagram of an electronic device according to an embodiment of the present application. The electronic device 10 is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (such as headsets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0127] As Figure 5As shown, the electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., connected to the at least one processor 11 in communication. The memory stores computer programs executable by the at least one processor 11, and the processor 11 can perform various appropriate actions and processes according to the computer programs stored in the read-only memory (ROM) 12 or loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0128] Various components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a camera, an ultrasonic sensor, an infrared sensor, etc., an output unit 17, such as various types of speakers, etc., a storage unit 18, such as a magnetic disk, a solid state drive, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0129] The processor 11 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the offshore target detection method.

[0130] In some embodiments, the offshore target detection method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the offshore target detection method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the offshore target detection method by any other appropriate means, such as by means of firmware.

[0131] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system on a chip systems (SOCs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0132] Computer programs used to implement the offshore target detection method of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program running on the processor implements the functions / operations specified in the flow diagrams and / or block diagrams. The computer program can execute entirely on a machine, partly on a machine, partly on a remote machine or entirely on a remote machine or server.

[0133] The embodiments of the present application also provide a computer readable storage medium, which stores computer instructions, and the computer instructions are used for causing a processor to execute an offshore target detection method, the method comprises:

[0134] Generating a motion consistent feature according to distance information measured by a navigation radar, an optical-electric tracking system, a laser device, and a visible light camera device; determining a motion distance according to the motion consistent feature;

[0135] Determining an apparent consistent feature vector of the target object according to image information acquired by the visible light camera device, the apparent consistent feature vector comprising a target type, a video target width-height ratio, a point cloud target length-width-height, and a length-width-height ratio; determining an apparent distance according to the apparent consistent feature;

[0136] Dynamically determining a motion weight and an apparent weight according to real-time sea state information and a membership function used for determining the motion weight; determining a target distance according to the motion distance, the motion weight, the apparent distance, and the apparent weight.

[0137] On the basis of the above-mentioned embodiments, optionally, the dynamically determining the motion weight and the apparent weight according to the real-time sea state information and the membership function used for determining the motion weight comprises:

[0138] acquire wave height data and visibility data;

[0139] substitute the wave height data and the visibility data as input variables into a membership function, and an output of the membership function is a motion weight;

[0140] determine an apparent weight according to the motion weight and a weight sum threshold.

[0141] On the basis of the above-mentioned embodiments, the wave height data and the visibility data are acquired, including:

[0142] determine the wave height data according to a ratio of a current wave height value to a maximum sea wave value;

[0143] determine the visibility data according to a ratio of a current visibility to a maximum visibility.

[0144] On the basis of the above-mentioned embodiments, the wave height data and the visibility data are substituted as input variables into the membership function, and an output of the membership function is the motion weight, including:

[0145] determine a consistency weight according to the wave height data and the visibility data;

[0146] determine a target membership degree corresponding to the consistency weight according to a matching of the consistency weight and expressions of a plurality of membership degree intervals of the membership function;

[0147] determine a quantitative motion weight according to the target membership degree.

[0148] On the basis of the above-mentioned embodiments, the quantitative motion weight is determined according to the target membership degree, including:

[0149] acquire a center value of a target membership degree interval to which the target membership degree belongs;

[0150] determine a quantitative motion weight according to the center value and a membership degree corresponding to the target membership degree interval.

[0151] On the basis of the above-mentioned embodiments, the apparent consistency feature vector of the target object is determined according to the image information acquired by the visible light camera device, including:

[0152] determine the apparent consistency feature vector of the target object according to a Manhattan distance between a historical apparent consistency feature vector and a current apparent consistency feature vector of the target object.

[0153] On the basis of the above-mentioned embodiments, the motion consistency feature is generated according to distance information measured by the navigation radar, the photoelectric tracking system, the laser device and the visible light camera device, including:

[0154] According to the multi-sensor target space-time synchronization method based on inertial navigation and robust Kalman filtering, the navigation radar, the photoelectric tracking system, the laser device and the visible light camera device are spatially aligned.

[0155] According to the robust Kalman filtering algorithm, the distance information measured by the navigation radar, the photoelectric tracking system, the laser device and the visible light camera device is predicted to obtain the motion consistent feature at the current time.

[0156] In the context of the present application, a computer readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium can be a machine readable signal medium. More specific examples of the machine readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0157] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0158] The systems and techniques described herein can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described herein, or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0159] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0160] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, and the present disclosure is not limited in this regard.

[0161] The specific embodiments described above are not intended to limit the scope of the present disclosure. Those skilled in the art will understand that various modifications, combinations, sub-combinations, and alternatives can be made to the specific embodiments without departing from the spirit and principles of the present disclosure. Any further modifications, equivalents, and / or alternatives come within the scope of the present disclosure as recited by the claims.

Claims

1. A method of detecting a target at sea, characterized in that, The method comprises the following steps: Motion consistency features are generated according to distance information measured by a navigation radar, an optoelectronic tracking system, a laser device and a visible light camera device; and a motion distance is determined according to the motion consistency features; An apparent consistency feature vector of a target object is determined according to image information obtained by the visible light camera device, the apparent consistency feature vector comprising a target type, a video target width-height ratio, a point cloud target length-width-height and a length-width-height ratio; and an apparent distance is determined according to the apparent consistency features; Wave height data is determined according to a ratio of a current wave height value to a maximum wave height value; visibility data is determined according to a ratio of a current visibility to a maximum visibility; a consistency weight is determined according to the wave height data and the visibility data; a target membership degree corresponding to the consistency weight is determined by matching the consistency weight with expressions of a plurality of membership degree intervals of a membership function; and a center value of a target membership degree interval to which the target membership degree belongs is obtained; A quantitative motion weight is determined according to the center value and a membership degree corresponding to the target membership degree interval; and an apparent weight is determined according to the motion weight and a weight sum threshold. A target distance is determined according to the motion distance, the motion weight, the apparent distance and the apparent weight.

2. The method of claim 1, wherein, The method for determining the apparent consistency feature vector of the target object according to the image information obtained by the visible light camera device comprises the following steps: The apparent consistency feature vector of the target object is determined according to a Manhattan distance between a historical apparent consistency feature vector of the target object and a current apparent consistency feature vector of the target object.

3. The method of claim 1, wherein, The method for generating the motion consistency features according to distance information measured by a navigation radar, an optoelectronic tracking system, a laser device and a visible light camera device comprises the following steps: The navigation radar, the optoelectronic tracking system, the laser device and the visible light camera device are spatially aligned according to a multi-sensor target space-time synchronization method of inertial navigation and robust Kalman filtering; The motion consistency features at a current time are obtained by predicting distance information measured by the navigation radar, the optoelectronic tracking system, the laser device and the visible light camera device according to a robust Kalman filtering algorithm.

4. A marine target detection apparatus, characterized by comprising: The method comprises the following steps: A motion consistency feature determination module is configured to generate motion consistency features according to distance information measured by a navigation radar, an optoelectronic tracking system, a laser device and a visible light camera device; and a motion distance is determined according to the motion consistency features; An apparent consistency feature determination module is configured to determine an apparent consistency feature vector of a target object according to image information obtained by the visible light camera device, the apparent consistency feature vector comprising a target type, a video target width-height ratio, a point cloud target length-width-height and a length-width-height ratio; and an apparent distance is determined according to the apparent consistency features; The distance fusion module is configured to determine wave height data according to a ratio of a current wave height value to a maximum sea wave value, determine visibility data according to a ratio of a current visibility to a maximum visibility, determine a consistency weight according to the wave height data and the visibility data, match the consistency weight with expressions of a plurality of membership intervals of a membership function to determine a target membership corresponding to the consistency weight, obtain a center value of a target membership interval to which the target membership belongs, determine a quantitative motion weight according to the center value and a membership corresponding to the target membership interval, determine an apparent weight according to the motion weight and a weight sum threshold, and determine a target distance according to the motion distance, the motion weight, an apparent distance, and the apparent weight.

5. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the offshore target detection method of any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to implement the offshore target detection method of any one of claims 1-3 when executed.

Images