Intelligent Detection Method and Device for In-river and In-sea Water Discharge Ports Mounted on Boats

Abstract

The present invention relates to the technical field of water quality prediction, and discloses an intelligent detection method for river inlet and sea inlet water discharge outlets carried on boats; constructing a river discharge outlet data set, and dividing the river discharge outlet data into a training set, a validation set and a test set; constructing a discharge outlet detection model based on the YOLOV5 model to obtain real-time video data collected by a camera, and extracting frames from the real-time video data to generate real-time original images; performing image sliding window processing on the real-time original images to generate a set of images to be detected; using the discharge outlet detection model to detect the set of images to be detected to generate a set of feature images; performing Hough circle detection and Hough rectangle detection on the set of feature images to filter out the wrong pictures in the set of feature images; judging whether there is a phenomenon of flowing water on sunny days at the discharge outlet according to the filtered set of feature images; if so, reporting the abnormality of the discharge outlet; if not, reporting the discharge outlet information; having the advantages of high detection efficiency and high accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of water quality prediction, and particularly to an intelligent detection method and device for river and sea water intake outlets carried on a boat. Background Art

[0002] Currently, the river patrol and management work mainly relies on manpower, and there are also a small number of semi-automatic river detection products carried by drones.

[0003] Under the manual inspection method, the operation quality and efficiency of the project are relatively low. The operation personnel need to walk along the river bank. If an outlet is found, they need to take pictures and register information (location, status, material, whether it drains on sunny days, etc.). Coupled with the extremely complex installation positions of various outlets in the actual river, the uneven work experience of the inspection personnel and the actual inspection work difficulty and other factors, the scientific inspection and management of outlets have always been important problems that are difficult to solve.

[0004] Manual inspection and re-inspection first require higher labor costs, which sets a certain threshold for environmental protection enterprises. Secondly, the process of manually reporting information responds much slower to emergencies such as drainage on sunny days than intelligent devices. More importantly, the accuracy of manual inspection is limited by the operation standardization, work attitude and judgment experience of on-site operation personnel, and is not objective enough. Once an engineering problem occurs, it cannot be immediately reflected in the results, and ultimately it is easy to lead to unqualified project acceptance or additional cost investment.

[0005] Therefore, an intelligent detection method and device for river and sea water intake outlets carried on a boat are provided to solve the above problems. Summary of the Invention

[0006] The main purpose of the present invention is to solve the problems in the prior art that the river patrol and management work mainly relies on manpower, and there are also a small number of semi-automatic river detection products carried by drones; the operation quality and efficiency of the project under the manual inspection method are relatively low; manual inspection and re-inspection first require higher labor costs, etc.

[0007] The first aspect of the present invention provides an intelligent detection method for river and sea water intake outlets carried on a boat, and the intelligent detection method for river and sea water intake outlets carried on a boat includes:

[0008] Construct a river outlet data set, and divide the river outlet data into a training set, a validation set and a test set;

[0009] Based on the YOLOV5 model, construct an outlet detection model, train the outlet detection model using the training set, test the outlet detection model using the test set, and determine whether the outlet detection model meets the preset conditions. If so, the establishment of the outlet detection model is completed;

[0010] Obtain the real-time video data collected by the camera, extract frames from the real-time video data to generate real-time original images; perform image sliding window processing on the real-time original images to generate a set of images to be detected;

[0011] Use the outfall detection model to detect the set of images to be detected and generate a set of feature images;

[0012] Perform Hough circle detection and Hough rectangle detection on the set of feature images to filter out the wrong pictures in the set of feature images;

[0013] Judge whether there is a phenomenon of sunny-day water flow at the outfall according to the filtered set of feature images; if so, report the outfall anomaly; if not, report the outfall information.

[0014] Optionally, the construction of the river outfall data set and the division of the river outfall data into a training set, a validation set and a test set include:

[0015] Obtain the shoreline video data collected by the camera;

[0016] Extract frames from the shoreline video data to construct a set of original images;

[0017] Label the original images in the set of original images with the Label Img image annotation tool, divide the completed labeled original image data into several categories, and divide the original images of each category into a training set, a validation set and a test set at a ratio of 7:2:1.

[0018] Optionally, it also includes if the outfall detection model does not meet the preset conditions;

[0019] Then reconstruct the river outfall data set, and divide the river outfall data into a training set, a validation set and a test set;

[0020] Build an outfall detection model based on the YOLOV5 model, train the outfall detection model with the training set, and test the outfall detection model with the test set.

[0021] Optionally, the construction of the outfall detection model based on the YOLOV5 model includes:

[0022] Build a YOLOV5 model, add alternative detection frames for small targets in the Anchors part of the YOLOV5 model; extract the information of the large-size images from the P3 / 8 layer in the head of the YOLOV5 model, and merge this part of the information into the head information fusion part; the newly fused content in the head part of the YOLOV5 model is fused again in the stage of finally generating the detection result.

[0023] Optionally, performing image sliding window processing on the real-time original image to generate the image set to be detected includes:

[0024] The real-time original image is divided into 3*3 sub-blocks, each 2*2 sub-block is cropped to generate sub-block image information, the sub-block image information is enlarged to the size of the original image, and the sub-block image information and the real-time original image are used as the image set to be detected.

[0025] Optionally, performing Hough rectangle detection on the feature image set includes:

[0026] Record the coordinates of all the coordinates in the feature image with the preset values, and record them as the set

[0027] Points=[(x0,y0),...(x n ,y n )];

[0028] Create an accumulation matrix M with a height of H, a width of 0 to 180 degrees, and an initial value of 0. H is a preset multiple of the image height, which is greater than or equal to times;

[0029] For each coordinate point in Points from 0 degrees to 180 degrees, calculate the given θ based on the following formula i The corresponding ρ i , and fill the calculation results into the corresponding positions of the accumulation matrix M;

[0030] x0 cosθ+y0 sinθ=ρ, where x0, y0 are the coordinate values ​​of each coordinate point;

[0031] Get the maximum value of each column in the matrix M, recorded as vector P;

[0032] Perform mean filtering on P (kernel_size=10) to make the curve smoother, and the resulting new vector is recorded as Q;

[0033] The main purpose of performing NMS filtering on Q is to find the maximum value within the region, and the obtained new vector is recorded as QNms;

[0034] Compare QNms and Q bit by bit. If the comparison result is the same in Q, keep it, and if it is different, set it to 0.

[0035] Determine that the maximum value in Q is a vertex P1 of the suspected rectangle. Find out whether there is a point P2 as another vertex of the rectangle in the neighborhood with a θ interval of exactly 90. If not, give up;

[0036] It is known that P1 and P2 are the maximum values ​​of the specified column in the M matrix, so find the second largest value from the same column of the M matrix as the other two vertices of the rectangle;

[0037] Map the coordinates of the four vertices back to the Cartesian coordinate system and draw a rectangle by calculating the straight line equation and the focus; determine whether the area ratio of the rectangle reaches more than 50% of the entire image. If it meets the requirement, it is considered that the rectangle detection is passed.

[0038] Optionally, the Hough circle detection on the feature image set includes:

[0039] Perform object detection on the original image Img0 to obtain a number of detection frames (B0...B n )

[0040] Crop based on the detection frames to obtain the cropped images (C0...C n )

[0041] Perform the following processing on the cropped image C i respectively:

[0042] Perform median filtering on the image to remove the noise contained in the background;

[0043] Perform edge detection on the filtered image to extract the contour of the detected object;

[0044] Traverse the white points in the image and calculate the normal equation of the point in its neighborhood; accumulate all the points on this normal line once;

[0045] Find all possible circle centers according to the set lower threshold;

[0046] Traverse each circle center P f Calculate the distances from all points to this circle center, so as to select a suitable radius; for the suitable radius, add 1 to the accumulator;

[0047] Select all the circle centers that meet the requirements, and the circles with the highest radius value in their radius accumulators;

[0048] Compare these circles with the detection frames of the object detection respectively. If the offset degree between the circle center and the center of the detection frame is less than the threshold Threshold1, and the area of the circle accounts for more than the threshold Threshold2 of the area of the detection frame, it is considered that the detection meets the requirements.

[0049] The second aspect of the present invention provides an intelligent detection device for river inlet and sea inlet water discharge ports carried on a boat, including:

[0050] A river discharge port data set construction module, used to construct a river discharge port data set and divide the river discharge port data into a training set, a validation set and a test set;

[0051] The outfall detection model construction module is used to construct an outfall detection model based on the YOLOV5 model, train the outfall detection model using the training set, test the outfall detection model using the test set, and determine whether the outfall detection model meets the preset conditions. If so, the establishment of the outfall detection model is completed;

[0052] The image set to be detected generation module is used to obtain the real-time video data collected by the camera, extract frames from the real-time video data to generate real-time original images; perform image sliding window processing on the real-time original images to generate an image set to be detected;

[0053] The feature image set generation module is used to detect the image set to be detected using the outfall detection model to generate a feature image set;

[0054] The error image filtering module is used to perform Hough circle detection and Hough rectangle detection on the feature image set to filter out the error images in the feature image set;

[0055] The sunny day water flow phenomenon judgment module is used to judge whether there is a sunny day water flow phenomenon at the outfall according to the filtered feature image set; if so, report the outfall anomaly; if not, report the outfall information.

[0056] The third aspect of the present invention provides an electronic device, which includes a memory and at least one processor, and instructions are stored in the memory;

[0057] The at least one processor calls the instructions in the memory so that the electronic device executes each step of the intelligent detection method for the river and sea water outfalls carried on the boat as described above.

[0058] The fourth aspect of the present invention provides a computer-readable storage medium, on which instructions are stored, and when the instructions are executed by a processor, each step of the intelligent detection method for the river and sea water outfalls carried on the boat as described above is realized.

[0059] In the technical solution of the present invention, the outfall investigation can be carried out objectively and efficiently, while reducing costs, ensuring the standardization of the inspection; the present invention provides a complete process method from the image data collection of boat operations to the final detection result reporting, which is easy to realize the construction of the whole system; the detection system of the present invention proposes an optimized method for Hough circle and Hough rectangle detection based on the actual scenario of outfall investigation, and an improved method for the yolov5 model based on small outfalls, which improves the recall ability and accuracy of detection in this scenario; compared with manual inspection, it can achieve faster early warning and management. Description of the Drawings

[0060] Figure 1Flow chart of the intelligent detection method for river and sea water discharge outlets mounted on boats provided by the embodiments of the present invention;

[0061] Figure 2 Structural schematic diagram of the intelligent detection device for river and sea water discharge outlets mounted on boats provided by the embodiments of the present invention;

[0062] Figure 3 Schematic diagram during the working process of the present invention;

[0063] Figure 4 Schematic diagram of the sliding window segmentation of the present invention;

[0064] Figure 5 Structural schematic diagram of the electronic device provided by the embodiments of the present invention;

[0065] Figure 6 Comparison diagram between the discharge outlet detection basic model and the original model of the present invention;

[0066] Figure 7 Schematic diagram of the mutual mapping of points between the Cartesian coordinate system and the Huffman coordinate system;

[0067] Figure 8 Schematic diagram of the rectangle detection principle in the Huffman coordinate system. Detailed implementation manners

[0068] The embodiments of the present invention provide an intelligent detection method for river and sea water discharge outlets mounted on boats, including constructing a river discharge outlet data set, dividing the river discharge outlet data into a training set, a validation set and a test set; constructing a discharge outlet detection model based on the YOLOV5 model, training the discharge outlet detection model with the training set, testing the discharge outlet detection model with the test set, and judging whether the discharge outlet detection model meets the preset conditions. If so, the establishment of the discharge outlet detection model is completed; obtaining the real-time video data collected by the camera, extracting frames from the real-time video data to generate real-time original images; performing image sliding window processing on the real-time original images to generate a set of images to be detected; using the discharge outlet detection model to detect the set of images to be detected to generate a set of feature images; performing Hough circle detection and Hough rectangle detection on the set of feature images to filter out the wrong pictures in the set of feature images; judging whether there is a phenomenon of clear-day water flow at the discharge outlet according to the filtered set of feature images; if so, reporting the discharge outlet anomaly; if not, reporting the discharge outlet information. The present invention solves the problems in the prior art that the river channel inspection and management work mainly relies on manpower, and there are also a small number of semi-automatic river channel detection products carried by drones; the operation quality and efficiency of projects under the manual inspection method are relatively low; manual investigation and recheck first require higher labor costs, etc.

[0069] The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and the above-mentioned drawings of the present invention are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" or "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily limit to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0070] For ease of understanding, the specific process of the embodiment of the present invention will be described below. Please refer to Figure 1 , the first embodiment of the intelligent detection method for the river and sea water intake and discharge ports carried on the boat in the embodiment of the present invention includes:

[0071] Construct a river discharge port data set, and divide the river discharge port data into a training set, a validation set and a test set;

[0072] Based on the YOLOV5 model, construct a discharge port detection model, train the discharge port detection model using the training set, test the discharge port detection model using the test set, and determine whether the discharge port detection model meets the preset conditions. If so, the establishment of the discharge port detection model is completed; if the discharge port detection model does not meet the preset conditions, then reconstruct the river discharge port data set, and divide the river discharge port data into a training set, a validation set and a test set;

[0073] Obtain the real-time video data collected by the camera, extract frames from the real-time video data to generate real-time original images; perform image sliding window processing on the real-time original images to generate a set of images to be detected;

[0074] Use the discharge port detection model to detect the set of images to be detected, and generate a set of feature images;

[0075] Perform Hough circle detection and Hough rectangle detection on the set of feature images to filter out the wrong pictures in the set of feature images;

[0076] Judge whether there is a phenomenon of flowing water on sunny days at the discharge port according to the filtered set of feature images; if so, report the abnormality of the discharge port; if not, report the discharge port information.

[0077] The construction of the river discharge port data set, and dividing the river discharge port data into a training set, a validation set and a test set includes:

[0078] Obtain the shoreline video data collected by the camera; the collection of this shoreline video data can arrange the boat in different positions and different scenarios to sail at various angles and collect video data.

[0079] Extract frames from the shoreline video data to construct the original image set;

[0080] Annotate the original images in the original image set with the LabelImg image annotation tool, and divide the completed annotated original image data into several categories. The specific classification methods include classification according to different scenarios (rural areas, seaports, towns, etc.) and different materials (PVC, cement, copper); divide the original images of each category into a training set, a validation set, and a test set at a ratio of 7:2:1. After the above dataset is constructed, data augmentation can also be performed on the dataset, including: horizontal flipping of images, optimization of image brightness and contrast

[0081] Build an outfall detection model based on the YOLOV5 model, train the outfall detection model using the training set, and test the outfall detection model using the test set.

[0082] The building of the outfall detection model based on the YOLOV5 model includes:

[0083] Build a YOLOV5 model, add alternative detection frames for small targets in the Anchors part of the YOLOV5 model; extract the information of the large-size images from the P3 / 8 layer in the head of the YOLOV5 model and merge this part of the information into the head information fusion part; the newly fused content in the head part of the YOLOV5 model is fused again at the stage of finally generating the detection result.

[0084] The present invention uses the YOLOV5 model as the basic outfall detection model and designs a new structure for the detection of small outfalls on the basis of the original model structure. For specific comparison, refer to the attached Figure 6 :

[0085] On the basis of the original model structure, the new structure of the present invention is designed as follows:

[0086] Add a New1 part in the Anchors part. This part is mainly to enable the model to add alternative detection frames for small targets and enrich the alternative types of detection frames.

[0087] Add a New2 part in the Head part. This part is mainly designed to extract the information of the large-size images from the P3 / 8 layer additionally and merge this part of the information into the head information fusion part. This design method helps to fuse the detailed features on the large-size images more clearly, enabling the model to enhance the recognition ability of detailed features.

[0088] Added a New3 section to the Head section. This section will cause the information newly incorporated into the head section to be fused again during the stage of generating the final detection results.

[0089] The other parts of this network structure are the main body of the YOLOV5 model network structure. It mainly consists of four columns: [from, number, module, args]. Among them, from represents the input source of the current module, and -1 means using the output of the previous layer as the input of this layer. number represents the number of repetitions of the current module. module represents the name of the module, and each named module has its own fixed network structure. args represents some possible configuration parameters. The backbone part is the feature extraction part, and its main function is to extract the features of the detection target. The main function of the head part is the fusion of features and the mapping to the detection results.

[0090] The image sliding window processing of the real-time original image to generate the image set to be detected includes:

[0091] Evenly divide the real-time original image into 3*3 sub-blocks, crop each 2*2 sub-block to generate sub-block image information, enlarge the sub-block image information to the size of the original image, and use the sub-block image information and the real-time original image as the image set to be detected.

[0092] In the classic YOLOV5, the model input is a picture with a resolution of 640*640, and the inference output result is 20*20*3 + 40*40*3 + 80*80*3 = 25200 detection boxes. Each prediction box contains six pieces of information: the horizontal and vertical coordinates of the center point of the box, the width and height of the prediction box, the confidence of the prediction box, and the confidence corresponding to each classification. Since the sliding window method mentioned above is adopted in the present invention, 5 pictures are independently detected when the model is input, and the number of alternative boxes here will be 5 times the original. Therefore, in this step, it is necessary to map the detection results to the original image coordinates based on the relative position relationship between each sub-image and the original image and sort them in descending order of the confidence of the boxes.

[0093] Select the first 25200 boxes as the alternative boxes for the detection image, and continue the subsequent process of the classic YOLOV5.

[0094] In this embodiment, the detection results of the outfall detection model are optimized. It is agreed to use the detection rate (Precision) and the recall rate (Recall) as evaluation indicators and define the following concepts:

[0095] According to whether the sample is real and whether the prediction is correct, the detection results can basically be divided into four categories. TP (True Positive): The true class is positive and the predicted class is positive; FP (False Positive): The true class is negative and the predicted class is positive; FN (False Negative): The true class is positive and the predicted class is negative; TN (True Negative): The true class is negative and the predicted class is negative. The definitions of Recall and Precision are as follows:

[0096]

[0097] Image noise refers to unnecessary or redundant interference information existing in image data. In this article, image noise specifically refers to the complex environment that interferes with the identification of outlets in image detection.

[0098] The Hough transform is a feature extraction technique in image processing. This technique can obtain a set that conforms to a certain specific shape as the Hough transform result by calculating the local maximum of the cumulative result in a certain parameter space. The initial application of this technique was for line detection, and the present invention applies this method to the contour detection of outlets.

[0099] In the Cartesian coordinate system, given the coordinates of two points, the trajectory equation of a straight line can be uniquely determined, that is, the unique slope k0 and intercept b0 can be determined. Conversely, constructing a coordinate system (hereinafter referred to as the Hough coordinate system) with k and b as the independent variable and the dependent variable respectively, it is easy to obtain that the straight line in the Cartesian coordinate system corresponds to a definite point in the Hough coordinate system, as shown in the attached Figure 7 as follows:

[0100] All the straight line equations passing through a fixed point (x0, y0) on the Cartesian coordinate system satisfy: b = -x0k + y0. Therefore, all the straight line equations passing through a fixed point in the Cartesian coordinate system can be expressed as a straight line in the Hough coordinate system.

[0101] Based on the above inferences, the geometric features of a rectangle in the Cartesian coordinate system can be summarized to a certain extent in the Hough coordinate system. If there are two parallel lines (with the same slope k) in the Cartesian coordinate system, then the points corresponding to these two lines in the Hough coordinate system have the same k value. If there are two perpendicular lines in the Cartesian coordinate system, then their slopes are reciprocal to each other and intersect at the orthocenter. Therefore, the points corresponding to these two lines in the Hough coordinate system must be collinear.

[0102] Considering that the definition of a rectangle in the Cartesian coordinate system is a quadrilateral with opposite sides parallel to each other and adjacent sides perpendicular to each other. In the Hough coordinate system, the projection of a rectangle must be four points with the following geometric relationships,

[0103] The mapping points of two pairs of opposite sides respectively correspond to two identical k values.

[0104] The mapping points of adjacent sides satisfy the collinear relationship.

[0105] As shown in the Figure 8 specification appendix:

[0106] Considering that the value range of the slope k has the cases of infinity and 0, which is not conducive to the direct calculation of the computer. Therefore, it is necessary to replace all Hough space coordinates with the polar coordinate form, represented by (ρ, θ) (hereinafter referred to as the Hough polar coordinate space). Among them, ρ represents the line segment distance from the pole to this point, and θ represents the included angle between the line segment and the polar axis. In the Hough polar coordinate space, the straight line equation in the original Hough space will be converted into a trigonometric function equation, thus avoiding the problem that the slope cannot be calculated.

[0107] Any point (x0, y0) on a certain straight line in the Cartesian coordinate system can be mapped in the Cartesian polar coordinate space as:

[0108] x0 = ρ0cosθ0

[0109] y0 = ρ0sinθ0

[0110] It is easy to obtain:

[0111] x0cosθ0 = ρ0cosθ0 2

[0112] y0sinθ0 = ρ0sinθ0 2

[0113] x0cosθ0 + y0sinθ0 = ρ0

[0114] Therefore, in the Cartesian polar coordinate space, the equation of any fixed-point straight line is:

[0115] x0cosθ + y0sinθ = ρ

[0116] When mapped to the Hough polar coordinate system with the polar angle θ and the polar axis ρ as variables, this equation indicates that there will be countless trigonometric function curves passing through a certain fixed point in the coordinate system.

[0117] Based on the above principle, the Hough rectangle detection of the feature image set includes:

[0118] Record all the coordinates in the feature image whose coordinate values are preset values, denoted as the set

[0119] Points = [(x0, y0),...(x n , y n )];

[0120] Create an accumulative matrix M with a height of H, a width ranging from 0 to 180 degrees, and an initial value of 0. H is a preset multiple of the height of the image, and the preset multiple is greater than or equal to times;

[0121] For each coordinate point in Points, traverse from 0 degrees to 180 degrees, and calculate the corresponding ρ for the given θ based on the following formula i and fill the calculation result into the corresponding position of the accumulative matrix M; i x0 cosθ + y0sinθ = ρ, where x0 and y0 are the coordinate values of each coordinate point;

[0122]

[0123] Obtain the maximum value of each column in the matrix M, denoted as vector P;

[0124] Perform mean filtering (kernel_size = 10) on P to make the curve smoother, and denote the new vector as Q;

[0125] Perform NMS filtering on Q, mainly aiming to find the maximum value within the region range, and denote the new vector as QNms;

[0126] Compare QNms and Q bit by bit. Retain the values with the same comparison result in Q, and set the different ones to 0;

[0127] Determine that the maximum value in Q is a certain vertex P1 of the suspected rectangle. Search for whether there is a point P2 as another vertex of the rectangle within the neighborhood where the θ interval is exactly 90. If not, discard it;

[0128] Since P1 and P2 are the maximum values of the specified columns in the M matrix, then find the second - largest value from the same columns in the M matrix as the other two vertices of the rectangle;

[0129] Map the coordinates of the four vertices back to the Cartesian coordinate system and draw the rectangle by calculating the straight - line equation and the focus; Determine whether the area ratio of the rectangle reaches more than 50% of the whole image. If it meets the requirement, it is considered that the rectangle detection has passed.

[0130] The Hough circle detection for the feature image set includes:

[0131] Perform object detection on the original image Img0 to obtain a number of detection frames (B0...B n );

[0132] Based on the detection frames, perform cropping to obtain the cropped images (C0...C n );

[0133] Perform the following processing on the cropped image C i respectively:

[0134] Perform median filtering on the image to remove the noise contained in the background;

[0135] Perform edge detection on the filtered image to extract the contour of the detected object;

[0136] Traverse the white points in the image and calculate the normal equation of the point within its neighborhood. Accumulate all the points on this normal line once;

[0137] Find all possible center points according to the set lower threshold;

[0138] Traverse each center point P f Calculate the distance from all points to this center point to select a suitable radius; for the suitable radius, increment by 1 in the accumulator;

[0139] Select all the center points that meet the requirements and the circles with the highest radius value in their radius accumulators;

[0140] Compare these circles with the detection frames of the target detection respectively. If the offset degree between the center of the circle and the center of the detection frame is less than the threshold Threshold1, and the area of the circle accounts for more than the threshold Threshold2 of the area of the detection frame, it is considered that the detection meets the requirements.

[0141] The above describes the intelligent detection method for the river and sea water intake and discharge ports carried on the boat in the embodiments of the present invention. Next, the intelligent detection device for the river and sea water intake and discharge ports carried on the boat in the embodiments of the present invention will be described. Please refer to Figure 2 The intelligent detection device for the river and sea water intake and discharge ports carried on the boat in the embodiments of the present invention includes, for the above embodiments:

[0142] The river outlet dataset construction module 201 is used to construct a river outlet dataset and divide the river outlet data into a training set, a validation set, and a test set;

[0143] The outlet detection model construction module 202 is used to construct an outlet detection model based on the YOLOV5 model, train the outlet detection model using the training set, test the outlet detection model using the test set, and determine whether the outlet detection model meets the preset conditions. If so, the establishment of the outlet detection model is completed;

[0144] The image set to be detected generation module 203 is used to obtain the real-time video data collected by the camera, extract frames from the real-time video data to generate real-time original images; perform image sliding window processing on the real-time original images to generate an image set to be detected;

[0145] The feature image set generation module 204 is used to detect the image set to be detected using the outlet detection model to generate a feature image set;

[0146] An error image filtering module 205 is configured to perform Hough circle detection and Hough rectangle detection on the feature image set to filter out the error images in the feature image set;

[0147] A sunny day water flow phenomenon determination module 206 is configured to determine whether a sunny day water flow phenomenon occurs at the drain outlet according to the filtered feature image set; if so, report the abnormality of the drain outlet; if not, report the drain outlet information.

[0148] above Figure 2 The intelligent detection device for the river and sea water inlet and outlet on the boat in the embodiment of the present invention is described in detail from the perspective of modular functional entities. Next, the electronic device in the embodiment of the present invention will be described in detail from the perspective of hardware processing.

[0149] Figure 5 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. The electronic device 700 may vary greatly due to different configurations or performances, and may include one or more processors (central processing units, CPUs) 710 (for example, one or more processors) and a memory 720, and one or more storage media 730 for storing application programs 733 or data 732 (for example, one or more mass storage devices). Among them, the memory 720 and the storage media 730 may be transient storage or persistent storage. The program stored in the storage media 730 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the electronic device 700. Further, the processor 710 may be configured to communicate with the storage media 730 and execute a series of instruction operations in the storage media 730 on the electronic device 700.

[0150] The electronic device 700 may further include one or more power supplies 740, one or more wired or wireless network interfaces 750, one or more input / output interfaces 760, and / or, one or more operating systems 731, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, and so on. Those skilled in the art can understand that Figure 5 The shown structure of the electronic device does not constitute a limitation on the electronic device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0151] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions run on a computer, the computer is caused to execute the steps of the intelligent detection method for the river-entry and sea-entry water discharge outlets carried on a boat.

[0152] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described system, device, or unit can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0153] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0154] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An intelligent detection method for an outlet for water entering a river or sea carried on a boat, characterized in that: The intelligent detection method of the water discharge outlet for entering a river or sea carried on a boat comprises: Construct a river outlet data set and divide the river outlet data into training set, validation set and test set; Build an outlet detection model based on the YOLOV5 model, train the outlet detection model using the training set, and test the outlet detection model using the test set to determine whether the outlet detection model meets the preset conditions. If so, complete the establishment of the outlet detection model; Acquire real-time video data collected by the camera, extract frames from the real-time video data to generate real-time original images; perform image sliding window processing on the real-time original images to generate a set of images to be detected; Use the port detection model to detect the image set to be detected and generate a feature image set; Perform Hough circle detection and Hough rectangle detection on the feature image set to filter out erroneous images in the feature image set; According to the filtered feature image set, it is determined whether the outlet has a sunny day water flow phenomenon; if so, the outlet abnormality is reported; if not, the outlet information is reported; Wherein, the construction of the outlet detection model based on the YOLOV5 model includes: Build the YOLOV5 model, and add alternative detection boxes for small targets in the Anchors part of the YOLOV5 model. In the head of the YOLOV5 model, extract the information of the large-size image from the P3 / 8 layer, and merge this part of the information into the head information fusion part. In the head part of the YOLOV5 model, the newly fused content is fused again in the stage of generating the final detection result. The performing image sliding window processing on the real-time original image to generate the image set to be detected comprises: The real-time original image is divided into 3*3 sub-blocks, each 2*2 sub-block is cropped to generate sub-block image information, the sub-block image information is enlarged to the size of the original image, and the sub-block image information and the real-time original image are used as the image set to be detected.

2. The intelligent detection method for the water discharge outlet for entering a river or sea carried on a boat according to claim 1, characterized in that: The constructing of the river outlet data set and dividing the river outlet data into a training set, a validation set and a test set comprises: Obtain shoreline video data collected by the camera; Extract frames from shoreline video data to construct an original image set; The original images in the original image set are annotated using the Label Img image annotation tool, and the annotated original image data are divided into several categories. The original images of each category are divided into training set, validation set and test set in a ratio of 7:2:

1.

3. The intelligent detection method for the water discharge outlet for entering a river or sea carried on a boat according to claim 2, characterized in that: It also includes if the outlet detection model does not meet the preset conditions; Then the river outlet data set is reconstructed, and the river outlet data is divided into a training set, a validation set, and a test set; A discharge outlet detection model is built based on the YOLOV5 model, the discharge outlet detection model is trained using the training set, and the discharge outlet detection model is tested using the test set.

4. The intelligent detection method for the water discharge outlet for entering a river or sea carried on a boat according to claim 1, characterized in that: The Hough rectangle detection on the feature image set comprises: Record the coordinates of all the coordinates in the feature image with the preset values, and record them as the set Points = [(x0, y0), ... (x n ,y n )]; Create an accumulation matrix M with a height of H, a width of 0 to 180 degrees, and an initial value of 0. H is a preset multiple of the height of the image, and the preset multiple is greater than or equal to times; For each coordinate point in Points from 0 degrees to 180 degrees, calculate the given θ based on the following formula i The corresponding ρ i , and fill the calculation results into the corresponding positions of the accumulation matrix M; x0 cosθ+y0 sinθ=ρ Among them, x0, y0 are the coordinate values ​​of each coordinate point, ρ represents the distance of the line segment from the pole to the point, and θ represents the angle between the line segment and the polar axis; Get the maximum value of each column in the matrix M, recorded as vector P; Perform mean filtering on P with kernel_size=10 to make the curve smoother, and the resulting new vector is recorded as Q; Perform NMS filtering on Q to find the maximum value within the region, and the new vector obtained is recorded as QNms; Compare QNms and Q bit by bit. If the comparison result is the same in Q, keep it, and if it is different, set it to 0. Determine that the maximum value in Q is a vertex P1 of the suspected rectangle; find out whether there is a point P2 as another vertex of the rectangle in the neighborhood with a θ interval of exactly 90; if not, give up; It is known that P1 and P2 are the maximum values ​​of the specified column in the M matrix, so find the second largest value from the same column of the M matrix as the other two vertices of the rectangle; The coordinates of the four vertices are mapped back to the Cartesian coordinate system and a rectangle is drawn by calculating the equation of the line and the focus; it is determined whether the area of ​​the rectangle accounts for more than 50% of the entire image. If so, it is considered that the rectangle detection has passed.

5. The intelligent detection method for the water discharge outlet for entering a river or sea carried on a boat according to claim 1, characterized in that: The Hough circle detection on the feature image set comprises: Perform target detection on the original image Img0 and obtain several detection frames (B0...B n ); Crop based on the detection frame to obtain the cropped image (C0...C n ); For the cropped image C i Perform the following processing respectively: Perform median filtering on the image to remove noise contained in the background; Perform edge detection on the filtered image and extract the outline of the detection subject; Traverse the white points in the image and calculate the normal equation of the point in its neighborhood; accumulate all points on the normal once; Find all possible circle centers based on the set threshold lower limit; Traverse each circle center P f Calculate the distance from all points to the center of the circle, and select the appropriate radius; for the appropriate radius, add 1 to the accumulator; Select all circle centers that meet the requirements and the circles with the highest radius value in their radius accumulators; These circles are respectively checked against the detection frame of the target detection. If the deviation between the center of the circle and the center of the detection frame is less than the threshold Threshold1, and the area of ​​the circle occupies the area of ​​the detection frame greater than the threshold Threshold2, the detection is considered to meet the requirements.

6. An intelligent detection device for the water discharge outlet of a ship or boat, characterized in that: include: A river outlet data set construction module is used to construct a river outlet data set and divide the river outlet data into a training set, a validation set, and a test set; The outlet detection model construction module is used to construct the outlet detection model based on the YOLOV5 model, train the outlet detection model using the training set, test the outlet detection model using the test set, and determine whether the outlet detection model meets the preset conditions. If so, the outlet detection model is established; the outlet detection model is constructed based on the YOLOV5 model, including: constructing the YOLOV5 model, adding alternative detection boxes for small targets in the Anchors part of the YOLOV5 model; extracting information from large-size images of the P3 / 8 layer in the head of the YOLOV5 model, and merging the information of this part into the head information fusion part; the newly fused content in the head part of the YOLOV5 model is fused again in the stage of finally generating the detection result; The module for generating a set of images to be detected is used to obtain real-time video data collected by a camera, extract frames from the real-time video data to generate real-time original images; perform image sliding window processing on the real-time original images to generate a set of images to be detected; the process of performing image sliding window processing on the real-time original images to generate a set of images to be detected includes: dividing the real-time original images into 3*3 sub-blocks, cropping each 2*2 sub-block to generate sub-block image information, enlarging the sub-block image information to the size of the original image, and using the sub-block image information and the real-time original image as the set of images to be detected; A feature image set generation module, used to detect the image set to be detected using the outlet detection model to generate a feature image set; An error image filtering module is used to perform Hough circle detection and Hough rectangle detection on a feature image set to filter error images in the feature image set; The sunny day water flow phenomenon judgment module is used to judge whether the sunny day water flow phenomenon occurs at the outlet according to the filtered feature image set; if so, the outlet abnormality is reported; if not, the outlet information is reported.

7. An electronic device, comprising a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory so that the electronic device executes each step of the intelligent detection method for an outlet for entering a river or sea carried on a boat as described in any one of claims 1 to 5.

8. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the various steps of the intelligent detection method for the upper outlet for entering a river or sea water carried on a boat as claimed in any one of claims 1 to 5 are implemented.

Images