Intelligent detection method and device for river-entering and seawater-entering upper discharge port carried on boat

By equipping boats with intelligent detection systems and utilizing YOLOv5 models and Hough transform technology, river outlets can be automatically identified, solving the problems of low efficiency and low accuracy of manual inspections, and achieving efficient and accurate outlet detection and anomaly warning.

CN119007062AActive Publication Date: 2024-11-22SHANGHAI UBIQUITOUS NAVIGATION TECHNOLOGYCO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202410961097.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-17
Publication Date
2024-11-22
Estimated Expiration
2044-07-17

AI Technical Summary

Technical Problem

In existing technologies, river patrol and management mainly rely on manpower, resulting in low operation quality and efficiency, high labor costs, and the accuracy of manual patrols is not objective enough, making it difficult to respond quickly to emergencies and achieve scientific management of discharge outlets.

Method used

An intelligent detection method is adopted that is mounted on a vessel. A discharge port detection model is constructed using the YOLOv5 model. Combined with Hough circle and rectangle detection, video data is collected by a camera and image processing is performed to automatically identify and report discharge port anomalies, thereby achieving automated detection.

Benefits of technology

It has achieved efficient and low-cost outlet detection, improved detection accuracy and response speed, and ensured the standardization and rapid early warning capabilities of inspections.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007062A_ABST
    Figure CN119007062A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of water quality prediction, and discloses an intelligent detection method for a river-entering and seawater-entering upper discharge port carried on a boat. Constructing a river discharge port data set, and dividing river discharge port data into a training set, a verification set and a test set; constructing a mouth detection model based on a YOLOV5 model to obtain real-time video data collected by a camera, and performing frame extraction on the real-time video data to generate a real-time original image; performing image sliding window processing on the real-time original image to generate a to-be-detected image set; detecting the to-be-detected image set by using the discharge port detection model to generate a feature image set; performing Hough circle detection and Hough rectangle detection on the feature image set, and filtering wrong pictures in the feature image set; according to the filtered feature image set, whether a sunny day water flowing phenomenon occurs at the discharge port is judged; if yes, the discharge port abnormity is reported; if not, the exit information is reported; the method has the advantages of high checking efficiency and high accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of water quality prediction, and in particular to an intelligent detection method and device for an upper outlet for entering a river or sea carried on a boat. Background Art

[0002] Currently, river inspection and management work is mainly based on manpower, with a small number of semi-automatic river inspection products using drones as carriers.

[0003] The quality and efficiency of the project under manual inspection are relatively low. Operators need to walk along the river bank, and if they find an outlet, they need to take photos and register information (location, status, material, whether it is discharged on sunny days), etc. In addition, the installation locations of various outlets in the actual river are extremely complex, the work experience of the inspectors varies, and the actual inspection work is difficult. The scientific inspection and management of outlets has always been an important problem that is difficult to solve.

[0004] Manual inspection and re-inspection firstly require higher manpower costs, which sets a certain threshold for environmental protection companies. Secondly, the process of manual inspection and reporting information is far less responsive to emergency situations such as sunny day drainage than intelligent devices. More importantly, the accuracy of manual inspection is limited by the operational norms, work attitude and judgment experience of on-site operators, and is not objective enough. Once an engineering problem occurs, the results cannot be immediately fed back, which may eventually lead to unqualified project acceptance or additional cost investment.

[0005] Therefore, an intelligent detection method and device for an upper outlet for entering a river or sea water carried on a boat is provided to solve the above-mentioned problem. Summary of the invention

[0006] The main purpose of the present invention is to solve the problems in the prior art that river inspection and management work is mainly manpower-based, and there are a small number of semi-automatic river inspection products using drones as carriers; the operation quality and efficiency of projects under manual inspection are relatively low; manual investigation and re-inspection first require higher manpower costs, etc.

[0007] A first aspect of the present invention provides an intelligent detection method for an upper outlet for water entering a river or sea carried on a boat, the intelligent detection method for an upper outlet for water entering a river or sea carried on a boat comprising:

[0008] Construct a river outlet data set and divide the river outlet data into training set, validation set and test set;

[0009] Build an outlet detection model based on the YOLOV5 model, train the outlet detection model using the training set, and test the outlet detection model using the test set to determine whether the outlet detection model meets the preset conditions. If so, complete the establishment of the outlet detection model;

[0010] Acquire real-time video data collected by the camera, extract frames from the real-time video data to generate real-time original images; perform image sliding window processing on the real-time original images to generate a set of images to be detected;

[0011] Use the port detection model to detect the image set to be detected and generate a feature image set;

[0012] Perform Hough circle detection and Hough rectangle detection on the feature image set to filter out erroneous images in the feature image set;

[0013] According to the filtered feature image set, it is judged whether the outlet has the phenomenon of sunny water flow; if so, the outlet abnormality is reported; if not, the outlet information is reported.

[0014] Optionally, the constructing of the river outlet data set, dividing the river outlet data into a training set, a validation set and a test set comprises:

[0015] Obtain shoreline video data collected by the camera;

[0016] Extract frames from shoreline video data to construct an original image set;

[0017] The original images in the original image set are annotated using the LabelImg image annotation tool, and the annotated original image data are divided into several categories. The original images of each category are divided into training set, validation set and test set in a ratio of 7:2:1.

[0018] Optionally, it also includes if the outlet detection model does not meet the preset conditions;

[0019] Then the river outlet data set is reconstructed, and the river outlet data is divided into a training set, a validation set, and a test set;

[0020] A discharge outlet detection model is built based on the YOLOV5 model, the discharge outlet detection model is trained using the training set, and the discharge outlet detection model is tested using the test set.

[0021] Optionally, building a discharge outlet detection model based on the YOLOV5 model includes:

[0022] Build the YOLOV5 model, and add alternative detection boxes for small targets in the Anchors part of the YOLOV5 model. Extract the information of large-size images from the P3 / 8 layer in the head of the YOLOV5 model, and merge this part of information into the head information fusion part. The newly integrated content in the head part of the YOLOV5 model is integrated again in the stage of finally generating the detection results.

[0023] Optionally, performing image sliding window processing on the real-time original image to generate the image set to be detected includes:

[0024] The real-time original image is divided into 3*3 sub-blocks, each 2*2 sub-block is cropped to generate sub-block image information, the sub-block image information is enlarged to the size of the original image, and the sub-block image information and the real-time original image are used as the image set to be detected.

[0025] Optionally, performing Hough rectangle detection on the feature image set includes:

[0026] Record the coordinates of all the coordinates in the feature image with the preset values, and record them as the set

[0027] Points=[(x0,y0),...(x n ,y n )];

[0028] Create an accumulation matrix M with a height of H, a width of 0 to 180 degrees, and an initial value of 0. H is a preset multiple of the height of the image, and the preset multiple is greater than or equal to times;

[0029] For each coordinate point in Points from 0 degrees to 180 degrees, calculate the given θ based on the following formula i The corresponding ρ i , and fill the calculation results into the corresponding positions of the accumulation matrix M;

[0030] x0cosθ+y0sinθ=ρ, where x0, y0 are the coordinate values ​​of each coordinate point;

[0031] Get the maximum value of each column in the matrix M, recorded as vector P;

[0032] Perform mean filtering on P (kernel_size=10) to make the curve smoother, and the resulting new vector is recorded as Q;

[0033] The main purpose of performing NMS filtering on Q is to find the maximum value within the region, and the obtained new vector is recorded as QNms;

[0034] Compare QNms and Q bit by bit. If the comparison result is the same in Q, keep it, and if it is different, set it to 0.

[0035] Determine that the maximum value in Q is a vertex P1 of the suspected rectangle. Find out whether there is a point P2 as another vertex of the rectangle in the neighborhood with a θ interval of exactly 90. If not, give up;

[0036] It is known that P1 and P2 are the maximum values ​​of the specified column in the M matrix, so find the second largest value from the same column of the M matrix as the other two vertices of the rectangle;

[0037] The coordinates of the four vertices are mapped back to the Cartesian coordinate system and a rectangle is drawn by calculating the equation of the line and the focus; it is determined whether the area of ​​the rectangle accounts for more than 50% of the entire image. If so, it is considered that the rectangle detection has passed.

[0038] Optionally, performing Hough circle detection on the feature image set includes:

[0039] Perform target detection on the original image Img0 and obtain several detection frames (B0...B n );

[0040] Crop based on the detection frame to obtain the cropped image (C0...C n );

[0041] For the cropped image C i Perform the following processing respectively:

[0042] Perform median filtering on the image to remove noise contained in the background;

[0043] Perform edge detection on the filtered image and extract the contour of the detected subject;

[0044] Traverse the white points in the image and calculate the normal equation of the point in its neighborhood. Accumulate all points on the normal once;

[0045] Find all possible circle centers based on the set threshold lower limit;

[0046] Traverse each circle center P i Calculate the distance from all points to the center of the circle, and select the appropriate radius; for the appropriate radius, add 1 to the accumulator;

[0047] Select all circle centers that meet the requirements and the circles with the highest radius value in their radius accumulators;

[0048] These circles are respectively checked against the detection frame of the target detection. If the deviation between the center of the circle and the center of the detection frame is less than the threshold Threshold1, and the area of ​​the circle occupies the area of ​​the detection frame greater than the threshold Threshold2, the detection is considered to meet the requirements.

[0049] A second aspect of the present invention provides an intelligent detection device for an outlet for water entering a river or sea carried on a vessel, comprising:

[0050] A river outlet data set construction module is used to construct a river outlet data set and divide the river outlet data into a training set, a validation set, and a test set;

[0051] The outlet detection model construction module is used to build an outlet detection model based on the YOLOV5 model, train the outlet detection model using the training set, test the outlet detection model using the test set, and determine whether the outlet detection model meets the preset conditions. If so, the outlet detection model is established;

[0052] The module for generating the image set to be detected is used to obtain the real-time video data collected by the camera, extract frames from the real-time video data to generate real-time original images; and perform image sliding window processing on the real-time original images to generate the image set to be detected;

[0053] A feature image set generation module, used to detect the image set to be detected using the outlet detection model to generate a feature image set;

[0054] An error image filtering module is used to perform Hough circle detection and Hough rectangle detection on a feature image set to filter error images in the feature image set;

[0055] The sunny day water flow phenomenon judgment module is used to judge whether the sunny day water flow phenomenon occurs at the outlet according to the filtered feature image set; if so, the outlet abnormality is reported; if not, the outlet information is reported.

[0056] A third aspect of the present invention provides an electronic device, the electronic device comprising a memory and at least one processor, the memory storing instructions;

[0057] The at least one processor calls the instructions in the memory to enable the electronic device to execute the various steps of the intelligent detection method for the water discharge outlet for entering a river or sea carried on a boat as described above.

[0058] A fourth aspect of the present invention provides a computer-readable storage medium having instructions stored thereon, which, when executed by a processor, implement the various steps of the intelligent detection method for the surface outlet of water entering a river or sea carried on a vessel as described above.

[0059] In the technical solution of the present invention, the drainage outlet inspection can be carried out objectively and efficiently, and the standard specification of the inspection is guaranteed while reducing the cost; the present invention provides a complete process method from image data collection of boat operations to reporting of final detection results, which is easy to realize the construction of a whole system; the detection system of the present invention proposes an optimization method of Hough circle and Hough rectangle detection based on the actual scenario of drainage outlet inspection, as well as an improved method of yolov5 model based on small drainage outlets, which improves the detection capability and accuracy based on this scenario; compared with manual inspection, it can achieve faster early warning and management. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1A flow chart of an intelligent detection method for an upper outlet for entering a river or sea water carried on a boat provided by an embodiment of the present invention;

[0061] Figure 2 A schematic diagram of the structure of an intelligent detection device for an upper outlet for entering a river or sea water carried on a vessel provided by an embodiment of the present invention;

[0062] Figure 3 It is a schematic diagram of the working process of the present invention;

[0063] Figure 4 A schematic diagram of the sliding window segmentation of the present invention;

[0064] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention;

[0065] Figure 6 A comparison diagram of the basic model for outlet detection of the present invention and the original model;

[0066] Figure 7 It is a schematic diagram of the mutual mapping between Cartesian coordinate system and Hoffman coordinate system points;

[0067] Figure 8 Schematic diagram of the rectangle detection principle in the Huffman coordinate system. DETAILED DESCRIPTION

[0068] The embodiment of the present invention provides an intelligent detection method for water discharge outlets carried on boats and entering rivers or seas, including constructing a river discharge outlet data set, dividing the river discharge outlet data into a training set, a validation set and a test set; constructing a discharge outlet detection model based on a YOLOV5 model, training the discharge outlet detection model using the training set, testing the discharge outlet detection model using the test set, judging whether the discharge outlet detection model meets the preset conditions, and if so, completing the establishment of the discharge outlet detection model; acquiring real-time video data collected by a camera, extracting frames from the real-time video data to generate real-time original images; performing image sliding window processing on the real-time original images to generate a set of images to be detected; The outlet detection model is used to detect the image set to be detected to generate a feature image set; Hough circle detection and Hough rectangle detection are performed on the feature image set to filter out erroneous images in the feature image set; whether the outlet has a sunny day water flow phenomenon is determined according to the filtered feature image set; if so, the outlet abnormality is reported; if not, the outlet information is reported. The present invention solves the problems in the prior art that the river inspection and management work is mainly manpower-based, and there are a small number of semi-automatic river inspection products with drones as carriers; the operation quality and efficiency of the project under the manual inspection mode are relatively low; manual inspection and re-inspection first require higher labor costs, and the like.

[0069] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0070] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 The first embodiment of the intelligent detection method of the water discharge outlet for entering a river or sea carried on a boat in the embodiment of the present invention includes:

[0071] Construct a river outlet data set and divide the river outlet data into training set, validation set and test set;

[0072] Based on the YOLOV5 model, an outlet detection model is constructed, the outlet detection model is trained using the training set, and the outlet detection model is tested using the test set to determine whether the outlet detection model meets the preset conditions. If so, the outlet detection model is established; if the outlet detection model does not meet the preset conditions, the river outlet data set is reconstructed, and the river outlet data is divided into a training set, a validation set, and a test set;

[0073] Acquire real-time video data collected by the camera, extract frames from the real-time video data to generate real-time original images; perform image sliding window processing on the real-time original images to generate a set of images to be detected;

[0074] Use the port detection model to detect the image set to be detected and generate a feature image set;

[0075] Perform Hough circle detection and Hough rectangle detection on the feature image set to filter out erroneous images in the feature image set;

[0076] According to the filtered feature image set, it is judged whether the outlet has the phenomenon of sunny water flow; if so, the outlet abnormality is reported; if not, the outlet information is reported.

[0077] The constructing of the river outlet data set and dividing the river outlet data into a training set, a validation set and a test set comprises:

[0078] The shoreline video data collected by the camera is obtained; the collection of the shoreline video data can arrange the boat to sail at various angles in different positions and scenes, and collect video data.

[0079] Extract frames from shoreline video data to construct an original image set;

[0080] The original images in the original image set are annotated using the LabelImg image annotation tool, and the annotated original image data is divided into several categories. The specific classification methods include classification according to different scenes (rural areas, seaports, towns, etc.) and different materials (PVC, cement, copper); the original images of each category are divided into training sets, validation sets, and test sets in a ratio of 7:2:1. After the above dataset is constructed, the dataset can also be enhanced, including: horizontal flipping of the image, image brightness and contrast optimization

[0081] A discharge outlet detection model is built based on the YOLOV5 model, the discharge outlet detection model is trained using the training set, and the discharge outlet detection model is tested using the test set.

[0082] The outlet detection model constructed based on the YOLOV5 model includes:

[0083] Build the YOLOV5 model, and add alternative detection boxes for small targets in the Anchors part of the YOLOV5 model. Extract the information of large-size images from the P3 / 8 layer in the head of the YOLOV5 model, and merge this part of information into the head information fusion part. The newly integrated content in the head part of the YOLOV5 model is integrated again in the stage of finally generating the detection results.

[0084] The present invention adopts the YOLOV5 model as the basic model for outlet detection, and designs a new structure for small outlet detection based on the original model structure. For specific comparison, refer to the attached manual. Figure 6 ;

[0085] Based on the original model structure, the new structure of the present invention is designed as follows:

[0086] The New1 section is added to the Anchors section. This section is mainly to enable the model to add alternative detection boxes for small targets and enrich the alternative types of detection boxes.

[0087] The New2 part is added to the Head part. The main design of this part is to extract additional information from the large-size image of the P3 / 8 layer and merge this part of information into the head information fusion part. This design method helps to fuse the detailed features of the clearer large-size image, so that the model can enhance its recognition ability of detailed features.

[0088] A New3 part is added to the Head part. This part will cause the newly integrated information into the Head part to be integrated again in the final stage of generating the detection result.

[0089] The rest of the network structure is the main body of the YOLOV5 model network structure. It is mainly composed of four columns [from, number, module, args]. Among them, from represents the input source of the current module, and -1 represents the output of the previous layer as the input of this layer. Number represents the number of repetitions of the current module. Module represents the name of the module, and each named module has its own fixed network structure. Args represents some configuration parameters that may be required. The backbone part is the feature extraction part, and its main function is to extract the features of the detection target. The main function of the head part is the fusion of features and mapping to the detection results.

[0090] The performing image sliding window processing on the real-time original image to generate the image set to be detected comprises:

[0091] The real-time original image is divided into 3*3 sub-blocks, each 2*2 sub-block is cropped to generate sub-block image information, the sub-block image information is enlarged to the size of the original image, and the sub-block image information and the real-time original image are used as the image set to be detected.

[0092] In the classic YOLOV5, the model input is a picture with a resolution of 640*640, and the inference output result is 20*20*3+40*40*3+80*80*3=25200 detection boxes. Each predicted box contains 6 pieces of information, including the horizontal and vertical coordinates of the center point of the box, the width and height of the predicted box, the confidence of the predicted box, and the confidence corresponding to each classification. Since the present invention adopts the sliding window method mentioned above, 5 pictures are detected independently when the model is input, and the number of candidate boxes here will be 5 times the original. Therefore, this step needs to map the detection results to the original image coordinates based on the relative position relationship between each sub-image and the original image and arrange them in descending order according to the confidence of the box.

[0093] The first 25,200 boxes are intercepted as candidate boxes for detecting images, and the subsequent process of the classic YOLOV5 is continued.

[0094] In this embodiment, the detection results of the outlet detection model are optimized, and the detection rate (Precision) and recall rate (Recall) are used as evaluation indicators and the following concepts are agreed upon:

[0095] According to whether the sample is real and whether the prediction is correct, the test results can be basically divided into four categories. TP (True Positive): the real category is a positive example, and the predicted category is a positive example; FP (False Positive): the real category is a negative example, and the predicted category is a positive example; FN (False Negative): the real category is a positive example, and the predicted category is a negative example; TN (True Negative): the real category is a negative example, and the predicted category is a negative example. The definitions of Recall and Precision are as follows:

[0096]

[0097] Image noise refers to unnecessary or redundant interference information in image data. In this paper, image noise specifically refers to the complex environment that interferes with port recognition in image detection.

[0098] Hough transform is a feature extraction technology in image processing. This technology can obtain a set that conforms to a certain specific shape as the Hough transform result by calculating the local maximum of the cumulative result in a certain parameter space. This technology was first proposed and applied to straight line detection. The present invention applies this method to the contour detection of the outlet.

[0099] In the Cartesian coordinate system, the coordinates of two points can be uniquely determined to determine the trajectory equation of a straight line, that is, to determine the unique slope k0 and intercept b0. On the contrary, a coordinate system (hereinafter referred to as the Hough coordinate system) is constructed with k and b as the independent variable and dependent variable respectively. It is easy to obtain that the straight line in the Cartesian coordinate system corresponds to a certain point in the Hough coordinate system, as shown in the attached manual. Figure 7 As shown;

[0100] All equations of a straight line passing through a fixed point (x0, y0) in the Cartesian coordinate system satisfy: b = -x0k + y0. Therefore, all equations of a straight line passing through a fixed point in the Cartesian coordinate system can be expressed as a straight line in the Hough coordinate system.

[0101] Based on the above inferences, the geometric characteristics of rectangles in the Cartesian coordinate system can be summarized in the Hough coordinate system. If there are two parallel lines (with the same slope k) in the Cartesian coordinate system, then the points corresponding to these two lines in the Hough coordinate system have the same k value. If there are two perpendicular lines in the Cartesian coordinate system, then their slopes are reciprocal and intersect at the orthocenter, so the points corresponding to these two lines in the Hough coordinate system must be collinear.

[0102] Considering that the definition of a rectangle in the Cartesian coordinate system is a quadrilateral with parallel opposite sides and perpendicular adjacent sides, in the Hough coordinate system, the projection of a rectangle must be four points with the following geometric relationship:

[0103] The two sets of mapping points on opposite sides correspond to two identical k values.

[0104] The mapping points of adjacent edges satisfy the collinear relationship.

[0105] As the instruction manual Figure 8 As shown;

[0106] Considering that the range of slope k has infinity and 0, it is not conducive to direct calculation by computer. Therefore, it is necessary to replace all Hough space coordinates with polar coordinates in the form of (ρ, θ) (hereinafter referred to as Hough polar coordinate space). Among them, ρ represents the distance of the line segment from the pole to the point, and θ represents the angle between the line segment and the polar axis. In the Hough polar coordinate space, the straight line equation in the original Hough space will be converted into a trigonometric function equation, thereby avoiding the problem that the slope cannot be calculated.

[0107] Any point (x0, y0) on a straight line in the Cartesian coordinate system can be mapped in the Cartesian polar coordinate space as follows:

[0108] x0=ρ0cosθ0

[0109] y0=ρ0sinθ0

[0110] Easy to get:

[0111] x0cosθ0=ρ0cosθ0 2

[0112] y0sinθ0=ρ0sinθ0 2

[0113] x0cosθ0+y0sinθ0=ρ0

[0114] Therefore, in Cartesian polar coordinate space, the equation of a straight line at any fixed point is:

[0115] x0cosθ+y0sinθ=ρ

[0116] When mapped to the Hough polar coordinate system with polar angle θ and polar axis ρ as variables, the equation indicates that there will be countless trigonometric function curves passing through a fixed point in the coordinate system.

[0117] Based on the above principle, the Hough rectangle detection on the feature image set includes:

[0118] Record the coordinates of all the coordinates in the feature image with the preset values, and record them as the set

[0119] Points=[(x0,y0),...(x n ,y n )];

[0120] Create an accumulation matrix M with a height of H, a width of 0 to 180 degrees, and an initial value of 0. H is a preset multiple of the height of the image, and the preset multiple is greater than or equal to times;

[0121] For each coordinate point in Points from 0 degrees to 180 degrees, calculate the given θ based on the following formula i The corresponding ρ i , and fill the calculation results into the corresponding positions of the accumulation matrix M;

[0122] x0cosθ+y0sinθ=ρ, where x0, y0 are the coordinate values ​​of each coordinate point;

[0123] Get the maximum value of each column in the matrix M, recorded as vector P;

[0124] Perform mean filtering on P (kernel_size=10) to make the curve smoother, and the resulting new vector is recorded as Q;

[0125] The main purpose of performing NMS filtering on Q is to find the maximum value within the region, and the obtained new vector is recorded as QNms;

[0126] Compare QNms and Q bit by bit. If the comparison result is the same in Q, keep it, and if it is different, set it to 0.

[0127] Determine that the maximum value in Q is a vertex P1 of the suspected rectangle. Find out whether there is a point P2 as another vertex of the rectangle in the neighborhood with a θ interval of exactly 90. If not, give up;

[0128] It is known that P1 and P2 are the maximum values ​​of the specified column in the M matrix, so find the second largest value from the same column of the M matrix as the other two vertices of the rectangle;

[0129] The coordinates of the four vertices are mapped back to the Cartesian coordinate system and a rectangle is drawn by calculating the equation of the line and the focus; it is determined whether the area of ​​the rectangle accounts for more than 50% of the entire image. If so, it is considered that the rectangle detection has passed.

[0130] The Hough circle detection on the feature image set comprises:

[0131] Perform target detection on the original image Img0 and obtain several detection frames (B0...B n );

[0132] Crop based on the detection frame to obtain the cropped image (C0...C n );

[0133] For the cropped image C i Perform the following processing respectively:

[0134] Perform median filtering on the image to remove noise contained in the background;

[0135] Perform edge detection on the filtered image and extract the outline of the detection subject;

[0136] Traverse the white points in the image and calculate the normal equation of the point in its neighborhood. Accumulate all points on the normal once;

[0137] Find all possible circle centers based on the set threshold lower limit;

[0138] Traverse each circle center P i Calculate the distance from all points to the center of the circle, and select the appropriate radius; for the appropriate radius, add 1 to the accumulator;

[0139] Select all circle centers that meet the requirements and the circles with the highest radius value in their radius accumulators;

[0140] These circles are respectively checked against the detection frame of the target detection. If the deviation between the center of the circle and the center of the detection frame is less than the threshold Threshold1, and the area of ​​the circle occupies the area of ​​the detection frame greater than the threshold Threshold2, the detection is considered to meet the requirements.

[0141] The above describes the intelligent detection method of the upper outlet for river or sea water carried on a boat in an embodiment of the present invention. The following describes the intelligent detection device for the upper outlet for river or sea water carried on a boat in an embodiment of the present invention. Figure 2 In the embodiment of the present invention, the intelligent detection device for the discharge port of a ship for entering a river or sea water includes:

[0142] A river outlet data set construction module 201 is used to construct a river outlet data set, and divide the river outlet data into a training set, a validation set, and a test set;

[0143] The outlet detection model building module 202 is used to build an outlet detection model based on the YOLOV5 model, train the outlet detection model using a training set, test the outlet detection model using a test set, and determine whether the outlet detection model meets the preset conditions. If so, the outlet detection model is established;

[0144] The to-be-detected image set generation module 203 is used to obtain the real-time video data collected by the camera, extract frames from the real-time video data to generate real-time original images; perform image sliding window processing on the real-time original images to generate the to-be-detected image set;

[0145] The feature image set generating module 204 is used to detect the image set to be detected by using the outlet detection model to generate a feature image set;

[0146] The error image filtering module 205 is used to perform Hough circle detection and Hough rectangle detection on the feature image set to filter out the error images in the feature image set;

[0147] The sunny day water flow phenomenon judgment module 206 is used to judge whether the sunny day water flow phenomenon occurs at the outlet according to the filtered feature image set; if so, the outlet abnormality is reported; if not, the outlet information is reported.

[0148] above Figure 2 The intelligent detection device for the sea discharge outlet for entering a river or sea water carried on a vessel in an embodiment of the present invention is described in detail from the perspective of modular functional entities. The electronic device in an embodiment of the present invention is described in detail from the perspective of hardware processing.

[0149] Figure 5 7 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device 700 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 710 (for example, one or more processors) and a memory 720, and one or more storage media 730 (for example, one or more mass storage devices) storing application programs 733 or data 732. Among them, the memory 720 and the storage medium 730 can be temporary storage or permanent storage. The program stored in the storage medium 730 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the electronic device 700. Furthermore, the processor 710 can be configured to communicate with the storage medium 730 to execute a series of instruction operations in the storage medium 730 on the electronic device 700.

[0150] The electronic device 700 may also include one or more power supplies 740, one or more wired or wireless network interfaces 750, one or more input and output interfaces 750, and / or one or more operating systems 731, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 5 The structure of the electronic device shown does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine some components, or arrange the components differently.

[0151] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of an intelligent detection method for an outlet for water entering a river or sea carried on a boat.

[0152] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0153] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0154] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent detection method for an outlet for water entering a river or sea carried on a boat, characterized in that: The intelligent detection method of the water discharge outlet for entering a river or sea carried on a boat comprises: Construct a river outlet data set and divide the river outlet data into training set, validation set and test set; Build an outlet detection model based on the YOLOV5 model, train the outlet detection model using the training set, and test the outlet detection model using the test set to determine whether the outlet detection model meets the preset conditions. If so, complete the establishment of the outlet detection model; Acquire real-time video data collected by the camera, extract frames from the real-time video data to generate real-time original images; perform image sliding window processing on the real-time original images to generate a set of images to be detected; Use the port detection model to detect the image set to be detected and generate a feature image set; Perform Hough circle detection and Hough rectangle detection on the feature image set to filter out erroneous images in the feature image set; According to the filtered feature image set, it is judged whether the outlet has the phenomenon of water flowing on a sunny day; if so, the outlet abnormality is reported; if not, the outlet information is reported.

2. The intelligent detection method for the water discharge outlet for entering a river or sea carried on a boat according to claim 1, characterized in that: The constructing of the river outlet data set and dividing the river outlet data into a training set, a validation set and a test set comprises: Obtain shoreline video data collected by the camera; Extract frames from shoreline video data to construct an original image set; The original images in the original image set are annotated using the LabelImg image annotation tool, and the annotated original image data are divided into several categories. The original images of each category are divided into training set, validation set and test set in a ratio of 7:2:

1.

3. The intelligent detection method for the water discharge outlet for entering a river or sea carried on a boat according to claim 2, characterized in that: It also includes if the outlet detection model does not meet the preset conditions; Then the river outlet data set is reconstructed, and the river outlet data is divided into a training set, a validation set, and a test set; A discharge outlet detection model is built based on the YOLOV5 model, the discharge outlet detection model is trained using the training set, and the discharge outlet detection model is tested using the test set.

4. The intelligent detection method for the water discharge outlet for entering a river or sea carried on a boat according to claim 1, characterized in that: The outlet detection model constructed based on the YOLOV5 model includes: Build the YOLOV5 model, and add alternative detection boxes for small targets in the Anchors part of the YOLOV5 model. Extract the information of large-size images from the P3 / 8 layer in the head of the YOLOV5 model, and merge this part of information into the head information fusion part. The newly integrated content in the head part of the YOLOV5 model is integrated again in the stage of finally generating the detection results.

5. The intelligent detection method for the water discharge outlet for entering a river or sea carried on a boat according to claim 1, characterized in that: The performing image sliding window processing on the real-time original image to generate the image set to be detected comprises: The real-time original image is divided into 3*3 sub-blocks, each 2*2 sub-block is cropped to generate sub-block image information, the sub-block image information is enlarged to the size of the original image, and the sub-block image information and the real-time original image are used as the image set to be detected.

6. The intelligent detection method for the water discharge outlet for entering a river or sea carried on a boat according to claim 1, characterized in that: The Hough rectangle detection on the feature image set comprises: Record the coordinates of all the coordinates in the feature image with the preset values, and record them as the set Points = [(x0, y0), ... (x n ,y n )]; Create an accumulation matrix M with a height of H, a width of 0 to 180 degrees, and an initial value of 0, where H is a preset multiple of the height of the image, and the preset multiple is greater than or equal to 2√2 times; For each coordinate point in Points from 0 degrees to 180 degrees, calculate the given θ based on the following formula i The corresponding ρ i , and fill the calculation results into the corresponding positions of the accumulation matrix M; x0cosθ+y0sinθ=ρwhere x0, y0 are the coordinate values ​​of each coordinate point; Get the maximum value of each column in the matrix M, recorded as vector P; Perform mean filtering on P (kernel_size=10) to make the curve smoother, and the resulting new vector is recorded as Q; The main purpose of performing NMS filtering on Q is to find the maximum value within the region, and the obtained new vector is recorded as QNms; Compare QNms and Q bit by bit. If the comparison result is the same in Q, keep it, and if it is different, set it to 0. Determine that the maximum value in Q is a vertex P1 of the suspected rectangle. Find out whether there is a point P2 as another vertex of the rectangle in the neighborhood with a θ interval of exactly 90. If not, give up; It is known that P1 and P2 are the maximum values ​​of the specified column in the M matrix, so find the second largest value from the same column of the M matrix as the other two vertices of the rectangle; The coordinates of the four vertices are mapped back to the Cartesian coordinate system and a rectangle is drawn by calculating the equation of the line and the focus; it is determined whether the area of ​​the rectangle accounts for more than 50% of the entire image. If so, it is considered that the rectangle detection has passed.

7. The intelligent detection method for the water discharge outlet for entering a river or sea carried on a boat according to claim 1, characterized in that: The Hough circle detection on the feature image set comprises: Perform target detection on the original image Img0 and obtain several detection frames (B0...B n ); Crop based on the detection frame to obtain the cropped image (C0...C n ); For the cropped image C i Perform the following processing respectively: Perform median filtering on the image to remove noise contained in the background; Perform edge detection on the filtered image and extract the contour of the detected subject; Traverse the white points in the image and calculate the normal equation of the point in its neighborhood; accumulate all points on the normal once; Find all possible circle centers based on the set threshold lower limit; Traverse each circle center P i Calculate the distance from all points to the center of the circle, and select the appropriate radius; for the appropriate radius, add 1 to the accumulator; Select all circle centers that meet the requirements and the circles with the highest radius value in their radius accumulators; These circles are respectively checked against the detection frame of the target detection. If the deviation between the center of the circle and the center of the detection frame is less than the threshold Threshold1, and the area of ​​the circle occupies the area of ​​the detection frame greater than the threshold Threshold2, the detection is considered to meet the requirements.

8. An intelligent detection device for the water discharge outlet of a boat into a river or sea, characterized in that: include: A river outlet data set construction module is used to construct a river outlet data set and divide the river outlet data into a training set, a validation set, and a test set; The outlet detection model construction module is used to build an outlet detection model based on the YOLOV5 model, train the outlet detection model using the training set, test the outlet detection model using the test set, and determine whether the outlet detection model meets the preset conditions. If so, the outlet detection model is established; The module for generating the image set to be detected is used to obtain the real-time video data collected by the camera, extract frames from the real-time video data to generate real-time original images; and perform image sliding window processing on the real-time original images to generate the image set to be detected; A feature image set generation module, used to detect the image set to be detected using the outlet detection model to generate a feature image set; An error image filtering module is used to perform Hough circle detection and Hough rectangle detection on a feature image set to filter error images in the feature image set; The sunny day water flow phenomenon judgment module is used to judge whether the sunny day water flow phenomenon occurs at the outlet according to the filtered feature image set; if so, the outlet abnormality is reported; if not, the outlet information is reported.

9. An electronic device, comprising a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory so that the electronic device executes each step of the intelligent detection method for an outlet for entering a river or sea carried on a boat as described in any one of claims 1 to 7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the various steps of the intelligent detection method for the water discharge outlet for entering a river or sea carried on a boat as claimed in any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Video-image-based real-time identification method for drainage state of river channel drainage port

    CN113343923A

  • Method and device for detecting flow of water outlet based on image recognition

    CN113739863A

  • Urban drainage pipe network outlet abnormal drainage image identification method and system

    CN114419556A

  • Water quality detection method and device for water outlet, computer equipment and storage medium

    CN115980050A

  • Drainage port flow rapid detection method and device based on unmanned aerial vehicle

    CN117029937A