A video sewage detection method and system based on machine vision

Abstract

The present invention relates to the technical field of sewage discharge detection, and specifically discloses a video sewage discharge detection method and system based on machine vision, the method comprising: S100, arranging a full-color camera component and a fill light component at a sewage outlet to obtain image information of the sewage outlet, wherein the fill light component is used to adjust the fill light intensity and the fill light wavelength; obtaining adjustment parameters of the fill light component based on the image information; S200, obtaining monitoring image information under environmental conditions after the fill light component adjusts the parameters, slicing the monitoring image information according to preset time periods, and obtaining a frame image sequence in time order; S300, performing convolution processing on the frame image sequence after the slicing processing, and judging the sewage discharge status according to the classification result of the convolution processing result; in the sewage discharge judgment process, the method uses an algorithm at the video level as the processing level, can effectively use information in the time dimension, and is conducive to improving the accuracy of the results.

Description

Technical Field

[0001] The present invention relates to the technical field of sewage discharge detection, and in particular to a video sewage discharge detection method and system based on machine vision. Background Art

[0002] In the process of monitoring environmental conditions, monitoring of sewage discharge from pipelines, rivers and lakes is an important part. The monitoring method mainly uses water quality monitoring instruments to detect harmful substances in the water, judge the water quality based on the test results, and then realize continuous monitoring of sewage discharge status. However, the layout cost of water quality monitoring instruments is relatively high. At the same time, with the rapid development of machine learning technology, the sewage discharge status at the drainage outlet can be detected by obtaining image information at the drainage outlet for identification.

[0003] The existing technology mainly uses pixel optical flow velocity values ​​to determine whether a sewage outlet is discharging sewage. The obvious defect of this method is that other large-scale pixel changes in the image will also be judged as sewage discharge, such as rain or snow. When using image classification methods, although image classification methods can effectively determine whether there is sewage discharge, the coordinates of the sewage outlet need to be fixed, which is not suitable for large-scale deployment and has poor robustness. In addition, when the existing technology uses deep learning target detection, it processes a single frame of the image without considering that the video is a continuous frame, and cannot be combined with the time dimension for detection, which leads to poor robustness of the judgment result. Summary of the Invention

[0004] The purpose of the present invention is to provide a video sewage discharge detection method based on machine vision to solve the following technical problems:

[0005] How to improve the accuracy of sewage detection.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] A video sewage discharge detection method based on machine vision, the method comprising:

[0008] S100: Setting a full-color camera assembly and a fill light assembly at the sewage outlet to obtain image information of the sewage outlet, wherein the fill light assembly is used to adjust the fill light intensity and fill light wavelength;

[0009] Obtaining adjustment parameters of the fill light component based on the image information;

[0010] S200, obtaining monitoring image information under environmental conditions after adjusting parameters of the fill light component, slicing the monitoring image information according to preset time periods, and obtaining a frame image sequence in time order;

[0011] S300 , performing convolution processing on the frame image sequence after the slicing processing, and judging the sewage discharge state according to the classification result of the convolution processing result.

[0012] Furthermore, the convolution processing process includes:

[0013] S301, performing a conventional convolution operation on each frame image in the frame image sequence, and extracting four feature maps from each frame image;

[0014] S302, performing a shift operation on the extracted feature map;

[0015] The shift operation process includes:

[0016] Taking the channel of the feature map as the dimension, move the first dimension feature map downward, move the fourth dimension feature map upward, and keep the second dimension feature map and the third dimension feature map unchanged;

[0017] The content that moves out of bounds is discarded, and the gaps at the beginning and end are filled with feature maps of all 0s;

[0018] S303, using convolution processing on the feature map processed by the shift operation, using the residual structure to perform full connection processing on the result feature map, and performing classification calculation;

[0019] Determine the sewage discharge status based on the classification calculation results.

[0020] Furthermore, the convolution processing process is implemented based on the TSM model.

[0021] Furthermore, the process of obtaining the adjustment parameters of the fill light component includes:

[0022] Extract key frames based on the collected images and convert the key frames into grayscale;

[0023] By formula Calculate the brightness coefficient G of the current key frame L ;

[0024] The brightness coefficient G L With the preset threshold G thr To compare:

[0025] If G L ≥G thr , then fill light is performed, and the fill light intensity is L add =L0+f(G L -G thr );

[0026] Otherwise, the fill light intensity L add =0;

[0027] Among them, N is the number of pixels collected in the key frame, i∈[1,N]; grey i is the grayscale value of the i-th pixel; gt is the grayscale reference value, Indicates the number of pixels with grayscale values ​​greater than gt, x1 and x2 are preset coefficients; L0 is the reference brightness value, f is the comparison function, and f is an increasing function.

[0028] Furthermore, the process of obtaining the adjustment parameters of the fill light component further includes:

[0029] According to the fill light intensity L add and preset wavelength W pre Adjust the fill light component to collect image information under the current environmental conditions;

[0030] Identify the sewage area in the image information and collect M pixels;

[0031] By formula Get the first component F1 of the current sewage area color R 、F1 G 、F1 B ;

[0032] Among them, j∈[1,M]; R1 j To adjust the value of the red component of the j-th pixel; G1 j To adjust the value of the green component of the j-th pixel; B1 j To adjust the value of the blue component of the j-th pixel;

[0033] According to F1 R 、F1 G 、F1 B The range of the value determines the color range of the sewage color in the current area;

[0034] The wavelength judgment strategy is determined according to the color range of the sewage color in the current area, and the final fill light wavelength is obtained according to the result of the wavelength judgment strategy.

[0035] Furthermore, the color intervals are pre-set, and each color interval is correspondingly provided with a test fill light wavelength;

[0036] Adjust the wavelength of the fill light component to the test fill light wavelength corresponding to the color interval of the sewage color in the current area, and collect image information under the environmental state of the test fill light wavelength;

[0037] Identify the sewage area in the image information and collect P pixels;

[0038] By formula Get the second component F2 of the current sewage area colorR 、F2 G 、F2 B ;

[0039] Where k∈[1,P]; R2 k G2 is the value of the red component of the k-th pixel after adjustment; k B2 is the value of the green component of the k-th pixel after adjustment; k is the value of the blue component of the k-th pixel after adjustment;

[0040] By the formula ΔF R =F2 R -F1 R , ΔF G =F2 G -F1 G , ΔF B =F2 B -F1 B Calculate the component difference ΔF R , ΔF G , ΔF B ;

[0041] According to ΔF R , ΔF G , ΔF B and F1 R 、F1 G 、F1 B Determine the final fill light wavelength.

[0042] Furthermore, the process of obtaining the wavelength of the fill light is as follows:

[0043] Pre-set multiple color change intervals corresponding to each color interval, and set the optimal detection wavelength for each color change interval;

[0044] According to F1 R 、F1 G 、F1 B Determine the color interval, according to ΔF R , ΔF G , ΔF B The optimal wavelength is determined based on the color change range, and the optimal wavelength is used as the fill light wavelength.

[0045] A video pollution detection system based on machine vision, comprising an image acquisition component, a fill light component and an analysis module, and used to execute a video pollution detection method based on machine vision.

[0046] Beneficial effects of the present invention:

[0047] (1) In the process of sewage discharge judgment, the present invention uses an algorithm at the video level for processing, which can effectively use information in the time dimension, which is conducive to improving the accuracy of the results; at the same time, the fill light component obtains the adjustment parameters of the fill light component according to the image information, and then the image information can be obtained more clearly and accurately through the adjusted fill light intensity and fill light wavelength, thereby achieving a more accurate judgment of the sewage discharge status during the convolution processing.

[0048] (2) The present invention improves the accuracy of the judgment results in the subsequent convolution processing of the image information by obtaining the adjustment parameters of the fill light component; at the same time, it ensures the uniformity of the image state collected under different environmental conditions, and judges the color range of the sewage detection area and its corresponding characteristics under the change of light wavelength, thereby being able to judge the color state of the water quality, and then adjust the fill light component by selecting an adaptive wavelength, thereby making the sewage characteristics in the acquired image information more obvious, thereby improving the accuracy of the analysis results; at the same time, the method of obtaining the color change amount through wavelength adjustment can reduce the influence of the environmental color on the water quality color state judgment process, thereby making the determined fill light wavelength more compatible with the current water quality state. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The present invention will be further described below with reference to the accompanying drawings.

[0050] Figure 1 This is a flowchart of the steps of the video sewage detection method based on machine vision of the present invention;

[0051] Figure 2 It is a flow chart of the steps of the convolution processing process of the present invention. DETAILED DESCRIPTION

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0053] See also Figure 1 As shown, in one embodiment, a video sewage discharge detection method based on machine vision is provided, the method comprising:

[0054] S100: Setting a full-color camera assembly and a fill light assembly at the sewage outlet to obtain image information of the sewage outlet, wherein the fill light assembly is used to adjust the fill light intensity and fill light wavelength;

[0055] Obtaining adjustment parameters of the fill light component based on the image information;

[0056] S200, obtaining monitoring image information under environmental conditions after adjusting parameters of the fill light component, slicing the monitoring image information according to preset time periods, and obtaining a frame image sequence in time order;

[0057] S300 , performing convolution processing on the frame image sequence after the slicing processing, and judging the sewage discharge state according to the classification result of the convolution processing result.

[0058] Through the above technical solution, the detection method in this embodiment first takes into account the time dimension of the pollution discharge algorithm in the process of pollution discharge judgment. The algorithm with the processing level being the video level can effectively use the information of the time dimension, which is conducive to improving the accuracy of the results; at the same time, the adjustment parameters of the fill light component are obtained according to the image information through the fill light component, and then the image information can be obtained more clearly and accurately through the adjusted fill light intensity and fill light wavelength, thereby achieving a more accurate judgment of the pollution discharge status during the convolution processing process.

[0059] As an embodiment of the present invention, please refer to Figure 2 As shown, the convolution processing process includes:

[0060] S301, performing a conventional convolution operation on each frame image in the frame image sequence, and extracting four feature maps from each frame image;

[0061] S302, performing a shift operation on the extracted feature map;

[0062] The shift operation process includes:

[0063] Taking the channel of the feature map as the dimension, move the first dimension feature map downward, move the fourth dimension feature map upward, and keep the second dimension feature map and the third dimension feature map unchanged;

[0064] The content that moves out of bounds is discarded, and the gaps at the beginning and end are filled with feature maps of all 0s;

[0065] S303, using convolution processing on the feature map processed by the shift operation, using the residual structure to perform full connection processing on the result feature map, and performing classification calculation;

[0066] Determine the sewage discharge status based on the classification calculation results.

[0067] The convolution processing process is implemented based on the TSM model.

[0068] Through the above technical solution, the convolution processing process in this embodiment is implemented based on the TSM model. TSM is an algorithm with video input and a video processing level. It can effectively use information in the time dimension, which is conducive to improving the accuracy of the results. At the same time, compared with traditional image judgment methods, it can extract all valid semantics of video images. When the data volume is sufficient and the marking is correct, it will not misjudge the situation of no sewage discharge under climatic conditions such as rain and snow, and can promptly detect the sewage outlet in the image without deliberately marking the sewage outlet, which can be conducive to large-scale deployment.

[0069] At the same time, the convolution processing process includes S301, performing a conventional convolution operation on each frame image in the frame image sequence, and extracting 4 feature maps from each frame image; S302, performing a shift operation on the extracted feature map; the shift operation process includes: taking the channel of the feature map as the dimension, moving the first dimension feature map down, moving the fourth dimension feature map up, and keeping the second dimension feature map and the third dimension feature map stationary; discarding the content that moves out of bounds, and filling the gaps at the beginning and end with feature maps of all 0; S303, using convolution processing on the feature map after the shift operation, using the residual structure to fully connect the resulting feature map, and performing classification calculation; judging the sewage discharge status according to the classification calculation results; therefore, through the above process, using the optimized residual model, a good time series modeling can be obtained, and then in the convolution analysis process, the time dimension in the collected image information can be fully utilized for judgment, thereby improving the accuracy of the judgment result.

[0070] As an embodiment of the present invention, the process of obtaining the adjustment parameters of the fill light component includes:

[0071] Extract key frames based on the collected images and convert the key frames into grayscale;

[0072] By formula Calculate the brightness coefficient G of the current key frame L ;

[0073] The brightness coefficient G L With the preset threshold G thr To compare:

[0074] If G L ≥G thr , then fill light is performed, and the fill light intensity is L add =L0+f(G L -G thr );

[0075] Otherwise, the fill light intensity L add =0;

[0076] Among them, N is the number of pixels collected in the key frame, i∈[1,N]; grey i is the grayscale value of the i-th pixel; gt is the grayscale reference value, Indicates the number of pixels with grayscale values ​​greater than gt, x1 and x2 are preset coefficients; L0 is the reference brightness value, f is the comparison function, and f is an increasing function.

[0077] Through the above technical solution, this embodiment provides a method for obtaining adjustment parameters of a fill light component, specifically a method for obtaining fill light intensity, by extracting key frames from the collected image information, graying the key frames, and using the formula The brightness state of the image is judged, and then the lighting state of the environment is indirectly judged. L ≥G thr The fill light component is used to perform fill light operation to ensure that the collected image clearly reflects the water quality characteristics, and improve the accuracy of the judgment result in the subsequent convolution processing of the image information; In addition, in order to ensure the uniformity of the image state collected under different environmental conditions, this embodiment uses the formula L add =L0+f(G L -G thr ) Get the fill light intensity L add , that is, adaptability to meet the requirements of image uniformity in different environments.

[0078] It should be noted that the number of collected pixel points N of the key frame in the above technical solution is selectively set according to the accuracy requirements and computing power status, and is not limited here. The grayscale reference value gt and the reference brightness value L0 are selected and set according to empirical data, and the control function f is obtained after simulation based on the test data. The preset coefficients x1 and x2 are fixed values, which are obtained by fitting based on empirical data and are not described in detail here.

[0079] As an embodiment of the present invention, the process of obtaining the adjustment parameters of the fill light component further includes:

[0080] According to the fill light intensity L add and preset wavelength W pre Adjust the fill light component to collect image information under the current environmental conditions;

[0081] Identify the sewage area in the image information and collect M pixels;

[0082] By formula Get the first component F1 of the current sewage area color R 、F1 G 、F1 B ;

[0083] Among them, j∈[1,M]; R1j To adjust the value of the red component of the j-th pixel; G1 j To adjust the value of the green component of the j-th pixel; B1 j To adjust the value of the blue component of the j-th pixel;

[0084] According to F1 R 、F1 G 、F1 B The range of the value determines the color range of the sewage color in the current area;

[0085] The wavelength judgment strategy is determined according to the color range of the sewage color in the current area, and the final fill light wavelength is obtained according to the result of the wavelength judgment strategy.

[0086] The color intervals are pre-set, and each color interval is correspondingly provided with a test fill light wavelength;

[0087] Adjust the wavelength of the fill light component to the test fill light wavelength corresponding to the color interval of the sewage color in the current area, and collect image information under the environmental state of the test fill light wavelength;

[0088] Identify the sewage area in the image information and collect P pixels;

[0089] By formula Get the second component F2 of the current sewage area color R 、F2 G 、F2 B ;

[0090] Where k∈[1,P]; R2 k G2 is the value of the red component of the k-th pixel after adjustment; k B2 is the value of the green component of the k-th pixel after adjustment; k is the value of the blue component of the k-th pixel after adjustment;

[0091] By the formula ΔF R =F2 R -F1 R , ΔF G =F2 G -F1 G , ΔF G =F2 B -F1 B Calculate the component difference ΔF R , ΔF G , ΔF B ;

[0092] According to ΔF R , ΔF G, ΔF B and F1 R 、F1 G 、F1 B Determine the final fill light wavelength.

[0093] The process of obtaining the wavelength of the fill light is as follows:

[0094] Pre-set multiple color change intervals corresponding to each color interval, and set the optimal detection wavelength for each color change interval;

[0095] According to F1 R 、F1 G 、F1 B Determine the color interval, according to ΔF R , ΔF G , ΔF B The optimal wavelength is determined based on the color change range, and the optimal wavelength is used as the fill light wavelength.

[0096] Through the above technical solution, this embodiment provides a method for obtaining the wavelength of fill light. Due to different environmental conditions and water quality conditions, the water quality characteristics in the acquired image will have a certain impact on the accuracy of the result during the convolution process due to the problem of clarity. In this embodiment, the fill light intensity L is obtained. add and preset wavelength W pre By judging the color range of the sewage detection area and correspondingly judging its characteristics under the change of light wavelength, the color state of the water quality can be judged, and then the fill light component is adjusted by selecting an adaptive wavelength, so that the sewage characteristics in the obtained image information are more obvious, thereby improving the accuracy of the analysis results; in this process, the method of obtaining the color change amount by wavelength adjustment in this embodiment can reduce the influence of the environmental color on the water quality color state judgment process, thereby making the determined fill light illumination wavelength more compatible with the current water quality state.

[0097] It should be noted that the preset wavelength W in the above technical solution pre The set color range and the test fill light wavelength corresponding to the color range, the color change range and its corresponding optimal detection wavelength are all set after analyzing the empirical detection data under different environmental conditions, and will not be further described here.

[0098] In one embodiment, a video pollution detection system based on machine vision is also provided. The system includes an image acquisition component, a fill light component and an analysis module. The system is used to execute a video pollution detection method based on machine vision.

[0099] The above is a detailed description of an embodiment of the present invention. However, the content described is only a preferred embodiment of the present invention and should not be considered to limit the scope of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A video sewage detection method based on machine vision, characterized in that: The method comprises: S100: Setting a full-color camera assembly and a fill light assembly at the sewage outlet to obtain image information of the sewage outlet, wherein the fill light assembly is used to adjust the fill light intensity and fill light wavelength; Obtaining adjustment parameters of the fill light component based on the image information; S200, obtaining monitoring image information under environmental conditions after adjusting parameters of the fill light component, slicing the monitoring image information according to preset time periods, and obtaining a frame image sequence in time order; S300, performing convolution processing on the frame image sequence after the slicing processing, and judging the sewage discharge state according to the classification result of the convolution processing result; The process of obtaining the adjustment parameters of the fill light component includes: Extract key frames based on the collected images and convert the key frames into grayscale; By formula Calculate the brightness coefficient of the current key frame ; The brightness coefficient With preset threshold To compare: like , then fill light is performed, and the fill light intensity ; Otherwise, fill light intensity ; Where N is the number of pixels collected in the key frame, i∈[1,N]; is the grayscale value of the i-th pixel; gt is the grayscale reference value, Indicates the number of pixels with grayscale values ​​greater than gt, 、 is the preset coefficient; is the reference brightness value, f is the comparison function, and f is the increasing function; The process of obtaining the adjustment parameters of the fill light component further includes: According to the fill light intensity and preset wavelength Adjust the fill light component to collect image information under the current environmental conditions; Identify the sewage area in the image information and collect M pixels; By formula 、 、 Get the first component of the current sewage area color 、 、 ; Where, j∈[1,M]; To adjust the value of the red component of the j-th pixel; To adjust the value of the green component of the j-th pixel; To adjust the value of the blue component of the j-th pixel; According to 、 、 The range of the value determines the color range of the sewage color in the current area; The wavelength judgment strategy is determined according to the color range of the sewage color in the current area, and the final fill light wavelength is obtained according to the result of the wavelength judgment strategy.

2. The method for detecting sewage discharge by video based on machine vision according to claim 1, characterized in that: The convolution process includes: S301, performing a conventional convolution operation on each frame image in the frame image sequence, and extracting four feature maps from each frame image; S302, performing a shift operation on the extracted feature map; The shift operation process includes: Taking the channel of the feature map as the dimension, move the first dimension feature map downward, move the fourth dimension feature map upward, and keep the second dimension feature map and the third dimension feature map unchanged; The content that moves out of bounds is discarded, and the gaps at the beginning and end are filled with feature maps of all 0s; S303, using convolution processing on the feature map processed by the shift operation, using the residual structure to perform full connection processing on the result feature map, and performing classification calculation; Determine the sewage discharge status based on the classification calculation results.

3. The video sewage detection method based on machine vision according to claim 2 is characterized in that: The convolution processing process is implemented based on the TSM model.

4. The method for detecting sewage discharge using video based on machine vision according to claim 1, wherein: The color intervals are pre-set, and each color interval is correspondingly provided with a test fill light wavelength; Adjust the wavelength of the fill light component to the test fill light wavelength corresponding to the color interval of the sewage color in the current area, and collect image information under the environmental state of the test fill light wavelength; Identify the sewage area in the image information and collect P pixels; By formula 、 、 Get the second component of the current sewage area color 、 、 ; Where k∈[1,P]; is the value of the red component of the k-th pixel after adjustment; is the value of the green component of the k-th pixel after adjustment; is the value of the blue component of the k-th pixel after adjustment; By formula 、 、 Calculate the component difference 、 、 ; according to 、 、 and 、 、 Determine the final fill light wavelength.

5. The method for detecting sewage discharge by video based on machine vision according to claim 4, characterized in that: The process of obtaining the wavelength of the fill light is as follows: Pre-set multiple color change intervals corresponding to each color interval, and set the optimal detection wavelength for each color change interval; according to 、 、 Determine the color range, according to 、 、 The optimal wavelength is determined based on the color change range, and the optimal wavelength is used as the fill light wavelength.

6. A video sewage detection system based on machine vision, characterized in that: The system includes an image acquisition component, a fill light component and an analysis module, and the system is used to execute the video pollution detection method based on machine vision as described in any one of claims 1 to 5.

Images