A river congestion condition judgment system and method based on deep learning, and a storage medium

Abstract

The application provides a river congestion condition judgment system and method based on deep learning, and a storage medium, the method uses a position information positioning module in a detection device to acquire geographic coordinate information of a current detection device in real time, verifies whether the flow rate of the current river position is normal by means of a flow rate information acquisition module, when there is an abnormal change, uses an image acquisition unit to shoot an image of the river at the geographic position and submits the image to a server for river congestion degree determination, the server stores a trained neural network model, the neural network model can automatically analyze the congestion degree corresponding to the current river geographic position in combination with the image information uploaded by the detection device and the geographic data corresponding to the current geographic position information, and output processing strategy information to the detection device or a mobile terminal. Through the above scheme, the accuracy and timeliness of the river congestion condition judgment can be improved, thereby improving the monitoring efficiency of the river congestion condition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a river congestion condition judgment system and method based on deep learning and a storage medium. BACKGROUND

[0002] The current river dredging management method is mainly to uniformly carry out river dredging engineering by the relevant part when the river is seriously silted and threatens the safety of the river basin. When the river is partially silted or temporarily silted due to a sudden situation, the existing method cannot timely discover and has poor real-time performance. Moreover, the hydrogeology and human environment of different rivers are different, so the siltation standards in the rivers are different. The existing judgment of the congestion conditions of different rivers is often carried out by manually carrying detection equipment to the relevant river positions for regular and fixed-point screening, which has low accuracy and efficiency and is time-consuming and laborious. SUMMARY

[0003] In view of the above problems, the present application provides a river congestion condition judgment system and method based on deep learning and a storage medium, which solves the problems of low efficiency and low accuracy of the existing screening method of the river congestion condition.

[0004] To achieve the above-mentioned purpose, in a first aspect, the present application provides a river congestion condition judgment system based on deep learning. The system comprises a detection device and a server. The detection device comprises a position information positioning module, a flow rate information acquisition module, a control module and an image acquisition unit. The position information positioning module is used to acquire the geographical position information of the current river in real time. The flow rate information acquisition module is used to acquire the flow rate information of the water flow in the current river in real time. The control module is used to send an image acquisition control signal to the image acquisition unit when the difference between the change value of the flow rate information corresponding to a certain geographical position information and the preset flow rate information of the geographical position information in the historical database is greater than a preset error value. The image acquisition unit is used to acquire the image information of the current river according to the image acquisition control signal and return the acquired image information to the control module.

[0005] The server is used to receive the image information uploaded by the control module and the geographical data corresponding to the current geographical position information, input the image information and the geographical data into the trained neural network model, output the congestion degree information corresponding to the current river, determine the corresponding processing strategy information according to the congestion degree information corresponding to the current river, and issue the processing strategy information to the detection device or the mobile terminal.

[0006] In some embodiments, the control module is further configured to, when determining that the change value of the flow rate information corresponding to the geographic location information is greater than the preset error value from the preset flow rate information of the geographic location information in the historical database, acquire the river tributary direction information corresponding to the current geographic location information, and generate a direction adjustment instruction according to the river tributary direction information; the detection device further comprises a steering driving unit, which is configured to adjust the shooting angle of the image acquisition unit according to the direction adjustment instruction; the image acquisition unit is further configured to acquire image information before and after the shooting angle adjustment, and after obtaining the first image information corresponding to the current river and the second image information corresponding to the tributary river of the current river, the first image information and the second image information are transmitted back to the control module.

[0007] In some embodiments, when the control module acquires that the current river does not have corresponding river tributary information, the image acquisition unit is configured to acquire the first image information corresponding to the current river and transmit it back to the control module; the control module is configured to send the first image information to the server, and receive the congestion degree information and processing strategy information corresponding to the first image information fed back by the server.

[0008] In some embodiments, when the congestion degree information of the current river is not congested, the control module sends a prompt information and reports it to the server or the mobile terminal.

[0009] In some embodiments, the neural network model is trained in the following manner:

[0010] A neural network model is constructed, the input parameters of the neural network model are sample images and geographic data in a sample database, and the output results are the congestion degree information of the river and the corresponding processing strategy information; the sample images include river image information without congestion and multiple river image information with different congestion degrees; the geographic data includes any one or more of river length data, rainfall data on both banks of the river, and river geographic shape features; the sample images and the geographic data corresponding to the sample images in the sample database are repeatedly trained to improve the accuracy of the neural network model, and a trained neural network model is obtained.

[0011] In some embodiments, repeatedly training the neural network model by reading the sample images and the geographic data corresponding to the sample images in the sample database to improve the accuracy of the neural network model, and obtaining a trained neural network model comprises:

[0012] A group of training sample images are taken, the geographic data corresponding to the group of sample images, the sample image features, and the congestion degree information are input into the neural network;

[0013] The neural network adopts a gradient algorithm to perform matrix operation on the weight of the geographical data corresponding to the sample image and the weight of the sample image feature, and sets a basic value for the learning rate and the offset of the learning rate. After the operation is completed, a mathematical model about the input sample image and the congestion type and congestion degree information corresponding to the sample image is obtained; the weight of the geographical data is calculated according to the rainfall data and the geographical shape features of the river channel; the weight of the sample image is determined according to the proportion of the pixel points of the congestion position in the sample image; and the congestion type is determined according to the position distribution of the congestion in the sample image.

[0014] A set of training data is replaced to perform operation again. The model is corrected by the gradient algorithm and becomes closer to the real model reflected by the data each time the operation is performed.

[0015] After all the training sample data sets are replaced, one round of training is completed. After at least one round of training is completed, the model is evaluated.

[0016] The training sample image of the test is read to evaluate the model. The input sample image is obtained from the model, and the output congestion degree information y1 is obtained. The actual value y of the congestion degree information corresponding to the sample image is compared. If the theoretical value y1 is equal to the actual value y, it is recorded as a prediction success, that is, the recognition is correct. The model is evaluated by the accuracy rate:

[0017]

[0018] If the accuracy rate a of the mathematical model is greater than or equal to 80%, the model is considered suitable, and the code is saved. Otherwise, the model is optimized by increasing the sample data, increasing the number of training rounds, fine-tuning the learning rate and / or offset algorithm parameters.

[0019] The finally optimized model is deployed on the server and combined with the front-end interface function. The front-end interface is used to receive the river channel image uploaded by the control module.

[0020] In some embodiments, the position of a congestion in a sample image is determined in the following manner:

[0021] The gray value of each pixel point on the sample image is sequentially determined. When the gray value is greater than a preset value, the pixel point is set to 0, and when the gray value is less than the preset value, the pixel point is set to 1.

[0022] All the pixel points set to 1 are sequentially subjected to expansion processing and corrosion processing to determine a plurality of connected domains of 1 with large sizes on the current sample image after the expansion and corrosion processing.

[0023] The plurality of connected domains of 1 with large sizes are determined as the positions of the congestions.

[0024] The expansion processing includes: setting all the pixel points in the 8-neighborhood of each pixel point as the center to 1;

[0025] The erosion processing includes: judging whether there is a pixel point set to 0 in the 8-neighborhood of each pixel point after the expansion processing, if yes, eliminating the pixel point; otherwise, keeping the pixel point.

[0026] In some embodiments, the input parameters of the neural network further include historical siltation data of the current river channel, and the congestion degree information is calculated according to the geographical data of the current river channel, the collected image information of the river channel and the historical siltation data;

[0027] The processing strategy information is obtained by combining the historical dredging scheme of the river channel and the dredging equipment used; when the historical dredging scheme of the river channel and the dredging equipment used do not meet the dredging demand corresponding to the current congestion degree information, the current processing strategy information is obtained by combining the historical dredging scheme of the existing river channel and the dredging equipment used.

[0028] In the second aspect, the application further provides a river channel congestion condition judgment method based on deep learning, which is suitable for the above-mentioned judgment system, and the method comprises the following steps:

[0029] S1, a position information positioning module collects the geographical position information of the current river channel in real time;

[0030] S2, a flow rate information collection module collects the flow rate information of the water flow in the current river channel in real time;

[0031] S3, a control module sends an image collection control signal to an image collection unit when the difference between the change value of the flow rate information corresponding to a certain geographical position information and the preset flow rate information of the geographical position information in the historical database is greater than a preset error value;

[0032] S4, the image collection unit collects the image information of the current river channel according to the image collection control signal, and returns the collected image information to the control module;

[0033] S5, a server receives the image information uploaded by the control module and the geographical data corresponding to the current geographical position information, inputs the image information and the geographical data into the trained neural network model, outputs the congestion degree information corresponding to the current river channel, determines the corresponding processing strategy information according to the congestion degree information corresponding to the current river channel, and issues the processing strategy information to the mobile terminal.

[0034] In a third aspect, the present application also provides a storage medium comprising a first storage medium and a second storage medium, the first storage medium comprising a first computer program, and the second storage medium comprising a second computer program, the first storage medium being located on the detection device, and the second storage medium being located on the server, the first computer program and the second computer program being executed by a processor to implement the method of the second aspect.

[0035] Unlike the prior art, the above technical solution cooperates the detection device with the server, uses the position information positioning module in the detection device to obtain the geographic coordinate information of the current detection device in real time, verifies whether the flow rate of the current river position is normal by means of the flow rate information collection module, uses the image collection unit to shoot the image of the river at the geographic position when there is abnormal change, and submits the image to the server for river congestion degree determination. The trained neural network model is stored on the server, which can automatically analyze the congestion degree corresponding to the current river geographic position in combination with the image information uploaded by the detection device and the geographic data corresponding to the current geographic position information, and output the processing strategy information to the detection device or the mobile terminal, so as to facilitate the subsequent dredging work. The detection device travels in the river, and uploads the relevant image in real time when the flow rate is abnormal, so as to realize real-time monitoring of the river congestion. Since the congestion degree information is determined by the trained model on the server in real time, the accuracy and timeliness of the river congestion determination can be improved, thereby improving the monitoring efficiency of the river congestion.

[0036] The above content is only a summary of the technical solution of the present application. In order to enable those skilled in the art to more clearly understand the technical solution of the present application, and then implement the content recorded in the specification and drawings, and in order to enable the above and other purposes, characteristics and advantages of the present application to be more easily understood, the following will be described in combination with the specific embodiments of the present application and the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0037] The drawings only serve to show the principles, implementation modes, applications, characteristics and effects of the specific embodiments of the present application and other related contents, and cannot be considered as limiting the present application.

[0038] In the drawings of the specification:

[0039] Figure 1 A schematic diagram of a river congestion condition judgment system based on deep learning according to an embodiment of the present application;

[0040] Figure 2 A schematic diagram of a river and its tributaries according to an embodiment of the present application;

[0041] Figure 3A schematic diagram of the distribution of congestion pixel in the image information of the river involved in a specific embodiment of the present application;

[0042] Figure 4 A schematic diagram of the image involved in the present application Figure 3 after binarization processing;

[0043] Figure 5 A schematic diagram of the image involved in the present application Figure 4 after inflation processing;

[0044] Figure 6 A schematic diagram of the distribution of congestion pixel after the image involved in the present application Figure 5 erodes processing;

[0045] Figure 7 A flow chart of a river congestion condition judgment method based on deep learning involved in a specific embodiment of the present application;

[0046] Figure 8 A module schematic diagram of the storage medium involved in a specific embodiment of the present application;

[0047] The reference signs involved in the above-mentioned each figure are explained as follows:

[0048] 10, detection device;

[0049] 101, position information positioning module;

[0050] 102, flow rate information acquisition module;

[0051] 103, control module;

[0052] 104, image acquisition unit;

[0053] 105, steering driving unit;

[0054] 106, first storage medium;

[0055] 20, server;

[0056] 201, second storage medium

[0057] 30, mobile terminal. DETAILED DESCRIPTION

[0058] In order to explain the possible application scenarios, technical principles, specific schemes that can be implemented, purposes and effects that can be achieved of the present application in detail, the following will be explained in detail in combination with the specific embodiments listed and the accompanying drawings. The embodiments recorded in this paper are only used to more clearly explain the technical scheme of the present application, therefore only as an example, and cannot limit the protection scope of the present application.

[0059] The term "embodiment" is mentioned herein means that the specific features, structures, or characteristics described in connection with the embodiment can be included in at least one embodiment of the present application. The term "embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment, nor does it particularly limit the independence or association between other embodiments. In principle, in the present application, as long as there is no technical contradiction or conflict, each technical feature mentioned in each embodiment can be combined in any way to form a corresponding implementable technical solution.

[0060] Unless otherwise defined, the meanings of the technical terms used herein are the same as those commonly understood by those skilled in the art to which the present application belongs; the use of related terms herein is only for the purpose of describing specific embodiments, and is not intended to limit the present application.

[0061] In the description of the present application, the phrase "and / or" is a description of the logical relationship between the objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases: A exists, B exists, and A and B exist at the same time. In addition, the character " / " herein generally represents that the associated objects before and after are a "or" logical relationship.

[0062] In the present application, the terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual quantity, primary and secondary or order relationship between the entities or operations.

[0063] Without more limitations, in the present application, the phrases "include", "contain", "have" or other similar open expressions used in the statement are intended to cover non-exclusive inclusion, and these expressions do not exclude the presence of other elements in the process, method or product including the described elements, so that the process, method or product including a series of elements can not only include those limited elements, but also include other elements not explicitly listed, or also include the elements inherent to such process, method or product.

[0064] In the present application, "greater than", "less than", "exceed" and the like are understood as not including the number; "above", "below", "within" and the like are understood as including the number. In addition, in the description of the embodiments of the present application, the meaning of "multiple" is more than two (including two), and similar expressions related to "multiple" are also understood in this way, for example, "multiple groups", "multiple times" and the like, unless otherwise explicitly limited.

[0065] In the description of the embodiments of the present application, the spatially relative terms used, such as "central", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "vertical", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. The indicated orientation or positional relationship is based on the orientation or positional relationship shown in the specific embodiment or the drawing, and is only for the convenience of describing the specific embodiment of the present application or for the reader to understand, and does not indicate or imply that the indicated device or component must have a particular position, a particular orientation, or be constructed or operated in a particular orientation, and therefore cannot be understood as a limitation on the embodiments of the present application.

[0066] Unless otherwise expressly specified or limited, the terms "mount", "connect", "connection", "fixed", "set", and the like used in the description of the embodiments of the present application should be interpreted broadly. For example, the "connection" can be fixed connection, or detachable connection, or integral setting; it can be mechanical connection, or electrical connection, or communication connection; it can be direct connection, or indirect connection through intermediate medium; it can be the internal communication of two elements or the interaction relationship between two elements. For those skilled in the art to which the present application belongs, the specific meaning of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0067] Please refer to Figure 1 The embodiment provides a river congestion condition judgment system based on deep learning, which comprises a detection device 10 and a server 20. The detection device 10 comprises a position information positioning module 101, a flow rate information acquisition module 102, a control module 103 and an image acquisition unit 104. The position information positioning module is used to acquire the geographical position information of the current river in real time. The flow rate information acquisition module is used to acquire the flow rate information of the water flow in the current river in real time. The control module is used to send an image acquisition control signal to the image acquisition unit when the difference between the change value of the flow rate information corresponding to a certain geographical position information and the preset flow rate information of the geographical position information in the historical database is greater than the preset error value. The image acquisition unit is used to acquire the image information of the current river according to the image acquisition control signal, and returns the acquired image information to the control module. The server is used to receive the image information uploaded by the control module and the geographical data corresponding to the current geographical position information, input the image information and the geographical data into the trained neural network model, output the congestion degree information corresponding to the current river, and determine the corresponding processing strategy information according to the congestion degree information corresponding to the current river, and issue the processing strategy information to the detection device 10 or the mobile terminal 30.

[0068] In the embodiment, the detection device further comprises a power module for supplying power and a walking mechanism comprising a driving unit and a moving assembly, the detection device body is arranged on the moving assembly, and the moving assembly is driven to move by the driving unit, thereby driving the detection device body to walk along the river direction in the riverbed below the water surface. The moving assembly can be a moving mechanism comprising wheels or sprockets, and the driving unit can be a motor. The control module is also used to receive the measured river channel geographic data information and the measured river length, so that the detection device searches for the congestion degree in the river channel according to the predetermined route. Specifically, during detection, the detection device can be placed in the river, and the detection device can move forward along the water flow on the riverbed to realize dredging detection of the river channel.

[0069] The position information positioning module can be a GPS positioning system or a Beidou positioning system. Through the position information positioning module, the geographic position information of the current river channel can be collected in real time, which specifically represents the geographic position coordinates of the current detection device in the river channel, and is input to the control module. The predetermined route information of the river channel to be detected received by the control module is compared, and when the current geographic position coordinates of the detection device and the predetermined route have a large deviation, the control module timely sends a warning information to the mobile terminal.

[0070] The flow rate information collection module can be a flowmeter, and the number of flow rate information collection modules can be adjusted according to the length of the current river channel. When the flow rate information collection module is multiple, multiple flow rate information collection modules can be arranged around the four sides of the detection device body to realize flow rate information collection in multiple directions. Then, the flow rate information provided by the multiple flow rate information collection modules is collected, the average flow rate of the river channel at the position of the detection device is calculated, and the average flow rate is sent to the control module. The control module corrects the current average flow rate, and then judges whether the current flow rate changes abnormally.

[0071] It should be noted that, since the river flow rate is affected by hydrology, climate, slope and other factors, the normal change range of the flow rate exhibited by different length river channels at different geographic coordinate positions is different, so after the detection device receives the input river length and route, the preset flow rate curve corresponding to the river route length (recording the corresponding relationship between each geographic coordinate point in the river route length and the normal flow rate range) stored in the historical database can be obtained. The control module compares the received average flow rate with the normal flow rate range corresponding to the geographic coordinate position in the preset flow rate curve, and then judges whether the currently collected flow rate information is abnormal.

[0072] For example, the control module can obtain preset flow rate information of the geographic location information in the historical database of the current river channel. When the geographic location information has no preset flow rate information, the flow rate value measured at the previous geographic location information in the reference database and the geographic features at the geographic location are weighted to obtain the flow rate value. When there is no measurement information of the previous measurement point in the reference database, the two nearest preset flow rate information of the geographic location information and whether there is a rapid flow at the geographic location are weighted to obtain the flow rate value. The change value of the flow rate information measured at the current geographic location information is calculated by difference with the preset flow rate information. When the change value is greater than the preset error value, an image acquisition control signal is sent to the image acquisition unit.

[0073] The preset error value can be obtained by comprehensively calculating the water level change data and the water flow change amount of the measured section of the current river channel within a certain time interval. For example, the water level change data and the water flow change amount of the current river channel in the last month are collected, and the water flow difference change interval of the current river channel in the same geographic feature (geographic feature represents gentle slope, steep slope, curve and straight road) is obtained, that is, the preset error value. When the change value is greater than the preset error value, it indicates that the current detected geographic location information has flow rate abnormality.

[0074] In the embodiment, the detection device further comprises a silt detection module, and the silt detection module comprises a sonar generator. The sonar generator can emit sound waves of different frequencies, and the depth of silt corresponding to the geographic coordinates of the current detection device can be calculated by the return time of the sound waves. At the same time, the sonar generator can also be used to measure the data of the current river channel, for example, the width of the river channel. When there is a tributary river channel, the sonar can be used to measure the distance between the detection device and the entrance of the tributary river channel, so as to adjust the focal length and shooting angle of the image acquisition module through the control unit.

[0075] The image acquisition unit can be a camera or an electronic device with a camera. After receiving the image acquisition signal sent by the control module, the image acquisition unit shoots the image of the current river channel and transmits the acquired image information to the control module. The processing strategy information is a specific dredging scheme, and the congestion degree information contains the siltation data of the geographic location of the current detection device. According to the congestion degree information, the corresponding processing strategy information is specified and sent to the detection device or the mobile terminal.

[0076] In the embodiment, sending the processing strategy information to the detection device includes the following two cases:

[0077] The first case is that in some preferred embodiments, the detection device is equipped with a simple dredging module, for example, a scheme of rotating and crushing silt with a fan blade, and the fan blade for dredging is integrated on the detection device. When it is determined that the current river channel has a small congestion degree, the corresponding processing strategy information is to dredge by the detection device, and the processing strategy information is sent to the detection device, and the dredging module on the detection device is used for dredging.

[0078] The second case is that when the current congestion degree is small, dredging action can not be needed, and the detection device is sent to the control module to control the detection device to bypass the current silt and continue to detect, or when the current congestion degree is too large and the detection device cannot continue to advance to realize the detection work, the control module controls the detection device to stop advancing and waits for the operator to take out the detection device.

[0079] In the embodiment, the mobile terminal is preferably an operation terminal monitored by the operator, for example, a center console, a handheld terminal or the like. By sending the congestion degree information and the processing strategy information to the mobile terminal, the operator can record the congestion degree of the current test river channel in real time, so as to plan and process the dredging scheme of the river channel.

[0080] In some embodiments, the detection device can further include a jacking module including an air bag and a compressed gas tank. The compressed gas is stored in the compressed gas tank. When a detection end signal of the control module is received, the compressed gas is released into the air bag, so that the detection device can quickly rise from the riverbed to the river surface, and the operator can perform the salvage action.

[0081] The above technical scheme cooperates the detection device with the server, uses the position information positioning module in the detection device to obtain the geographic coordinate information of the current detection device in real time, verifies whether the flow rate of the current river channel position is normal by means of the flow rate information collection module, uses the image collection unit to shoot the image of the river channel at the geographic position when there is an abnormal change, and submits the image to the server for river channel congestion degree determination. The server stores a trained neural network model. The neural network model can automatically analyze the congestion degree corresponding to the current river channel geographic position in combination with the image information uploaded by the detection device and the geographic data corresponding to the current geographic position information, and output the processing strategy information to the detection device or the mobile terminal, so as to facilitate the subsequent dredging processing work. The detection device travels in the river channel, and uploads the related image in real time when the flow rate is abnormal, so as to realize real-time monitoring of the river channel congestion. Since the congestion degree information is determined by the trained model on the server in real time, the accuracy and timeliness of the river channel congestion determination can be improved, so that the monitoring efficiency of the river channel congestion is improved.

[0082] In some embodiments, the control module is further configured to, when determining that the change value of the flow rate information corresponding to the geographic location information is greater than the preset error value from the preset flow rate information of the geographic location information in the historical database, acquire the river tributary orientation information corresponding to the current geographic location information, and generate an orientation adjustment instruction according to the river tributary orientation information; the detection device further comprises a steering driving unit 105, which is configured to adjust the shooting angle of the image acquisition unit according to the orientation adjustment instruction; the image acquisition unit is further configured to acquire image information before and after the shooting angle is adjusted, and after obtaining the first image information corresponding to the current river and the second image information corresponding to the tributary river of the current river, the first image information and the second image information are transmitted back to the control module.

[0083] The river tributary orientation information corresponding to the current geographic location information is acquired, specifically including: sending the current geographic location information to the server, and the server calling the river data corresponding to the geographic location information stored in the database to determine whether there is a river tributary corresponding to the current geographic coordinate, if there is, determining the number of river tributaries, and sending the determined number of river tributaries and the orientation data of each tributary relative to the current main river to the control module. When determining that the number of river tributaries is greater than or equal to 1, the control module acquires the orientation information of the river tributary, and generates an orientation adjustment instruction according to the river tributary orientation information. For example, the orientation adjustment instruction can be determined according to the following manner: determining the direction of the radial section of the current river according to the flow direction of the current river, or determining the direction of the radial section of the tributary river according to the coordinate position of the tributary river relative to the current river, and then generating an orientation adjustment instruction according to the direction of the radial section of the tributary river. Then the shooting direction of the image acquisition unit can be adjusted to the direction of the radial section of the tributary river for shooting. Figure 2 In the middle, Z2 represents the current river, and Z1 represents the river tributary.

[0084] The image acquisition unit is arranged on the steering driving unit, and the steering driving unit is configured to adjust the shooting angle of the image acquisition unit according to the orientation adjustment instruction, so that the image acquisition unit can collect the radial section of the photographed river to the greatest extent within the shooting range. When the number of river tributaries is greater than or equal to 1, the image collected by the detection main river is recorded as the first image information, and the image collected by the tributary river corresponding to the main river is recorded as the second image information. If there are other tributary rivers, they can be sequentially numbered and recorded as the third image information, the fourth image information, etc.

[0085] Since the flow rate at the intersection will inevitably change when there is a tributary in the river, it can suddenly increase (applicable to the case of river tributary confluence), or it can suddenly decrease (applicable to the case of river tributary separation), by determining the number of river tributaries, the collected image data can be more complete and more representative, so that the result output by the neural network model based on the collected multiple images can more accurately reflect the congestion degree of the river at the current geographic coordinate position.

[0086] In some embodiments, when the control module obtains that the current river does not have corresponding river tributary information, the image acquisition unit is configured to acquire first image information corresponding to the current river and return to the control module; the control module is configured to send the first image information to the server, and receive congestion degree information and processing strategy information corresponding to the first image information fed back by the server.

[0087] When it is determined that the current river tributary information does not exist, that is, the number of rivers determined is 1, it indicates that the abnormal change of water flow rate occurs at present, but the historical database has not queried that the river tributary exists at the geographic position information. At this time, the image acquisition unit only acquires the first image information corresponding to the current river and transmits it to the control module, and the control module sends it to the neural network of the server for image judgment.

[0088] In some preferred embodiments, when there is a shooting angle limitation, multiple first image information corresponding to the current river at different angles can be shot and transmitted to the server, and the neural network model will splice a complete image according to the multiple first image information at different angles, and then judge the congestion degree. For example, the image acquisition unit shoots the first image information of the current main river, in addition to shooting the image information of the river in front, it can also shoot the image in front of the detection device by tilting 30°, 45°, 60° respectively, and add the same identification information to this group of shooting images to indicate that the group of images belong to the input data for judging the congestion degree of the same geographic coordinate position. The neural network model will first splice a complete image after receiving this group of data, and then analyze the spliced complete image to judge the congestion degree information.

[0089] By pre-judging the number of river tributaries, when the number of river tributaries is 0, only the image of the current river is collected, which can effectively simplify the data range collected by the detection device, thereby improving the judgment efficiency and accuracy of the congestion degree.

[0090] In some embodiments, if the server determines that the congestion degree information of the current river channel is not congested, the control module sends a prompt information and reports to the server or the mobile terminal. When the congestion degree information of the current river channel is not congested, it means that there is no congestion phenomenon in the current river channel, but the flow rate information is in an abnormal state. At this time, the control module will send a prompt information and report to the server or the mobile terminal, so as to facilitate the operator to verify the abnormal situation. The prompt information contains the geographic coordinate information of the current image acquisition point, the abnormal flow rate information and the preset flow rate information (i.e. normal flow rate range information), the topographic features and basic parameters corresponding to the current geographic coordinate, the basic parameters including the water temperature and the rainfall in the past two days, etc., so as to facilitate the operator to analyze and dispose the abnormal situation.

[0091] In some embodiments, the neural network model is trained according to the following manner:

[0092] A neural network model is constructed, the input parameters of the neural network model are sample images and geographic data in a sample database, and the output results are congestion degree information and corresponding processing strategy information of the river channel; the sample images include river channel image information without congestion and multiple river channel image information with different congestion degrees; the geographic data includes any one or more of river length data, rainfall data on both banks of the river channel, and river channel geographic shape features; the sample images and the geographic data corresponding to the sample images in the sample database are repeatedly trained to improve the accuracy of the neural network model, and a trained neural network model is obtained.

[0093] The sample images include river channel image information without congestion and multiple river channel image information with different congestion degrees. The river channel image information without congestion can be the river channel image information taken before the current river channel, or the river channel image information taken by other river channels. The multiple river channel image information with different congestion degrees should correspond to the river channel image information without congestion one by one. For example, the sample images include multiple geographic coordinate points of A, B, C……N river channels without congestion. The river channel image information with different congestion degrees should also be the river channel congestion image at multiple geographic coordinate points in A, B, C……N river channels.

[0094] The geographic data includes any one or more of river length data, rainfall data on both banks of the river, and river geographic shape features. The river congestion is greatly affected by natural factors, and it is difficult to check the river congestion by technical means in the prior art. Therefore, it is necessary to reasonably utilize the observed geographic data to correct the congestion information of the current river. The river length data includes the measured river length of the river, for example, an image of a certain river from a coordinate to a coordinate can be collected to determine the congestion degree. The rainfall data on both banks of the river is preferably the rainfall data of the past week, and the rainfall data in the past five years within the time interval is obtained for comparison to determine the change of the rainfall data. The river geographic shape features include the degree of winding, slope, and the situation of converging tributaries, river width and depth, and other geographic information within the river length interval of the current detected river.

[0095] Through repeated training of multiple rivers and sample data required by the rivers, the neural network model can continuously correct the judgment of whether the current river is congested and the congestion degree information in combination with the geographic data corresponding to the sampling points of the image data, so that the judgment of the congestion degree information of the river is more accurate.

[0096] In some embodiments, the sample images in the sample database and the geographic data corresponding to the sample images are read to repeatedly train the neural network model, improve the accuracy of the neural network model, and obtain a trained neural network model, including:

[0097] A set of training sample images is taken, and the geographic data corresponding to the set of sample images, the sample image features, and the congestion degree information are input into the neural network.

[0098] The neural network uses a gradient algorithm to perform matrix operations on the weight of the geographic data corresponding to the sample image and the weight of the sample image features, and sets a basic value for the learning rate and the offset of the learning rate. After the operation is completed, a mathematical model about the input sample image and the corresponding congestion object type and congestion degree information is obtained. The weight of the geographic data is calculated according to the rainfall data and the river geographic shape features of the geographic location. The weight of the sample image is determined according to the proportion of the pixel points of the congestion object position in the sample image. The congestion object type is determined according to the position distribution of the determined congestion object on the sample image.

[0099] A set of training data is replaced and the operation is performed again. The model is corrected by the gradient algorithm and becomes closer to the real model reflected by the data each time the operation is performed.

[0100] After all the training sample data sets are rotated, it is considered that one round of training is completed. After at least one round of training is completed, the model is evaluated.

[0101] The training sample image of the test is used to evaluate the model, the sample image is input, the output congestion degree information y1 from the model is obtained, and the actual value y of the congestion degree information corresponding to the sample image is compared. If the theoretical value y1 is equal to the actual value y, it is recorded as a prediction success, that is, a correct identification, and the model is evaluated in terms of accuracy:

[0102]

[0103] If the accuracy a of the mathematical model is greater than or equal to 80%, the model is considered suitable, and its code is saved, otherwise, the model is optimized by increasing the sample data, increasing the number of training rounds, fine-tuning the learning rate and / or offset algorithm parameters;

[0104] The finally optimized model is deployed on the server and combined with the front-end interface function. The front-end interface is used to receive the river image uploaded by the control module.

[0105] The congestion type includes silt, large metal congestion (abandoned vehicles, equipment, etc.), small metal congestion (metal garbage such as pop cans), concrete debris, and large stones. The congestion degree information output by the model can be in the form of congestion degree and congestion type. The congestion degree can be set to have multiple levels, and each level can be identified by different colors from high to low. For example, the congestion degree can be set to three levels: low congestion degree, medium congestion degree, and high congestion degree. After combining the identified congestion type (which can be determined according to the size of the congestion accumulation and its relative position in the river), the output congestion degree information can be: silt accumulation + low congestion degree, large metal accumulation + medium congestion degree, etc.

[0106] In the present embodiment, the congestion type is determined according to the position distribution of the congestion on the sample image. For example, when the congestion floats in the river, it is determined that the congestion does not cause congestion to the current river, and the congestion is determined as non-congestion material. When the congestion is deposited at the bottom of the riverbed, the specific type of the congestion is obtained by combining the detection of the congestion by the sonar. When the congestion is scattered on the bottom of the current riverbed, the options of large metal congestion and large stones are excluded, and the type of the congestion is determined again to improve the accuracy of the congestion type determination.

[0107] Please refer to Figures 3 to 6 In some embodiments, the position of a certain congestion in the sample image is determined as follows:

[0108] The gray value of each pixel point on the sample image is determined in sequence. When the gray value is greater than a preset value, the pixel point is set to 0, and when the gray value is less than the preset value, the pixel point is set to 1.

[0109] The pixel points with value 1 are sequentially subjected to expansion and erosion treatment to determine the connected domains with value 1 in the current sample image after the expansion and erosion treatment.

[0110] The connected domains with value 1 in the current sample image after the expansion and erosion treatment are determined as the positions of the congestion objects.

[0111] The expansion treatment includes: setting the pixel points in the 8-neighborhood of each pixel point with value 1 to value 1.

[0112] The erosion treatment includes: judging whether there is a pixel point with value 0 in the 8-neighborhood of each pixel point after the expansion treatment, and if so, the pixel point is removed; otherwise, the pixel point is retained.

[0113] The preset value is a preset value obtained by collecting and calculating the congestion images in the past. Generally, due to the existence of the congestion objects, the pixel points corresponding to the water flow have higher brightness values than the congestion objects in the captured river image. Therefore, the pixel points with a gray value exceeding a predetermined value (in the image, the darker the pixel point, the closer to 0, and the whiter the pixel point, the closer to 255) can be temporarily determined as the pixel points corresponding to the congestion objects. By classifying the pixel points with value 0 and value 1 according to the gray values, the data in the image can be preliminarily screened, and the gray image that may contain the congestion objects is retained. The retained image needs to be further processed.

[0114] Through the erosion treatment, it can be judged whether there is a pixel point with value 0 in the neighborhood of the pixel point with value 1 after the expansion treatment. If so, the pixel point with value 1 is removed. If there is no pixel point with value 0, i.e., there is a newly added pixel point with value 1 between the two adjacent pixel points with value 1, the pixel point with value 1 is retained.

[0115] Through the expansion and erosion treatment, the pixel points corresponding to the congestion objects that are relatively dense but not intersected can be reconnected as a whole, i.e., a connected domain, and the pixel points that are relatively scattered and not intersected can be removed to avoid interference with the subsequent congestion degree judgment. The connected domains with value 1 in the ranking indicate the larger whole. Through the expansion and erosion treatment, the larger pixel point whole and the relatively scattered pixel point whole can be obtained. At this time, the pixel point whole (i.e., the connected domain) with the largest area is selected to determine the position of the congestion object, and then the distribution of the congestion object in the river at the current geographic coordinate is used to judge the congestion degree of the river at the current geographic coordinate.

[0116] For example Figure 3 The river image information in the A region, the B region and the C region involves the positions of the congestion objects. Through the preliminary gray value size screening, the pixel points in the A region, the B region and the C region can be set to 1, and the other parts can be set to 0, as shown in Figure 4The image is then dilated, specifically, all edge pixel points in the A, B and C regions are obtained, and each edge pixel point is set to 1, so that the positions of the relatively close congestion objects can be connected together, and the type of the congestion object and the congestion degree can be further screened and judged. The dilated image is as shown in FIG. 4C. Figure 5 The dilated image is then eroded, specifically, each pixel in the A, B and C regions is taken as the center, and it is judged whether the 8-neighborhood pixel points of the pixel are set to 0. If yes, the pixel is a noise point and is removed. Otherwise (i.e., all the 8-neighborhood pixel points are set to 1), the pixel is reserved. The eroded image is as shown in FIG. 4D. Figure 6

[0117] In some embodiments, the input parameters of the neural network further include historical siltation data of the current river channel, the congestion degree information is calculated according to the geographical data of the current river channel, the collected river channel image information and the historical siltation data, the processing strategy information is obtained by combining the historical dredging scheme of the river channel and the used dredging equipment, and when the historical dredging scheme of the river channel and the used dredging equipment do not meet the dredging demand corresponding to the current congestion degree information, the current processing strategy information is obtained by combining the historical dredging scheme of the existing river channel and the used dredging equipment.

[0118] The historical siltation data is an important reference index. For example, in a curved river channel, the siltation degree at the bend of the river channel is accumulated year by year, so the siltation possibility at the bend is larger. For another example, at the confluence point of a tributary, there is a characteristic that the water flow continuously erodes the river bed, so the siltation possibility at the confluence point is smaller. By adding the historical siltation data as a parameter, the learning of the neural network can be more accurate, and the judgment of the siltation state of different river channels can be more targeted.

[0119] ​The processing strategy information is preferably obtained by combining the historical dredging scheme of the detected river channel and the dredging equipment used, so that the operator can obtain the dredging equipment within the shortest distance, gather the dredging unit with the dredging experience of the river channel within a shorter time, dredge the detected river channel in time, and improve the dredging efficiency of the detected river channel. When the historical dredging scheme of the river channel and the dredging equipment used do not meet the dredging demand corresponding to the current congestion degree information, the current processing strategy information is obtained by combining the historical dredging scheme of the existing river channel and the dredging equipment used. Optionally, the dredging scheme and the dredging equipment of the river channel near the detected river channel are given priority, so that the operator can obtain the dredging equipment within the shortest distance, gather the dredging unit with the dredging experience of the river channel within a shorter time, dredge the detected river channel in time, and improve the dredging efficiency of the detected river channel.

[0120] As shown in Figure 7 The embodiment also provides a river channel congestion condition judgment method based on deep learning, which is suitable for the judgment system and includes the following steps:

[0121] S1, a position information positioning module collects real-time geographic position information of a current river channel;

[0122] S2, a flow rate information collection module collects real-time flow rate information of water flow in the current river channel;

[0123] S3, a control module sends an image collection control signal to an image collection unit when the difference between the change value of the flow rate information corresponding to a certain geographic position information and the preset flow rate information of the geographic position information in the historical database is greater than a preset error value;

[0124] S4, the image collection unit collects image information of the current river channel according to the image collection control signal and returns the collected image information to the control module;

[0125] S5, a server receives the image information uploaded by the control module and the geographic data corresponding to the current geographic position information, inputs the image information and the geographic data into the trained neural network model, outputs the congestion degree information corresponding to the current river channel, determines the corresponding processing strategy information according to the congestion degree information corresponding to the current river channel, and sends the processing strategy information to a mobile terminal.

[0126] The geographic position information includes geographic position information of the river channel where the detection device is located and specific geographic coordinate position information of the detection device in the river channel, and the flow velocity information of the water flow can be achieved by arranging a plurality of flow velocity information collection modules, so as to improve the accuracy of the data of the flow velocity information of the water flow. When the difference between the change value of the flow velocity information corresponding to a certain geographic position information and the preset flow velocity information of the geographic position information in the historical database is greater than a preset error value, it indicates that the current water flow velocity changes abnormally, and there may be congestion, and then an image acquisition control signal needs to be sent to the image acquisition unit for further data acquisition.

[0127] The collected image data is input into the service end neural network model for judgment, and the congestion degree information and the processing strategy information corresponding to the current river channel are output, and the processing strategy information is sent to the mobile end. The processing strategy information here represents the dredging scheme and the required dredging equipment determined according to the current congestion degree information, which is sent to the mobile end, so as to facilitate the operator to timely dredge the detection point in the river channel. Through the learning and judgment of the neural network model, with the aid of image acquisition and flow velocity judgment, the detection and judgment of the river channel siltation degree can be realized, and the corresponding dredging scheme and dredging equipment are output, which reduces the labor cost in the river channel dredging detection process, and improves the accuracy and timeliness of the river channel siltation detection.

[0128] As shown in Figure 8 The present embodiment also provides a storage medium, including a first storage medium 106 and a second storage medium 201, the first storage medium includes a first computer program, and the second storage includes a second computer program, the first storage medium 106 is located on the detection device 10, and the second storage medium 201 is located on the service end 20, and the first computer program and the second computer program are executed by the processor to realize the method described above.

[0129] The storage medium / memory includes but is not limited to RAM, ROM, magnetic disk, magnetic tape, optical disk, flash memory, U disk, mobile hard disk, memory card, memory stick, network server storage, network cloud storage, etc. The processor includes but is not limited to CPU (central processing unit), GPU (graphics processor), MCU (microprocessor), etc.

[0130] Finally, it should be noted that although the above-mentioned embodiments have been described in the specification and drawings of the present application, but it cannot limit the patent protection scope of the present application. Any equivalent structure or equivalent process substitution or modification based on the essential concept of the present application, using the content described in the specification and drawings of the present application, and directly or indirectly implementing the technical solutions of the above embodiments in other related technical fields, etc., are all included in the patent protection scope of the present application.

Claims

1. A river congestion condition judgment system based on deep learning, characterized by, The system comprises a detection device and a server, the detection device comprises: a position information positioning module for real-time collection of geographical position information of a current river channel; a flow rate information collection module for real-time collection of flow rate information of water flow in the current river channel; a control module for sending an image collection control signal to an image collection unit when determining that a change value of the flow rate information corresponding to a certain geographical position information is greater than a preset error value from a preset flow rate information of the geographical position information in a historical database; an image collection unit for collecting image information of the current river channel according to the image collection control signal and returning the collected image information to the control module; the server is configured to receive the image information uploaded by the control module and geographical data corresponding to the current geographical position information, input the image information and the geographical data into a trained neural network model, output congestion degree information corresponding to the current river channel, and determine corresponding processing strategy information according to the congestion degree information corresponding to the current river channel, and send the processing strategy information to the detection device or a mobile terminal; reading sample images and geographical data corresponding to the sample images in a sample database to repeatedly train the neural network model, improve the accuracy of the neural network model, and obtain a trained neural network model, comprising: taking a group of training sample images, inputting geographical data, sample image features and congestion degree information corresponding to the group of sample images into the neural network; the neural network uses a gradient algorithm to perform matrix operation on the weight of the geographical data corresponding to the sample images and the weight of the sample image features, sets a basic value for the learning rate and the offset of the learning rate, and obtains a mathematical model about the input sample images, the congestion object type and the congestion degree information after the operation; the weight of the geographical data is calculated according to the rainfall data and the geographical shape features of the river channel; the weight of the sample image is determined according to the pixel point proportion of the congestion object position in the sample image; the congestion object type is determined according to the position distribution of the determined congestion object on the sample image; replace a group of training data and perform operation again, the model is corrected and becomes closer to the real model reflected by the data every time the operation is performed; after all the training sample data groups are replaced, it is considered that one round of training is completed, and the model is evaluated after at least one round of training is completed; reading the test training sample images to evaluate the model, inputting the sample images, obtaining the output congestion degree information y1 from the model, and comparing the actual value y of the congestion degree information corresponding to the sample images, if the theoretical value y1 is equal to the actual value y, it is recorded as a prediction success, that is, the recognition is correct, and the model is evaluated in terms of accuracy: if the accuracy a of the mathematical model is greater than or equal to 80%, the model is considered suitable, and the code is saved, otherwise, the model is optimized by increasing sample data, increasing the number of training rounds, fine-tuning the learning rate and / or offset algorithm parameters; deploy the finally optimized model on the server and combine it with the front-end interface function, and the front-end interface is configured to receive the river channel image uploaded by the control module.

2. The river congestion condition judging system based on deep learning according to claim 1, characterized by, The control module is further configured to acquire river branch direction information corresponding to the current geographic position information and generate a direction adjustment instruction according to the river branch direction information when it is determined that a change value of the flow rate information corresponding to the geographic position information is greater than a preset error value from the preset flow rate information of the geographic position information in the historical database; The detection device further comprises: A steering driving unit configured to adjust a shooting angle of the image acquisition unit according to the direction adjustment instruction; The image acquisition unit is further configured to acquire image information before and after the shooting angle is adjusted, and to obtain first image information corresponding to the current river and second image information corresponding to a branch river of the current river, and then to return the first image information and the second image information to the control module.

3. The river congestion condition judging system based on deep learning according to claim 2, characterized by, When the control module acquires that there is no corresponding river branch information of the current river, the image acquisition unit is configured to acquire first image information corresponding to the current river and return the first image information to the control module; and the control module is configured to send the first image information to a server and receive congestion degree information and processing strategy information corresponding to the first image information fed back by the server.

4. The river congestion condition judging system based on deep learning according to claim 3, characterized by, When the congestion degree information of the current river is not congested, the control module sends a prompt information and reports to the server or a mobile terminal. 5.The river congestion condition judging system based on deep learning according to claim 1, wherein The neural network model is trained in the following manner: A neural network model is constructed, input parameters of the neural network model are sample images and geographic data in a sample database, and output results are congestion degree information and corresponding processing strategy information of a river; the sample images include river image information without congestion and multiple river image information with different congestion degrees; the geographic data includes any one or more of river length data, rainfall data on both banks of the river, and river geographic shape features; The sample images and geographic data corresponding to the sample images in the sample database are read to repeatedly train the neural network model, improve the accuracy of the neural network model, and obtain a trained neural network model. 6.The river congestion condition judging system based on deep learning according to claim 1, wherein The position of the congestion object in the sample image is determined in the following manner: The gray values of each pixel point on the sample image are sequentially judged, the pixel point is set to 0 when the gray value is greater than a preset value, and the pixel point is set to 1 when the gray value is less than the preset value; All pixel points set to 1 are sequentially subjected to expansion processing and corrosion processing to determine a plurality of connected domains of the pixel points set to 1 in a size ranking in the current sample image after the expansion and corrosion processing; The plurality of connected domains of the pixel points set to 1 in the size ranking are determined as the position of the congestion object; The expansion processing includes setting all pixel points in an 8-neighborhood of each pixel point set to 1 to 1; The corrosion processing includes judging whether there is a pixel point set to 0 in the 8-neighborhood of each pixel point after the expansion processing, and if there is, the pixel point is removed; otherwise, the pixel point is retained. 7.The river congestion condition judging system based on deep learning according to claim 5, wherein, The input parameters of the neural network further include historical siltation data of the current river, and the congestion degree information is obtained by weighted calculation according to the geographic data of the current river, the acquired river image information, and the historical siltation data; The processing strategy information is obtained by combining the historical dredging scheme of the river and the dredging equipment used; When the historical dredging scheme of the river and the dredging equipment used do not meet the dredging demand corresponding to the current congestion degree information, the current processing strategy information is obtained by combining the historical dredging scheme of the existing river and the dredging equipment used. 8.A method for judging a congestion condition of a river channel based on deep learning, characterized by The method is suitable for the judgment system of claims 1-7, and the method comprises the following steps: S1, a position information positioning module collects real-time geographic position information of the current river; S2, a flow rate information collection module collects real-time flow rate information of the water flow in the current river; S3, a control module sends an image acquisition control signal to an image acquisition unit when the difference between the change value of the flow rate information corresponding to a certain geographic position information and the preset flow rate information of the geographic position information in the historical database is greater than the preset error value; S4, the image acquisition unit collects image information of the current river according to the image acquisition control signal and returns the collected image information to the control module; S5, a server receives the image information uploaded by the control module and the geographic data corresponding to the current geographic position information, inputs the image information and the geographic data into the trained neural network model, outputs the congestion degree information corresponding to the current river, and determines the corresponding processing strategy information according to the congestion degree information corresponding to the current river, and sends the processing strategy information to the mobile terminal.

9. A storage medium, characterized by The first storage medium includes a first computer program, and the second storage medium includes a second computer program, the first storage medium is located on the detection device, the second storage medium is located on the server, and the first computer program and the second computer program are executed by the processor to realize the method of claim 8.

Images