River and lake disorder detection method, system and device based on video analysis and medium

Abstract

The invention provides a river-lake four-disorder detection method, system and device based on video analysis and a medium, and the method comprises the steps: employing an unmanned plane to carry a video camera, and obtaining video images, shot by the video camera in real time, of a river, a lake and along a line through a wireless image transmission technology; performing target identification on the video image by using a multi-target tracking model, and identifying target objects of rivers and lakes and along the rivers and lakes; converting the video image corresponding to the target object into an image sequence, inputting a video classification model to classify the target object, classifying the target object into illegal buildings, illegal sand dredgers, illegal construction engineering vehicles and illegal aquaculture net cages, and generating a classification result; and generating violation records according to a classification result, extracting related video clips and pictures, and displaying the video clips and pictures. According to the invention, violation and illegal behaviors of rivers and lakes and along lines can be automatically identified and captured, and real-time and accurate monitoring data is provided for managers, so that the efficiency and accuracy of river and lake supervision are greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of video analysis technology, and more specifically to a method, system, device and medium for detecting river and lake chaos based on video analysis. Background Art

[0002] At present, with the rapid development of economy and the accelerated advancement of urbanization, the protection and management of rivers and lakes, as the important cornerstones of the earth's precious natural resources and ecological environment, are becoming increasingly arduous. Rivers and lakes not only carry multiple ecological functions such as climate regulation, water conservation, flood control and drainage, but are also an indispensable source of water resources for human production and life. However, in the face of growing demand for development and utilization, the management problems of river and lake coastal areas have become increasingly prominent, especially the frequent occurrence of the "four chaos" phenomenon, which has become a stubborn disease restricting the healthy development of rivers and lakes.

[0003] The "four chaos" phenomenon, namely the construction of illegal buildings, the occupation of river and lake resources, the random extraction of river and lake sediments, and the random dumping of garbage, are like a cancer that erodes the ecological environment of rivers and lakes. Illegal buildings not only destroy the natural landscape of rivers and lakes, but may also cause the river channel to narrow and the water flow to be blocked, affecting flood control safety; the random occupation of resources leads to the irrational use of precious river and lake resources, or even their depletion; the random extraction of sediments aggravates the damage to the bottom of rivers and lakes, affecting water quality and biodiversity; and the random dumping of garbage directly pollutes the water body, threatening the safety of people's drinking water.

[0004] Faced with this grim situation, traditional manual inspection methods are stretched to the limit. Limited by manpower, material resources and time costs, manual inspections are difficult to achieve all-weather, all-angle monitoring of river and lake areas, resulting in the "four chaos" problems that are repeatedly banned and low management efficiency. In addition, manual inspections are highly subjective and prone to omissions, making it difficult to ensure the fairness and accuracy of supervision.

[0005] Therefore, exploring an efficient and intelligent river and lake supervision technology has become an important issue that needs to be urgently addressed in the current field of river and lake protection and management. Summary of the invention

[0006] In view of the above problems, the purpose of the present invention is to provide a method, system, device and medium for detecting the four chaos in rivers and lakes based on video analysis. By utilizing drones and combining them with advanced video algorithm models, it can automatically identify and capture violations and illegal behaviors in rivers, lakes and along their lines, and provide management personnel with real-time and accurate monitoring data, thereby greatly improving the efficiency and accuracy of river and lake supervision.

[0007] In order to achieve the above object, the present invention is implemented through the following technical solutions:

[0008] In a first aspect, the present invention discloses a method for detecting four kinds of disorder in rivers and lakes based on video analysis, comprising: using a drone equipped with a video camera, and obtaining video images of rivers, lakes and their surroundings taken by the video camera in real time through wireless image transmission technology;

[0009] Use a multi-target tracking model to perform target recognition on video images and identify target objects in rivers, lakes and along the coast, including buildings, sand mining vessels, engineering vehicles and aquaculture cages;

[0010] The video image corresponding to the target object is converted into an image sequence, and the image sequence is input into the video classification model to classify the target object, and the target object is classified into illegal buildings, illegal sand mining vessels, illegal construction vehicles and illegal aquaculture cages, and the classification results are generated;

[0011] Generate violation records based on the classification results, extract relevant video clips and pictures, and display them.

[0012] Furthermore, the video camera is used to carry a drone and obtain video images of rivers, lakes and the surrounding areas taken by the video camera in real time through wireless image transmission technology, including:

[0013] By connecting to drones through the Internet of Things, the automatic flight inspection route of the drone can be set according to the regional information of rivers, lakes and along the lines. The drone can be controlled to collect video images using the video camera on the drone along the automatic flight inspection route, and the video images can be transmitted back through wireless image transmission technology.

[0014] Furthermore, the generation process of the multi-target tracking model includes:

[0015] Use any of the FairMoT algorithm, JDE algorithm, and Sort algorithm to build a basic multi-target tracking model;

[0016] Collect historical four-chaos real scene pictures, classify the four-chaos real scene pictures according to the four-chaos types, and annotate the four-chaos real scene pictures with DI to generate a tracking training set;

[0017] The tracking training set is input into the basic multi-target tracking model for model training, and the model parameters are optimized using a multi-task loss function. After the training is completed, a multi-target tracking model is generated.

[0018] Furthermore, the multi-task loss function is specifically as follows:

[0019]

[0020] in, is the i-th learnable model parameter, is the i-th classification loss function L F , is the i-th position offset loss function L W , is the i-th identity embedding loss function L S .

[0021] Furthermore, the classification loss function is specifically as follows:

[0022]

[0023] Among them, N F is the number of all anchor boxes, is the true probability, p i is the probability that the i-th anchor box is a foreground box; the position offset loss function is as follows:

[0024]

[0025] Among them, N W is the number of all candidate boxes, t i ={t x ,t y ,t w ,t h} is the predicted coordinate offset,

[0026] is the offset of the true value;

[0027] The identity embedding loss function is as follows:

[0028]

[0029] Among them, f T is the anchor box, g + is the weight of the positive example, is the weight of the negative example.

[0030] Furthermore, the video classification model is a video classification model constructed using the I3D algorithm and the SlowFast algorithm; in the neural network architecture of the video classification model, a Non-local module is connected after the 3D convolution; the last layer of the neural network of the video classification model outputs two probability values, which respectively represent the probability values ​​of the algorithm judging the input image sequence as normal behavior and illegal behavior, and the judgment result with a larger probability value is selected as the output result.

[0031] Further, generating violation records according to the classification results, extracting relevant video clips and pictures, and displaying them, includes:

[0032] Generate violation records based on the classification results. Based on the violation records, use the video processing tool Adobe PremierePro and the content review tool Tencent Cloud Content Security to extract relevant violation clips and pictures from the video images.

[0033] The violation records, violation clips and pictures are sent to a preset user interface.

[0034] In a second aspect, the present invention also discloses a river and lake four disorder detection system based on video analysis, including: a video acquisition module for using a drone equipped with a video camera to obtain video images of rivers, lakes and their surroundings taken in real time by the video camera through wireless image transmission technology;

[0035] The target detection module is used to identify the target of the video image using a multi-target tracking model, and identify the target objects in the river, lake and along the line, such as buildings, sand mining vessels, engineering vehicles and aquaculture cages;

[0036] The target classification module is used to convert the video image corresponding to the target object into an image sequence, input the video classification model to classify the target object, classify the target object into illegal buildings, illegal sand mining vessels, illegal construction vehicles and illegal aquaculture cages, and generate classification results;

[0037] The display module is used to generate violation records based on the classification results, extract relevant video clips and pictures, and display them.

[0038] In a third aspect, the present invention further discloses a river and lake four disorder detection device based on video analysis, comprising:

[0039] A memory, used for storing a river and lake four disorder detection program based on video analysis;

[0040] A processor is used to implement the steps of the river and lake four disorder detection method based on video analysis as described in any one of the above items when executing the river and lake four disorder detection program based on video analysis.

[0041] In a fourth aspect, the present invention further discloses a readable storage medium, on which is stored a program for detecting the four kinds of river and lake disorders based on video analysis. When the program for detecting the four kinds of river and lake disorders based on video analysis is executed by a processor, the steps of the method for detecting the four kinds of river and lake disorders based on video analysis as described in any one of the above items are implemented.

[0042] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention discloses a method, system, device and medium for detecting the four chaos in rivers and lakes based on video analysis, adopts an unmanned aerial vehicle equipped with a video camera, and obtains the video images of rivers, lakes and along the lines taken by the unmanned aerial vehicle camera in real time through a wireless image transmission technology method, and recognizes in real time the target objects such as illegal buildings, sand mining vessels, illegal construction vehicles, illegally occupied cages, etc. in the picture, screens out the illegal and unlawful targets near rivers, lakes and along the lines, and classifies the types of different identified target objects, and sends the identification results to the display layer for secondary analysis and classification, thereby realizing the classification of the different identified target types into the four chaos illegal behaviors such as illegal mining, illegal occupation, illegal construction and illegal stacking.

[0043] The present invention can identify and collect evidence of the four illegal behaviors in a specific inspection area through an automatic flight inspection route set by a drone in complex environments and large areas, thereby greatly reducing the workload of inspection and law enforcement personnel, improving work efficiency, effectively enhancing inspection and law enforcement management capabilities, and saving unit human, financial, and material investment costs such as manpower, vehicles, and fuel consumption.

[0044] It can be seen that compared with the prior art, the present invention has outstanding substantive features and significant progress, and the beneficial effects of its implementation are also obvious. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0046] Figure 1 It is a method flow chart of a specific implementation mode of the present invention.

[0047] Figure 2 It is a system structure diagram of a specific implementation mode of the present invention.

[0048] Figure 3 It is a device structure diagram of a specific implementation mode of the present invention.

[0049] In the figure, 1. video acquisition module; 2. target detection module; 3. target classification module; 4. target classification module; 101. processor; 102. memory; 103. input interface; 104. output interface; 105. communication unit; 106. keyboard; 107. display; 108. mouse. DETAILED DESCRIPTION

[0050] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0051] See also Figure 1 As shown, this embodiment provides a method for detecting river and lake chaos based on video analysis, comprising the following steps:

[0052] S1: Use a drone equipped with a video camera to obtain real-time video images of rivers, lakes and along the route through wireless image transmission technology.

[0053] In a specific implementation, the Internet of Things is used to connect with drones, and the automatic flight inspection route of the drone is set according to the regional information of rivers, lakes and along the lines. The drone is controlled to collect video images using the video camera on the drone on the automatic flight inspection route, and the video images are transmitted back through wireless image transmission technology.

[0054] S2: Use a multi-target tracking model to perform target recognition on video images and identify target objects in rivers, lakes and along the coast, including buildings, sand mining vessels, engineering vehicles and aquaculture cages.

[0055] In a specific implementation, the present invention uses a multi-target detection and tracking algorithm to track each target object in the drone screen area. The multi-target detection and tracking algorithm used in the present invention can select a detection and tracking combined method, such as FairMoT, JDE algorithm, etc., or can use an algorithm that detects first and then tracks, such as Sort, etc., and use any of the above algorithms to build a basic multi-target tracking model.

[0056] In terms of training the basic multi-target tracking algorithm, you can collect public multi-target tracking data sets for training, or you can collect some real-life pictures of the four chaos on site, classify the four chaos types in each picture and mark the ID, so as to make a multi-target, multi-type tracking training set. After obtaining the training set, you can input the basic multi-target tracking model for training. During training, use a multi-task loss function to optimize the model parameters. After training is completed, a multi-target tracking model is generated.

[0057] The multi-task loss function used is as follows:

[0058]

[0059] in, is the i-th learnable model parameter, is the i-th classification loss function L F , is the i-th position offset loss function L W , is the i-th identity embedding loss function L S .

[0060] The classification loss function can use cross entropy loss, as follows:

[0061]

[0062] Among them, N F is the number of all anchor boxes, is the true probability, p i is the probability that the i-th anchor box is a foreground box.

[0063] The position shift loss function is as follows:

[0064]

[0065] Among them, N W is the number of all candidate boxes, t i ={t x ,t y ,t w ,t h} is the predicted coordinate offset, is the offset from the true value.

[0066] The identity embedding loss function can select triple loss, and the identity embedding loss function is defined as:

[0067]

[0068] Among them, f T is the anchor box, g + is the weight of the positive example, is the weight of the negative example.

[0069] S3: Convert the video image corresponding to the target object into an image sequence, input the video classification model to classify the target object, classify the target object into illegal buildings, illegal sand mining vessels, illegal construction vehicles and illegally occupied aquaculture cages, and generate classification results.

[0070] In this step, the video classification model is a video classification model constructed using the I3D algorithm and the SlowFast algorithm; in the neural network architecture of the video classification model, the Non-local module is connected after the 3D convolution; the last layer of the neural network of the video classification model outputs two probability values, which respectively represent the probability values ​​of the algorithm judging the input image sequence as normal behavior and illegal behavior, and the judgment result with a larger probability value is selected as the output result.

[0071] In a specific implementation, after the target object is detected, the target objects in a certain area sequence are collected for sequence arrangement, and then the image sequence is input into the video classification algorithm, and the four disorder behaviors are judged according to the video classification algorithm. The last layer of the neural network of the video classification algorithm is the output of 2 dimensions, and two probability values ​​are output here, which respectively represent the judgment results of the algorithm on the input mask image sequence. The first probability value is the probability of normal behavior, and the second probability value is the probability of illegal behavior. Finally, the one with the larger probability value is selected as the output category of the algorithm. Among them, illegal behavior refers to the behavior that violates the four disorder.

[0072] S4: Generate violation records based on the classification results, extract relevant video clips and pictures, and display them.

[0073] In a specific implementation, first, a violation record is generated based on the classification results. Based on the violation record, the video processing tool Adobe Premiere Pro and the content review tool Tencent Cloud Content Security are used to extract relevant violation clips and pictures from the video image. Then the violation record, violation clips and pictures are sent to the preset user interface. The management personnel can conduct secondary analysis and classification based on the information displayed on the user interface, and classify the identified different target types into four illegal behaviors such as illegal mining, illegal occupation, illegal construction and illegal stacking, and conduct event processing tracing.

[0074] In addition, it should be noted that in order to be more suitable for scene applications, this method also uses acceleration tools such as NCNN and TensorRT to accelerate related visual algorithms through model quantization, model pruning, low precision, multi-threading, etc., and then rewrites the network loading part and the model post-processing part.

[0075] The present invention provides a method for detecting the four kinds of illegal activities in rivers and lakes based on video analysis. The method can identify and collect evidence of the four kinds of illegal activities in a specific inspection area through an automatic flight inspection route set by a drone in a complex environment and a large area, thereby greatly reducing the workload of inspection and law enforcement personnel, improving work efficiency, effectively enhancing inspection and law enforcement management capabilities, and saving unit human, financial, and material investment costs such as manpower, vehicles, and fuel consumption.

[0076] See also Figure 2 As shown, the present invention also discloses a river and lake chaos detection system based on video analysis, including: a video acquisition module 1, a target detection module 2, a target classification module 3 and a display module 4.

[0077] The video acquisition module 1 is used to use a drone equipped with a video camera to obtain video images of rivers, lakes and along the route taken by the video camera in real time through wireless image transmission technology.

[0078] The target detection module 2 is used to perform target recognition on the video image using a multi-target tracking model, and identify target objects in rivers, lakes and along the lines, including buildings, sand mining vessels, engineering vehicles, and aquaculture cages.

[0079] The target classification module 3 is used to convert the video image corresponding to the target object into an image sequence, input the video classification model to classify the target object, classify the target object into illegal buildings, illegal sand mining vessels, illegal construction vehicles and illegally occupied aquaculture cages, and generate classification results.

[0080] The display module 4 is used to generate violation records according to the classification results, extract relevant video clips and pictures, and display them.

[0081] The specific implementation of the river and lake four disorder detection system based on video analysis in this embodiment is basically the same as the specific implementation of the river and lake four disorder detection method based on video analysis mentioned above, and will not be repeated here.

[0082] See also Figure 3 As shown, the present invention also discloses a river-lake four-disorder detection device based on video analysis, comprising a processor 101 and a memory 102; wherein, when the processor 101 executes the river-lake four-disorder detection program based on video analysis stored in the memory, the steps of the river-lake four-disorder detection method based on video analysis as described in any one of the above items are implemented.

[0083] The river and lake chaos detection device based on video analysis provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer or a desktop computer.

[0084] Among them, the processor 101 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 101 can be implemented in at least one hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 101 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 101 may be integrated with a graphics processing unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 101 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0085] The memory 102 may include one or more computer-readable storage media, which may be non-transitory. The memory 102 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 102 is at least used to store the following computer program, wherein, after the computer program is loaded and executed by the processor 101, it can implement the relevant steps of the river and lake four chaos detection method based on video analysis disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 102 may also include an operating system and data, etc., and the storage method may be short-term storage or permanent storage. Among them, the operating system may include Windows, Unix, Linux, etc. The data may include but is not limited to the data involved in the above-mentioned river and lake four chaos detection method based on video analysis, etc.

[0086] Furthermore, the river and lake four chaos detection device based on video analysis in this embodiment may also include:

[0087] The input interface 103 is used to obtain the video analysis-based river and lake four disorder detection program imported from the outside, and save the obtained video analysis-based river and lake four disorder detection program to the memory 102. It can also be used to obtain various instructions and parameters transmitted by external terminal devices, and transmit them to the processor 101, so that the processor 101 can use the above various instructions and parameters to carry out corresponding processing. In this embodiment, the input interface 103 can specifically include but is not limited to a USB interface, a serial interface, a voice input interface, a fingerprint input interface, a hard disk reading interface, etc.

[0088] The output interface 104 is used to output various data generated by the processor 101 to a terminal device connected thereto, so that other terminal devices connected to the output interface can obtain various data generated by the processor 101. In this embodiment, the output interface 104 may specifically include but is not limited to a USB interface, a serial interface, and the like.

[0089] The communication unit 105 is used to establish a remote communication connection between the server operation service optimization configuration device and the external server, so that the river and lake four chaos detection device based on video analysis can mount the image file to the external server. In this embodiment, the communication unit 105 can specifically include but is not limited to a remote communication unit based on wireless communication technology or wired communication technology.

[0090] The keyboard 106 is used to obtain various parameter data or instructions input by the user by tapping the keycaps in real time.

[0091] The display 107 is used to display in real time the relevant information of the river and lake four chaos detection process based on video analysis.

[0092] The mouse 108 can be used to assist the user in inputting data and simplify the user's operation.

[0093] The present invention also discloses a readable storage medium, wherein the readable storage medium includes a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable hard disk, a CD-ROM or any other form of storage medium known in the technical field. The readable storage medium stores a video analysis-based river and lake four disorder detection program, and when the video analysis-based river and lake four disorder detection program is executed by a processor, the steps of the video analysis-based river and lake four disorder detection method described in any one of the above items are implemented.

[0094] In summary, the present invention utilizes drones in combination with advanced video algorithm models to automatically identify and capture irregularities and illegal behaviors in rivers, lakes and along their lines, and provide management personnel with real-time and accurate monitoring data, thereby greatly improving the efficiency and accuracy of river and lake supervision.

[0095] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. As for the method disclosed in the embodiment, since it corresponds to the system disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0096] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0097] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0098] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0099] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, or each module may exist physically separately, or two or more modules may be integrated into one unit.

[0100] Similarly, each processing unit in each embodiment of the present invention may be integrated into one functional module, or each processing unit may exist physically, or two or more processing units may be integrated into one functional module.

[0101] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0102] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0103] The above is a detailed introduction to the river and lake four chaos detection method, system, device and readable storage medium based on video analysis provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A method for detecting river and lake disturbances based on video analysis, characterized in that: include: Use drones equipped with video cameras to obtain real-time video images of rivers, lakes and along the route through wireless image transmission technology; Use a multi-target tracking model to perform target recognition on video images and identify target objects in rivers, lakes and along the coast, including buildings, sand mining vessels, engineering vehicles and aquaculture cages; The video image corresponding to the target object is converted into an image sequence, and the image sequence is input into the video classification model to classify the target object, and the target object is classified into illegal buildings, illegal sand mining vessels, illegal construction vehicles and illegal aquaculture cages, and the classification results are generated; Generate violation records based on the classification results, extract relevant video clips and pictures, and display them.

2. The method for detecting river and lake disturbances based on video analysis according to claim 1, characterized in that: The video camera mounted on the drone is used to obtain the video images of the river, lake and the surrounding area taken by the video camera in real time through wireless image transmission technology, including: By connecting to drones through the Internet of Things, the automatic flight inspection route of the drone can be set according to the regional information of rivers, lakes and along the lines. The drone can be controlled to collect video images using the video camera on the drone along the automatic flight inspection route, and the video images can be transmitted back through wireless image transmission technology.

3. The method for detecting river and lake disturbances based on video analysis according to claim 1, characterized in that: The generation process of the multi-target tracking model includes: Use any of the FairMoT algorithm, JDE algorithm, and Sort algorithm to build a basic multi-target tracking model; Collect historical four-chaos real scene pictures, classify the four-chaos real scene pictures according to the four-chaos types, and annotate the four-chaos real scene pictures with DI to generate a tracking training set; The tracking training set is input into the basic multi-target tracking model for model training, and the model parameters are optimized using a multi-task loss function. After the training is completed, a multi-target tracking model is generated.

4. The method for detecting river and lake disturbances based on video analysis according to claim 3 is characterized in that: The multi-task loss function is as follows: in, is the i-th learnable model parameter, is the i-th classification loss function L F , is the i-th position offset loss function L W , is the i-th identity embedding loss function L S .

5. The method for detecting river and lake disturbances based on video analysis according to claim 4 is characterized in that: The classification loss function is as follows: Among them, N F is the number of all anchor boxes, is the true probability, p i is the probability that the i-th anchor box is a foreground box; the position offset loss function is as follows: Among them, N W is the number of all candidate boxes, t i ={t x ,t y ,t w ,t h } is the predicted coordinate offset, is the offset of the true value; The identity embedding loss function is as follows: Among them, f T is the anchor box, g + is the weight of the positive example, is the weight of the negative example.

6. The method for detecting river and lake disturbances based on video analysis according to claim 1, characterized in that: The video classification model is a video classification model constructed using the I3D algorithm and the SlowFast algorithm; In the neural network architecture of the video classification model, a Non-local module is connected after the 3D convolution; The last layer of the neural network of the video classification model outputs two probability values, which respectively represent the probability values ​​of the algorithm judging the input image sequence as normal behavior and illegal behavior. The judgment result with a larger probability value is selected as the output result.

7. The method for detecting river and lake disturbances based on video analysis according to claim 1, characterized in that: The violation record is generated according to the classification result, and relevant video clips and pictures are extracted and displayed, including: Generate violation records based on the classification results. Based on the violation records, use the video processing tool Adobe Premiere Pro and the content review tool Tencent Cloud Content Security to extract relevant violation clips and pictures from the video images. The violation records, violation clips and pictures are sent to a preset user interface.

8. A river and lake disorder detection system based on video analysis, characterized in that: include: The video acquisition module is used to use a drone equipped with a video camera to obtain the video images of rivers, lakes and along the route taken by the video camera in real time through wireless image transmission technology; The target detection module is used to identify the target of the video image using a multi-target tracking model, and identify the target objects in the river, lake and along the line, such as buildings, sand mining vessels, engineering vehicles and aquaculture cages; The target classification module is used to convert the video image corresponding to the target object into an image sequence, input the video classification model to classify the target object, classify the target object into illegal buildings, illegal sand mining vessels, illegal construction vehicles and illegal aquaculture cages, and generate classification results; The display module is used to generate violation records based on the classification results, extract relevant video clips and pictures, and display them.

9. A river and lake disorder detection device based on video analysis, characterized in that: include: A memory, used for storing a river and lake four disorder detection program based on video analysis; A processor is used to implement the steps of the river and lake four disorder detection method based on video analysis as described in any one of claims 1 to 7 when executing the river and lake four disorder detection program based on video analysis.

10. A readable storage medium, characterized in that: The readable storage medium stores a program for detecting the four kinds of river and lake disorders based on video analysis. When the program for detecting the four kinds of river and lake disorders based on video analysis is executed by a processor, the steps of the method for detecting the four kinds of river and lake disorders based on video analysis are implemented as described in any one of claims 1 to 7.

Images