A method, system, device and readable storage medium for identifying illegal construction in a river

Abstract

The present invention proposes a method, system, device and readable storage medium for identifying illegal buildings in rivers, which belong to the field of computer vision technology. The method realizes the identification of illegal buildings in rivers based on SipMaskv2 and improved YOLOv8. Specifically, the drone sets a fixed recovery point as the zero point to establish a coordinate system. The drone flies along a preset route to obtain video data, pre-processes the video information and extracts frames to annotate to establish a data set, and trains two models, namely a river segmentation model and an illegal building identification model. In a specific scenario, it is necessary to detect the edge of the river channel in the first flight, expand it according to actual requirements, and convert it to a flight coordinate system. After that, during the flight, when entering the expanded river channel, illegal buildings are identified, the detected illegal building coordinates are converted, and finally the final position of the illegal buildings in the river channel is obtained. The present invention provides a method for intelligent identification of illegal buildings in the detection of four chaos in the river channel, and provides strong technical support for the detection of four chaos in the river channel.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and more specifically to a method, system, device and readable storage medium for identifying illegal constructions in rivers. Background Art

[0002] In the key area of ​​river management, ensuring the ecological balance and flood safety of the river is of vital importance. Illegal buildings around the river not only destroy the natural landscape, but may also have a serious impact on the natural flow and water quality of the river, thereby threatening the safety of the surrounding residents and the sustainable development of the ecological environment. For a long time, managers have relied on traditional manual inspections to discover and deal with these illegal buildings, but this method faces many challenges.

[0003] With the advancement of science and technology, the integration of drone technology and computer vision technology has brought new solutions to river management. With its high maneuverability and wide field of view, drones can quickly cover large areas and conduct efficient and low-cost inspections. The introduction of deep learning algorithms has further improved the intelligence level of illegal construction identification, and is expected to achieve a fully automated process from data collection to illegal construction identification.

[0004] However, despite the broad prospects of this technology, the existing intelligent identification methods for illegal river construction still have certain limitations. Low recognition accuracy may lead to false alarms or missed alarms, increasing the difficulty and cost of subsequent processing. At the same time, the river environment and illegal construction types in different regions vary greatly, and the adaptability of existing technologies is not strong, making it difficult to meet the identification needs in different scenarios. Therefore, how to further improve the accuracy and adaptability of intelligent identification methods is a key issue that needs to be urgently addressed in the current field of river management. Summary of the invention

[0005] In view of the above problems, the purpose of the present invention is to provide a method, system, device and readable storage medium for identifying illegal construction in rivers. Based on the SipMaskv2 network and the improved yolov8 model, intelligent identification of illegal construction in rivers is realized, providing strong technical support for the detection of four chaos in rivers.

[0006] In order to achieve the above object, the present invention is implemented through the following technical solutions: In a first aspect, the present invention discloses a method for identifying illegal construction in a river, comprising: Use drones to collect video information along preset inspection routes along the river; Screening and expanding the collected video information to generate river image datasets and building image datasets; Based on the river image dataset, the SipMaskv2 network is used for model training to generate a river segmentation model; Based on the building image dataset, the improved yolov8 model is used for training to generate an illegal building recognition model; Use the drone to perform pre-inspection along the preset inspection route of the river, collect pre-inspection video information, and calculate the coordinate system transformation matrix H based on the equipment parameters and position information of the drone; Based on the initial inspection video information, the river segmentation model is used to identify the pixel coordinate information of the river edge; The pixel coordinate information of the river channel edge is converted into the flight coordinate system using the conversion matrix H, and the distance expansion processing is performed to generate the converted river channel edge coordinate information; When the inspection video information collected by the drone in real time is cropped and confirmed to contain the converted river edge coordinate information, the illegal building recognition model is used to identify the illegal buildings, and the identified illegal building location information is converted from the pixel coordinate system to the flight coordinate system.

[0007] Furthermore, the method of collecting video information using a drone along a preset inspection route of a river includes: Determine the fixed recovery position of the drone and use it as the origin to establish the drone flight coordinate system; Set inspection routes and use drones to perform inspections according to the inspection routes; When the drone reaches the preset height, video information is collected.

[0008] Furthermore, the process of screening and expanding the collected video information to generate a river image dataset and a building image dataset includes: Perform frame extraction on the video information, extract key frames, and crop the effective area in the key frames to generate data images; According to the river channel images in the data images, duplicate data images are removed; When the number of data images is lower than the preset threshold, the generative adversarial network (GAN) is used for data expansion; After the expansion is completed, an image dataset is generated; Based on the image dataset, the X-Anylabeling tool is used to label the river images and generate a river image dataset; Based on the image dataset, the X-Anylabeling tool is used to annotate the architectural images with rotating boxes to generate the architectural image dataset.

[0009] Furthermore, the pre-inspection video information is collected, and the coordinate system conversion matrix H is calculated according to the equipment parameters and position information of the UAV, including: Collect pre-inspection video information, and extract the 960×960 pixel area as the effective area as the video information to be identified; Get the camera parameters of the drone, the height of the camera from the ground and the position information of the drone; According to the camera parameters, camera height and drone position, a conversion matrix from the camera pixel coordinate system to the drone flight coordinate system is calculated as the coordinate system conversion matrix H.

[0010] Further, based on the initial inspection video information, the river segmentation model is used to identify the pixel coordinate information of the river edge, including: Import the video information to be identified into the river segmentation model to obtain the river identification mask; Perform image processing on the river identification mask Mask to extract the pixel coordinate information of the river edge. Further, the use of the conversion matrix H to convert the pixel coordinate information of the river edge into the flight coordinate system and perform distance expansion processing to generate the converted river edge coordinate information includes: Use the transformation matrix H to transform the edge pixel coordinate information of the river channel into the flight coordinate system to generate a transformed image; By comparing the river edge coordinates of any two converted images, the overlapping area of ​​the images is found; For the overlapping areas of the images, calculate the area of ​​the common area enclosed by the edges of the two relevant river channels; The river channel with the larger area is selected as the final representation of the overlapping area, and the corresponding river channel edge coordinates are updated; According to actual needs, the river edge coordinates are expanded outward by a preset distance to generate converted river edge coordinate information.

[0011] Furthermore, when the inspection video information collected by the drone in real time is cropped and confirmed to contain the converted river edge coordinate information, the illegal building recognition model is used to identify the illegal building, and the position information of the identified illegal building is converted from the pixel coordinate system to the flight coordinate system, including: The inspection video information collected by the drone in real time is collected into a 960×960 window size; Determine whether the window contains the converted river edge coordinate information; If yes, the image information of the window is imported into the illegal building recognition model for recognition, and the location information of the illegal building is obtained; The coordinate system conversion matrix H is used to convert the identified illegal building location information into the flight coordinate system, generate the corresponding location information, and output it as warning information. In a second aspect, the present invention further discloses a river illegal construction identification system, comprising: A data acquisition module is used to collect video information using a drone along a preset inspection route of the river; The data set building module is used to filter and expand the collected video information to generate river image data sets and building image data sets; The first model training module is used to perform model training based on the river channel image dataset using the SipMaskv2 network to generate a river channel segmentation model; The second model training module is used to generate an illegal building recognition model based on the building image dataset and trained using the improved yolov8 model; The conversion matrix generation module is used to perform pre-inspection along the preset inspection route of the river channel using the UAV, collect pre-inspection video information, and calculate the coordinate system conversion matrix H based on the equipment parameters and position information of the UAV; The edge recognition module is used to identify the pixel coordinate information of the river channel edge based on the initial inspection video information and the river channel segmentation model; A coordinate conversion module is used to convert the pixel coordinate information of the river channel edge into the flight coordinate system using the conversion matrix H, and perform distance expansion processing to generate the converted river channel edge coordinate information; The real-time recognition module is used to identify illegal buildings using the illegal building recognition model when the inspection video information collected by the drone in real time is cropped and confirmed to contain the converted river edge coordinate information, and to convert the identified illegal building location information from the pixel coordinate system to the flight coordinate system.

[0012] In a third aspect, the present invention further discloses a device for identifying illegal construction in a river, comprising: A memory, used for storing a program for identifying illegal river construction; A processor is used to implement the steps of the method for identifying illegal buildings in a river as described in any of the above items when executing the illegal buildings in a river identification program.

[0013] In a fourth aspect, the present invention further discloses a readable storage medium, on which is stored a program for identifying illegal constructions in a river channel. When the program for identifying illegal constructions in a river channel is executed by a processor, the steps of the method for identifying illegal constructions in a river channel as described in any one of the above items are implemented.

[0014] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention utilizes drones to inspect along preset routes, combined with advanced image processing and machine learning technologies, to achieve efficient and accurate identification of illegal construction in the river, significantly improving inspection efficiency and identification accuracy.

[0015] 2. The present invention generates a special river segmentation model and illegal building identification model through intelligent data processing and model training. At the same time, it uses precise coordinate system conversion technology to ensure the geographic location accuracy of the identification results, providing strong support for subsequent supervision and processing.

[0016] 3. The present invention can output warning information of illegal buildings in real time, helping relevant departments to take timely measures, effectively maintaining the safety and compliance of the river, and has important practical application value.

[0017] It can be seen that compared with the prior art, the present invention has outstanding substantive features and significant progress, and the beneficial effects of its implementation are also obvious. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0019] Figure 1 It is a method flow chart of a specific implementation mode of the present invention.

[0020] Figure 2 It is a system structure diagram of a specific implementation mode of the present invention.

[0021] Figure 3 It is a schematic diagram of the device structure of a specific implementation mode of the present invention.

[0022] In the figure, 1. data acquisition module; 2. data set establishment module; 3. first model training module; 4. second model training module; 5. transformation matrix generation module; 6. edge recognition module; 7. coordinate transformation module; 8. real-time recognition module; 101. processor; 102. memory; 103. input interface; 104. output interface; 105. communication unit; 106. keyboard; 107. display; 108. mouse. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0024] See also Figure 1 As shown, this embodiment provides a method for identifying illegal construction in a river, comprising the following steps: S1: Use drones to collect video information along the preset inspection route of the river.

[0025] In a specific implementation, the fixed recovery position of the drone is first determined, and the drone flight coordinate system is established with this as the origin. Then, the inspection route is set, and the drone is used to perform inspections according to the inspection route; When the drone reaches the preset height, the camera is set perpendicular to the ground to shoot and collect video information.

[0026] S2: Screening and expanding the collected video information to generate a river image dataset and a building image dataset.

[0027] In a specific implementation, first, the video information is subjected to frame extraction, key frames are extracted, and the effective area in the key frames is cropped to generate a data picture. As an example, the collected video information is subjected to frame extraction, and the middle 960×960 area is cropped to generate a data picture. Then, according to the river channel image in the data picture, the duplicate data pictures are removed; When the number of data images is lower than the preset threshold, the generative adversarial network (GAN) is used for data expansion.

[0028] After the expansion is completed, an image data set is generated. As an example, when the number of drone flights is small and the data images are not enough, river materials at different heights and angles can be collected from different channels. Generative Adversarial Network (GAN) can also be used to expand data. Among them, GAN is divided into generator and discriminator, both of which are a kind of neural network model. The task of the generator network is to generate distribution based on a set of random noise data to form new data; the task of the discriminator network is to evaluate whether the newly generated data is real, that is, whether it comes from a real training set. The GAN network can expand data in large quantities, and the image clarity can also be enhanced. GAN is highly operational and can learn various data features without supervision.

[0029] At this point, based on the image dataset, the X-Anylabeling tool is used to label the river images to generate a river image dataset. Specifically, the labeling requirements are: try to fit the river boundary, do not expose too much background, and maintain labeling consistency. The label of the label content is river_course. At the same time, based on the image dataset, the X-Anylabeling tool is used to annotate the building images with rotating boxes to generate a building image dataset. The annotation requirements are: try to fit the roof boundary and do not expose too much background. When the house is at the edge of the image and at an angle, the end that can be wrapped away from the edge of the image is given priority. The label is roof.

[0030] S3: Based on the river image dataset, the SipMaskv2 network is used for model training to generate a river segmentation model.

[0031] S4: Based on the building image dataset, the improved yolov8 model is used for training to generate an illegal building recognition model.

[0032] In the specific real-time method, the established building image dataset is trained using the improved yolov8 model. Yolov8 adds the Triplet Attention mechanism, and because most houses are rectangular and present at different angles, if the detection head is used and modified, a branch is added to predict the rotation angle.

[0033] S5: Use the drone to perform pre-inspection along the preset inspection route of the river, collect pre-inspection video information, and calculate the coordinate system transformation matrix H based on the equipment parameters and position information of the drone.

[0034] In a specific real-time method, first collect the pre-inspection video information, extract the 960×960 pixel area as the valid area, and use it as the video information to be identified. Then, obtain the camera parameters of the drone, the height of the camera from the ground, and the position information of the drone. Finally, according to the camera parameters, camera height, and drone position, calculate the conversion matrix from the camera pixel coordinate system to the drone flight coordinate system as the coordinate system conversion matrix H.

[0035] As an example, a drone is used for the first test flight inspection. Taking the drone camera pixel of 1080×1920 as an example, the middle 960×960 pixel area is taken as the effective area. According to the camera parameters, the height of the camera from the ground and the location of the drone, the transformation matrix H that maps the camera pixel coordinate system to the drone flight coordinate system is calculated.

[0036] S6: Based on the initial inspection video information, the river segmentation model is used to identify the pixel coordinate information of the river edge.

[0037] In a specific implementation, firstly, the video information to be identified is imported into the river segmentation model to obtain a river identification mask Mask; then, the river identification mask Mask is subjected to image processing to extract the pixel coordinate information of the river edge.

[0038] As an example, the video information to be identified is imported into the river segmentation model to obtain the river identification mask Mask, the Mask is closed, and then the contour is searched to obtain the edge pixel coordinate information of the river.

[0039] S7: Use the transformation matrix H to transform the pixel coordinate information of the river channel edge into the flight coordinate system, and perform distance expansion processing to generate the transformed river channel edge coordinate information.

[0040] In a specific implementation, the edge pixel coordinate information of the river channel is first converted into the flight coordinate system using the conversion matrix H to generate a converted image; then, the overlapping area of ​​the image is found by comparing the river channel edge coordinates of any two frames of the converted image.

[0041] For the overlapping areas of the images, the area of ​​the common area enclosed by the edges of the two relevant rivers is calculated; at this time, the river with the larger area is selected as the final representation of the overlapping area, and the corresponding river edge coordinates are updated. Finally, according to actual needs, the river edge coordinates are expanded outward by a preset distance to generate the converted river edge coordinate information.

[0042] In this step, if there is an overlapping river area between the two converted images, the river with the largest area is used as the reference. The river edge coordinates are expanded outward by a certain distance, and the specific situation is subject to actual requirements.

[0043] S8: When the inspection video information collected by the drone in real time is cropped and confirmed to contain the converted river edge coordinate information, the illegal building recognition model is used to identify the illegal building, and the identified illegal building location information is converted from the pixel coordinate system to the flight coordinate system.

[0044] In a specific implementation, during the subsequent flight inspection process, the inspection video information collected by the drone in real time is collected into a 960×960 window size.

[0045] First, determine whether the window contains the converted river edge coordinate information; if so, import the image information of the window into the illegal building recognition model for recognition to obtain the location information of the illegal building.

[0046] Then, the coordinate system conversion matrix H is used to convert the identified illegal building location information into the flight coordinate system, generate corresponding location information, and output it as warning information.

[0047] Since the warning information includes the flight coordinates of the illegal buildings, relevant departments can conduct on-site verification and processing after receiving the warning information. In addition, the necessary adjustments and optimizations can be made to the drone inspection and identification methods based on the processing results.

[0048] The present invention provides a method for identifying illegal buildings in rivers, and the identification of illegal buildings in rivers is realized based on SipMaskv2 and improved YOLOv8. Specifically, a fixed recovery point is set as the zero point for the drone to establish a coordinate system. The drone flies along a preset route to obtain video data, pre-processes the video information and extracts frames to annotate to establish a data set, and trains two models, namely a river segmentation model and an illegal building identification model. In a specific scenario, it is necessary to detect the edge of the river channel in the first flight, expand it according to actual requirements, and convert it to a flight coordinate system. After that, during the flight, when entering the expanded river channel, illegal buildings are identified, the detected illegal building coordinates are converted, and finally the final position of the illegal buildings in the river channel is obtained. This method provides a method for intelligent identification of illegal buildings in the detection of four chaos in the river channel, and provides strong technical support for the detection of four chaos in the river channel.

[0049] See also Figure 2 As shown, the present invention also discloses a river illegal construction identification system, including: a data acquisition module 1, a data set establishment module 2, a first model training module 3, a second model training module 4, a transformation matrix generation module 5, an edge recognition module 6, a coordinate conversion module 7 and a real-time recognition module 8. The data acquisition module 1 is used to collect video information using a drone along a preset inspection route of the river.

[0050] The data set establishment module 2 is used to filter and expand the collected video information to generate a river image data set and a building image data set.

[0051] The first model training module 3 is used to perform model training based on the river image dataset using the SipMaskv2 network to generate a river segmentation model.

[0052] The second model training module 4 is used to generate an illegal building recognition model based on the building image data set and using the improved yolov8 model for training.

[0053] The conversion matrix generation module 5 is used to perform pre-inspection along the preset inspection route of the river channel using the UAV, collect pre-inspection video information, and calculate the coordinate system conversion matrix H according to the equipment parameters and position information of the UAV.

[0054] The edge recognition module 6 is used to identify the pixel coordinate information of the river channel edge based on the initial inspection video information and using the river channel segmentation model.

[0055] The coordinate conversion module 7 is used to convert the pixel coordinate information of the river channel edge into the flight coordinate system using the conversion matrix H, and perform distance expansion processing to generate the converted river channel edge coordinate information.

[0056] The real-time identification module 8 is used to identify illegal buildings using the illegal building identification model when the inspection video information collected by the drone in real time is cropped and confirmed to contain the converted river edge coordinate information, and to convert the identified illegal building location information from the pixel coordinate system to the flight coordinate system.

[0057] The specific implementation of the illegal river construction identification system of this embodiment is basically the same as the specific implementation of the illegal river construction identification method described above, and will not be repeated here.

[0058] See also Figure 3 As shown, the present invention also discloses a device for identifying illegal constructions in rivers, including a processor 101 and a memory 102; wherein, when the processor 101 executes the illegal construction identification program in rivers stored in the memory, the steps of the method for identifying illegal constructions in rivers as described in any one of the above items are implemented.

[0059] The device for identifying illegal river construction provided in this embodiment may include, but is not limited to, a smart phone, a tablet computer, a laptop computer, or a desktop computer.

[0060] Among them, the processor 101 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 101 can be implemented in at least one hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 101 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 101 may be integrated with a graphics processing unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 101 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0061] The memory 102 may include one or more computer-readable storage media, which may be non-transitory. The memory 102 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 102 is at least used to store the following computer program, wherein, after the computer program is loaded and executed by the processor 101, the relevant steps of the river channel illegal construction identification method disclosed in any of the aforementioned embodiments can be implemented. In addition, the resources stored in the memory 102 may also include an operating system and data, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system may include Windows, Unix, Linux, etc. The data may include, but is not limited to, the data involved in the above-mentioned river channel illegal construction identification method, etc.

[0062] Furthermore, the river illegal construction identification device in this embodiment may also include: The input interface 103 is used to obtain the river channel illegal construction identification program imported from the outside, and save the obtained river channel illegal construction identification program to the memory 102. It can also be used to obtain various instructions and parameters transmitted by external terminal devices, and transmit them to the processor 101, so that the processor 101 can use the above various instructions and parameters to carry out corresponding processing. In this embodiment, the input interface 103 can specifically include but is not limited to a USB interface, a serial interface, a voice input interface, a fingerprint input interface, a hard disk reading interface, etc.

[0063] The output interface 104 is used to output various data generated by the processor 101 to a terminal device connected thereto, so that other terminal devices connected to the output interface can obtain various data generated by the processor 101. In this embodiment, the output interface 104 may specifically include but is not limited to a USB interface, a serial interface, and the like.

[0064] The communication unit 105 is used to establish a remote communication connection between the server operation service optimization configuration device and the external server, so that the river illegal construction identification device can mount the image file to the external server. In this embodiment, the communication unit 105 can specifically include but is not limited to a remote communication unit based on wireless communication technology or wired communication technology.

[0065] The keyboard 106 is used to obtain various parameter data or instructions input by the user by tapping the keycaps in real time.

[0066] The display 107 is used to display the relevant information of the river illegal construction identification process in real time.

[0067] The mouse 108 can be used to assist the user in inputting data and simplify the user's operation.

[0068] The present invention also discloses a readable storage medium, wherein the readable storage medium includes a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable hard disk, a CD-ROM or any other form of storage medium known in the technical field. The readable storage medium stores a program for identifying illegal river constructions, and when the program for identifying illegal river constructions is executed by a processor, the steps of the method for identifying illegal river constructions described in any one of the above items are implemented.

[0069] In summary, the present invention realizes the intelligent identification of illegal construction in the river based on the SipMaskv2 network and the improved yolov8 model, providing powerful technical support for the detection of four irregularities in the river.

[0070] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. As for the method disclosed in the embodiment, since it corresponds to the system disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0071] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0072] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0073] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0074] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, or each module may exist physically separately, or two or more modules may be integrated into one unit.

[0075] Similarly, each processing unit in each embodiment of the present invention may be integrated into one functional module, or each processing unit may exist physically, or two or more processing units may be integrated into one functional module.

[0076] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0077] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0078] The above is a detailed introduction to the illegal construction identification method, system, device and readable storage medium provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A method for identifying illegal construction in a river, characterized in that: include: Use drones to collect video information along preset inspection routes along the river; Screening and expanding the collected video information to generate river image datasets and building image datasets; Based on the river image dataset, the SipMaskv2 network is used for model training to generate a river segmentation model; Based on the building image dataset, the improved yolov8 model is used for training to generate an illegal building recognition model; Use the drone to perform pre-inspection along the preset inspection route of the river, collect pre-inspection video information, and calculate the coordinate system transformation matrix H based on the equipment parameters and position information of the drone; Based on the initial inspection video information, the river segmentation model is used to identify the pixel coordinate information of the river edge; The pixel coordinate information of the river channel edge is converted into the flight coordinate system using the conversion matrix H, and the distance expansion processing is performed to generate the converted river channel edge coordinate information; When the inspection video information collected by the drone in real time is cropped and confirmed to contain the converted river edge coordinate information, the illegal building recognition model is used to identify the illegal buildings, and the identified illegal building location information is converted from the pixel coordinate system to the flight coordinate system.

2. The method for identifying illegal river construction according to claim 1, characterized in that: The method of collecting video information by using a drone along a preset inspection route of a river includes: Determine the fixed recovery position of the drone and use it as the origin to establish the drone flight coordinate system; Set inspection routes and use drones to perform inspections according to the inspection routes; When the drone reaches the preset height, video information is collected.

3. The method for identifying illegal construction in a river according to claim 1, characterized in that: The screening and expansion of the collected video information to generate a river image dataset and a building image dataset includes: Perform frame extraction on the video information, extract key frames, and crop the effective area in the key frames to generate data images; According to the river channel images in the data images, duplicate data images are removed; When the number of data images is lower than the preset threshold, the generative adversarial network (GAN) is used for data expansion; After the expansion is completed, an image dataset is generated; Based on the image dataset, the X-Anylabeling tool is used to label the river images and generate a river image dataset; Based on the image dataset, the X-Anylabeling tool is used to annotate the architectural images with rotating boxes to generate the architectural image dataset.

4. The method for identifying illegal river construction according to claim 3, characterized in that: The collecting of pre-inspection video information and calculating the coordinate system conversion matrix H according to the equipment parameters and position information of the UAV include: Collect pre-inspection video information, and extract the 960×960 pixel area as the effective area as the video information to be identified; Get the camera parameters of the drone, the height of the camera from the ground and the location information of the drone; According to the camera parameters, camera height and drone position, a conversion matrix from the camera pixel coordinate system to the drone flight coordinate system is calculated as the coordinate system conversion matrix H.

5. The method for identifying illegal river construction according to claim 4, characterized in that: Based on the initial inspection video information, the river segmentation model is used to identify the pixel coordinate information of the river edge, including: Import the video information to be identified into the river segmentation model to obtain the river identification mask; Perform image processing on the river identification mask Mask to extract the pixel coordinate information of the river edge.

6. The method for identifying illegal river construction according to claim 5, characterized in that: The method of using the conversion matrix H to convert the pixel coordinate information of the river channel edge into the flight coordinate system and performing distance expansion processing to generate the converted river channel edge coordinate information includes: Use the transformation matrix H to transform the edge pixel coordinate information of the river channel into the flight coordinate system to generate a transformed image; By comparing the river edge coordinates of any two converted images, the overlapping area of ​​the images is found; For the overlapping areas of the images, calculate the area of ​​the common area enclosed by the edges of the two relevant river channels; The river channel with the larger area is selected as the final representation of the overlapping area, and the corresponding river channel edge coordinates are updated; According to actual needs, the river edge coordinates are expanded outward by a preset distance to generate converted river edge coordinate information.

7. The method for identifying illegal river construction according to claim 6, characterized in that: When the inspection video information collected by the drone in real time is cropped and confirmed to contain the converted river edge coordinate information, the illegal building recognition model is used to identify the illegal building, and the position information of the identified illegal building is converted from the pixel coordinate system to the flight coordinate system, including: The inspection video information collected by the drone in real time is collected into a 960×960 window size; Determine whether the window contains the converted river edge coordinate information; If yes, the image information of the window is imported into the illegal building recognition model for recognition, and the location information of the illegal building is obtained; The coordinate system conversion matrix H is used to convert the identified illegal building location information into the flight coordinate system, generate the corresponding location information, and output it as warning information.

8. A river illegal construction identification system, characterized in that: The system adopts the method for identifying illegal construction in a river as claimed in any one of claims 1 to 7; The system comprises: A data acquisition module is used to collect video information using a drone along a preset inspection route of the river; The data set building module is used to filter and expand the collected video information to generate river image data sets and building image data sets; The first model training module is used to perform model training based on the river channel image dataset using the SipMaskv2 network to generate a river channel segmentation model; The second model training module is used to generate an illegal building recognition model based on the building image dataset and trained using the improved yolov8 model; The conversion matrix generation module is used to perform pre-inspection along the preset inspection route of the river channel using the UAV, collect pre-inspection video information, and calculate the coordinate system conversion matrix H based on the equipment parameters and position information of the UAV; The edge recognition module is used to identify the pixel coordinate information of the river channel edge based on the initial inspection video information and the river channel segmentation model; A coordinate conversion module is used to convert the pixel coordinate information of the river channel edge into the flight coordinate system using the conversion matrix H, and perform distance expansion processing to generate the converted river channel edge coordinate information; The real-time recognition module is used to identify illegal buildings using the illegal building recognition model when the inspection video information collected by the drone in real time is cropped and confirmed to contain the converted river edge coordinate information, and to convert the identified illegal building location information from the pixel coordinate system to the flight coordinate system.

9. A device for identifying illegal construction in a river, characterized in that: include: A memory, used for storing a program for identifying illegal river construction; A processor is used to implement the steps of the method for identifying illegal buildings in a river as described in any one of claims 1 to 7 when executing the illegal building identification program in the river.

10. A readable storage medium, characterized in that: The readable storage medium stores a program for identifying illegal constructions in a river channel, and when the program for identifying illegal constructions in a river channel is executed by a processor, the steps of the method for identifying illegal constructions in a river channel as described in any one of claims 1 to 7 are implemented.

Images