Intelligent control system of trash cleaning machine based on visual AI recognition algorithm

Abstract

The application discloses a kind of intelligent control system of trash cleaner based on visual AI identification algorithm, including intelligent decision platform and intelligent drive control system, and the intelligent decision platform is carried out by visual AI algorithm and is judged to the logic of exceeding early warning threshold in shape and position identification, accumulation amount measurement, trash cleaning decision, trash cleaning plan command is automatically sent to the intelligent drive control system, and the intelligent drive control system receives the control command of intelligent decision platform and controls trash cleaner to carry out trash cleaning operation.The application realizes the intelligent identification of trash screen water inlet floating object, automatic trash cleaning and centralized control remote semi-automatic trash cleaning operation, to improve the intelligent level of power plant equipment, meet the needs of remote centralized control center to Liuping lock head water inlet trash screen water surface floating garbage supervision and cleaning, while reducing the work intensity of manual operation in main flood season.

Description

Technical Field

[0001] This invention relates to an intelligent control system for a cleaning machine, and more particularly to an intelligent control system for a cleaning machine based on a visual AI recognition algorithm. Background Technology

[0002] Artificial intelligence (AI) is the theory, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence. Currently, the domestic AI market mainly includes computer vision, speech recognition, natural language processing, basic algorithm platforms, and chips. Computer vision is the most important part of the AI ​​market and a major technological application, accounting for about 35%, and its industry chain is relatively mature. Existing technology for reservoirs results in a large amount of scum during the flood season, requiring timely scum removal and cleaning. However, the current Liuzha sludge removal machine can only be controlled locally via a handle and cannot be centrally controlled remotely. When roads are interrupted, it cannot perform emergency remote sludge removal operations, affecting the safe and stable operation of the unit.

[0003] Current cleaning operations primarily rely on manual inspection and judgment, as well as feedback from grid differentials. They lack intelligent identification and automatic cleaning capabilities. Furthermore, manual inspections have time intervals; if a grid differential alarm is triggered and there isn't enough time for cleaning operations, it will inevitably lead to reduced unit load or shutdown for peak shaving, resulting in economic losses for the cascade power plants in the river basin. The horizontal walking mechanism of the cleaning machine is approximately 60 meters long, and the slag removal operation is performed manually using a repeated "grab and release" cycle, which is labor-intensive. Summary of the Invention

[0004] The purpose of this invention is to provide an intelligent control system for a cleaning machine based on a visual AI recognition algorithm.

[0005] To achieve the above objectives, the present invention is implemented according to the following technical solution:

[0006] The present invention includes an intelligent decision-making platform and an intelligent drive and control system. The intelligent decision-making platform performs cleaning and decontamination inspection, shape and position recognition, volume measurement, cleaning and decontamination decision-making, and logical judgment of exceeding the warning threshold through visual AI algorithms, and automatically issues cleaning and decontamination plan commands to the intelligent drive and control system. The intelligent drive and control system receives the control commands from the intelligent decision-making platform and controls the cleaning machine to perform cleaning and decontamination operations.

[0007] The intelligent decision-making platform is based on sensing intelligence technology. It uses optical non-contact sensing devices to automatically receive a large number of water scene images, process them, and perform intelligent analysis to obtain information to control machines or processes. It also makes full use of the Internet, 5G, or Internet of Things to build an intelligent analysis and decision-making platform for early warning, treatment, and tracking of water pollution.

[0008] The intelligent drive and control system communicates with the orifice intelligent cleaning and control system via Ethernet, with the cleaning site centralized control system via Ethernet, with the cleaning site centralized control system via Zigbee, and with the environmental sensing lower-level system via Zigbee.

[0009] The image processing of the intelligent decision-making platform includes the following steps:

[0010] S1: The intelligent decision-making platform connects to the camera at the orifice terminal to acquire video streams. It identifies floating debris on the water surface using a floating debris recognition algorithm and sends an alarm to relevant personnel. The floating debris detection algorithm uses a Yolov series deep learning neural network. The DarkNet_53 network extracts features of floating debris through convolution operations and selects feature maps that have been sampled at 32, 16, and 8 times to construct prediction branches for target detection.

[0011] S2: The intelligent decision-making platform calculates the lateral and longitudinal distances of the floating debris from the orifice using image-based measurement technology based on captured images of floating debris, and intelligently determines the relative movement direction and distance of the cleaning machine. Distance measurement uses a lidar system, with the lidar parallel to the camera's optical axis at a distance of b. The laser emitted by the lidar is calibrated with the image, and the center is recorded as P. x l ， y l The measured distance is H, and the camera focal length is f; according to the camera imaging principle, the coordinates of the center of the obtained image plane are q( 0, 0 The corresponding sides of similar triangles are proportional. Therefore, the length corresponding to each pixel is The target spatial location can then be represented as

[0012]

[0013] S3: The intelligent decision-making platform uses image-based measurement technology to calculate the size of the target detection area as the area of ​​floating garbage based on the captured images of floating garbage. It then determines whether the area of ​​floating garbage exceeds the preset area threshold and issues an alarm if the area exceeds the threshold.

[0014] The intelligent drive and control system also includes: an online intelligent sensing terminal device for the water level at the dam orifice, mounted on the wall at one end of the dam orifice; an online intelligent sensing terminal device for ambient light intensity, mounted on the border of the dam orifice cleaning operation area; an online sensing array terminal device for the grab bucket gate position, mounted on one side of the horizontal travel track of the cleaning machine, located at the start and end points of the operation area of ​​each dam orifice; an online intelligent sensing terminal device for the grab bucket track position, mounted inside the grab bucket motion transmission box of the cleaning machine, with two laser displacement sensors collinearly mounted at opposite ends of the travel track; an online intelligent sensing terminal device for the grab bucket lifting height, mounted inside the grab bucket motion transmission box of the cleaning machine, measuring the lifting height of the grab bucket in the vertical direction; and an online intelligent sensing terminal device for the grab bucket working load, mounted on the slings of the grab bucket, monitoring the working load of the grab bucket.

[0015] The beneficial effects of this invention are:

[0016] This invention is an intelligent control system for a trash rack cleaning machine based on visual AI recognition algorithms. Compared with existing technologies, this invention fully utilizes modern information technology, employing advanced visual AI technology and computer processing capabilities, combined with current operation and maintenance management models and the existing status of the trash rack cleaning machine control system at the water intake, to establish an intelligent trash rack cleaning decision and control system based on visual AI. This system enables intelligent identification, automatic cleaning, and centralized remote semi-automatic cleaning operations of floating debris at the trash rack intake, thereby improving the intelligence level of power plant equipment, meeting the needs of the remote control center for monitoring and cleaning floating debris on the water surface of the Liuping sluice gate intake trash rack, and simultaneously reducing the workload of manual operations during the main flood season. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the overall system architecture of the present invention;

[0018] Figure 2 This is a schematic diagram of the decision-making platform architecture of the present invention;

[0019] Figure 3 This is a schematic diagram of the DarkNet_53 network of the present invention;

[0020] Figure 4 This is a flowchart of the garbage floating object detection algorithm of the present invention;

[0021] Figure 5 This is a schematic diagram of spatial position calculation according to the present invention;

[0022] Figure 6 This is a schematic diagram of the intelligent drive and control system of the present invention. Detailed Implementation

[0023] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. The illustrative embodiments and descriptions of the present invention are used to explain the present invention, but do not limit the present invention.

[0024] The overall system architecture includes an intelligent decision-making platform and an intelligent drive and control system. The intelligent decision-making platform is responsible for the system algorithm, performs logic judgments such as cleaning and inspection, shape and position recognition, volume measurement, cleaning decision-making, and exceeding the warning threshold through visual AI algorithms, and automatically issues cleaning plan commands; controls the cleaning machine to perform cleaning operations through the intelligent drive and control system (an existing PLC control system can be reused as part of the system). During the process, abnormal faults can be automatically judged, alarms and shutdowns can be linked in a timely manner, the cleaning process can be completed automatically, or it can be monitored by remote centralized control personnel throughout the process; at the same time, the on-site manual operation function is reserved for abnormal handling. The principle of the overall system architecture is as Figure 1 。

[0025] Deployment method:

[0026] Cloud computing (private cloud): The recognition algorithm can be deployed on the cloud service platform of the centralized control center, access the monitoring cameras of the trash rack openings of each power station (temporarily consider the cleaning machine at the Liuping lock head in the first phase), and transmit the real-time recorded video to the intelligent cleaning control system and cloud storage platform of the opening deployed on the cloud according to the RTSP stream and the video multicast or unicast protocol.

[0027] Edge computing: Use an edge intelligent terminal for edge computing for recognition. The edge intelligent terminal has edge computing capabilities and can process information such as videos collected by the information collection terminal in real time through the recognition algorithm built into the edge intelligent terminal. The edge computing terminal can be deployed to the opening industrial control computer to achieve linkage with the cleaning machine operation.

[0028] Cloud-edge collaboration: The cloud-edge collaboration method supports public cloud and private cloud deployments. The micro servers or edge intelligent terminals at the edge quickly execute simple scene recognition, and customized recognition services for complex scenes and multiple cameras are performed on the cloud. The video and data are finally transmitted to the intelligent cleaning control system of the opening on the cloud server.

[0029] According to the current "few people on duty" operation and maintenance management mode and equipment requirements, the cloud computing deployment method is preferred, that is, the intelligent decision-making platform is installed in the centralized control center. Access the monitoring cameras of the trash rack openings of each power station (temporarily consider the cleaning machine at the Liuping lock head in the first phase), and transmit the real-time recorded video to the intelligent cleaning control system and cloud storage platform of the opening deployed on the cloud according to the RTSP stream and the video multicast or unicast protocol.

[0030] The intelligent decision-making platform, centered on sensing intelligence technology, uses optical non-contact sensing devices (cameras) to automatically receive and process large amounts of images of water scenes, thereby obtaining information to control machines or processes. It also fully utilizes technologies such as the Internet, 5G, and the Internet of Things to build an intelligent analysis and decision-making platform, enabling early warning, treatment, and tracking of water pollution. Its architectural principles are as follows... Figure 2 .

[0031] (1) Floating garbage detection and capture: Video stream data is obtained by deploying monitoring equipment through the orifice to realize real-time detection and capture of floating garbage on the water surface. The intelligent agent actively identifies and captures the garbage for archiving.

[0032] (2) Map Measurement and Detection: Using map measurement and detection technology, the location of detected floating garbage and the area of ​​floating garbage are calculated, which helps to calculate the relative movement direction and distance of the cleaning machine.

[0033] (3) Alarm for abnormal situations: The function of timely alarm is implemented for abnormal situations such as the detection of floating garbage and the floating garbage area exceeding the threshold.

[0034] The intelligent drive and control system communicates with the orifice intelligent cleaning and decontamination control system via Ethernet, with the on-site cleaning and decontamination control system (industrial control computer) via Ethernet, with the on-site cleaning and decontamination control system (industrial control computer) via Zigbee, and with the environmental sensing lower-level system (embedded microcontroller) via Zigbee. The deployment scheme is as follows: Figure 3 ;

[0035] The intelligent decision-making platform is mainly divided into three modules: image algorithm module, abnormal situation alarm module, and algorithm configuration module. First, the image algorithm identifies floating debris on the water surface and alerts relevant personnel. Then, the location and area of ​​the floating debris are confirmed by on-map measurement and calculation, providing guidance for the cleaning machine. At the same time, the platform supports manual adjustment of algorithm configuration.

[0036] (1) Image Algorithm Module

[0037] A floating debris identification algorithm

[0038] The platform connects to the camera at the orifice to acquire video streams, identifies floating debris on the water surface using a floating debris recognition algorithm, and sends an alarm to the relevant personnel.

[0039] The floating debris detection algorithm uses a YOLOv series deep learning neural network, such as DarkNet_53. Figure 3As shown, the Darknet_53 network extracts features of floating debris through convolution operations, and selects feature maps sampled at 32, 16, and 8 times to construct prediction branches for target detection. The overall system framework of the algorithm is as follows: Figure 4 As shown;

[0040] 1. Based on the application scenario of Liuping sluice gate, complete the collection of basic data and functional requirements;

[0041] 2. Based on the data and functional requirements, develop visual AI recognition software and verify two algorithms: one for identifying the type of scum and the other for identifying the area of ​​scum. After 3-6 months of training, determine the optimal prototype algorithm based on accuracy and efficiency, which will serve as the data support for the final development and application of the system.

[0042] The floating debris detection algorithm uses a YOLOv series deep learning neural network, such as DarkNet_53. Figure 3 As shown, the Darknet_53 network extracts features of floating debris through convolution operations, and selects feature maps sampled at 32, 16, and 8 times to construct prediction branches for target detection. The overall system framework of the algorithm is as follows: Figure 4 As shown;

[0043] In this algorithm, k-means is mainly used to perform cluster analysis on the dataset to generate prior boxes, which are used in the three prediction branches. In order to further extract the target location points, CAM networks are added to the three prediction branches respectively. After the network training is completed, the target pixel weights corresponding to the localization boxes in each scale are extracted and added to the floating object detection weight model. In this way, during detection, the floating object test set is used to replace the localization box coordinate information with pixels through the weight model and drawn on the image to realize the localization of the target on the image.

[0044] b. Calculation of floating debris location

[0045] Based on images of floating debris, the platform uses image-based measurement technology to calculate the horizontal and vertical distances of the floating debris from the orifice, and intelligently determines the relative movement direction and distance of the cleaning machine.

[0046] In this technical module, distance measurement uses a lidar module, which is parallel to the camera's optical axis at a distance of b. The laser emitted by the lidar module is calibrated with the image, and the center is denoted as P. x l ， y l The measured distance is H, and the camera focal length is f. Based on the camera imaging principle, such as... Figure 5 As shown, the coordinates of the center of the obtained image plane are q( 0,0 The corresponding sides of similar triangles are proportional. Therefore, the length corresponding to each pixel is The target spatial location can then be represented as

[0047]

[0048] c. Calculation of floating debris area

[0049] The platform uses image-based measurement technology to calculate the size of the target detection area as the area of ​​the floating debris based on the captured images. It then determines whether the area of ​​the floating debris exceeds a preset area threshold and issues an alarm if the area exceeds the threshold.

[0050] Meanwhile, for cases where the area exceeds the threshold, the platform automatically calculates the number of cleaning cycles and the actual locations of multiple cleaning cycles to ensure the comprehensiveness of the cleaning process.

[0051] (2) Abnormal situation alarm module

[0052] The platform intelligently identifies abnormal situations on the water surface and pushes them to relevant staff via pop-up windows or text messages, reminding them to confirm and take action.

[0053] Supports manual setting and modification of alarm thresholds

[0054] (3) Algorithm configuration module

[0055] a Identification frequency configuration

[0056] The system can be configured to identify the frequency of the algorithm and adjust the interval for identifying floating debris according to actual needs, adjusting the configuration in the most efficient way.

[0057] b. Remote centralized configuration of computing power

[0058] The system can remotely and centrally configure computing power on the platform according to actual needs (on-site testing).

[0059] The intelligent drive and control system (industrial control computer) is installed in a 400V industrial TV cabinet and communicates with the orifice intelligent cleaning and control system via Ethernet.

[0060] The track-driven control lower-level system (PLC-1, formerly Siemens 200 PLC) is installed in the original cleaning machine control cabinet and communicates with the cleaning site centralized control system (industrial control computer) via Ethernet.

[0061] The grab bucket drive control lower-level system (PLC-2, a newly added PLC system) is installed in the original grab bucket motion transmission box of the cleaning machine, and communicates with the cleaning site centralized control system (industrial control computer) via Zigbee.

[0062] The environmental sensing lower-level system (embedded microcontroller) is deployed at the environmental sensing point and communicates with the on-site cleaning and pollution control system (industrial control computer) via Zigbee.

[0063] Layout plan diagram as follows Figure 6 As shown, the intelligent drive and control system communicates with the intelligent cleaning and control system at the orifice via Ethernet, communicates with the on-site cleaning and control system (industrial control computer) via Ethernet, communicates with the on-site cleaning and control system (industrial control computer) via Zigbee, and communicates with the environmental sensing lower-level system (embedded microcontroller) via Zigbee.

[0064] Dam orifice water level sensing:

[0065] The online intelligent sensing terminal device for water level at the dam orifice is installed on the wall next to the orifice at one end of the dam, which facilitates the installation and detection of the submersible water level sensor. It is connected to the on-site control system (industrial control computer) via Zigbee for host-response communication.

[0066] Ambient light intensity sensing:

[0067] The online intelligent sensing terminal device for ambient light intensity is deployed on the edge of the cleaning operation area at the dam orifice, facilitating the installation and detection of light intensity sensors. It communicates with the on-site centralized control system (industrial control computer) via Zigbee in a host-response communication manner.

[0068] Grab bucket gate hole position sensing:

[0069] The grab bucket gate position online sensing array terminal device is deployed on one side of the horizontal travel track of the cleaning machine, located at the start and end points of the working area of ​​each gate of the dam. It facilitates the sensing of the gate position area of ​​the cleaning machine grab bucket motion transmission box and grab bucket, and is connected to the track drive control lower system (PLC-1) via DIO.

[0070] Grab track position sensing:

[0071] The online intelligent sensing terminal device for the grab bucket track position is installed inside the grab bucket motion transmission box of the cleaning machine. Two laser displacement sensors are arranged collinearly along the opposite ends of the travel track to facilitate the measurement of the horizontal position of the grab bucket motion transmission box and the grab bucket. It communicates with the grab bucket drive control lower system (PLC-2) via a serial port.

[0072] Grab bucket lifting height sensor:

[0073] The online intelligent sensing terminal device for the lifting height of the grab bucket is installed inside the motion transmission box of the grab bucket of the cleaning machine. It measures the lifting height of the grab bucket in the vertical direction and communicates with the grab bucket drive control lower system (PLC-2) via RS485.

[0074] Grab bucket working condition load sensing:

[0075] The online intelligent sensing terminal device for the working load of the grab bucket is installed on the sling of the grab bucket of the cleaning machine to monitor the working load of the grab bucket. It communicates with the grab bucket drive and control lower system (PLC-2) via RS485 in a question-and-answer manner.

[0076] The technical solutions of the present invention are not limited to the specific embodiments described above. Any technical modifications made in accordance with the technical solutions of the present invention fall within the protection scope of the present invention.

Claims

1. A visual AI recognition algorithm-based intelligent control system for a trash cleaning machine, characterized in that: The intelligent decision platform and the intelligent driving control system are included, the intelligent decision platform carries out pollution inspection, shape and position identification, accumulation measurement, pollution decision and logic judgment of exceeding the early warning threshold through visual AI algorithm, and automatically sends the pollution cleaning plan command to the intelligent driving control system, and the intelligent driving control system receives the control command of the intelligent decision platform to control the pollution cleaning machine to carry out pollution cleaning operation; The intelligent decision platform takes the perception intelligent technology as the core, uses the optical non-contact sensing device to automatically receive a large number of water scene image processing and intelligent analysis to obtain information control machine or process, and fully utilizes the Internet, 5G or Internet of Things to build an intelligent analysis decision platform to carry out early warning, processing and tracking of water pollution; The image processing of the intelligent decision platform includes the following steps: S1: the intelligent decision platform is connected with the terminal camera of the orifice, video stream is obtained, floating garbage appearing on the water surface is identified through a floating garbage identification algorithm and alarm feedback is fed back to the relevant staff; the floating garbage detection algorithm adopts a deep learning neural network of the Yolov series, a DarkNet_53 network extracts garbage floating features through convolution operation, and a feature map selected through 32, 16 and 8 times sampling is used to construct a prediction branch for target detection; S2: The intelligent decision-making platform calculates the horizontal and vertical distance of the floating garbage from the orifice according to the floating garbage snapshot image, calculates the relative moving direction and distance of the garbage cleaning machine by using the on-image measurement technology; the distance measurement adopts a laser radar, the laser radar is parallel to the camera optical axis, the distance between them is b, the laser emitted by the laser radar will pass through the image after calibration, and the center is P( x l， y l ), the measured distance is H, and the focal length of the camera is f; according to the camera imaging principle, the center coordinates of the image plane are q( 0 , 0 ), which satisfies the proportional relationship of the corresponding sides of the similar triangle, Therefore, the length corresponding to each pixel is Therefore, the target space position can be represented as ; S3: the intelligent decision platform uses the image on the map measurement technology to regard the size of the target detection area as the area of the floating garbage, judges whether the area of the floating garbage exceeds the threshold according to the preset area threshold, and alarms for the case of exceeding the threshold.

2. The intelligent control system of the visual AI recognition algorithm-based silt remover according to claim 1, characterized in that: The intelligent driving control system is connected with the orifice intelligent pollution control system through Ethernet, connected with the pollution site centralized control system through Ethernet, connected with the pollution site centralized control system through Zigbee, and connected with the environment sensing lower system through Zigbee.

3. The intelligent control system of the visual AI recognition algorithm-based silt remover according to claim 2, characterized in that: The intelligent driving control system is further provided with a dam orifice water level online intelligent sensing terminal device arranged on the vertical wall at one end of the dam orifice, and an environmental illumination intensity online sensing terminal device arranged on the frame of the dam orifice pollution cleaning operation area; A grab bucket gate position online sensing array terminal device is arranged on one side of the horizontal travel track of the pollution cleaning machine and located at the starting point and the ending point of the dam orifice operation area of each hole; A grab bucket track position online intelligent sensing terminal device is arranged in the grab bucket movement transmission box, and two laser displacement sensors are arranged in line along the opposite ends of the travel track; A grab bucket lifting height online intelligent sensing terminal device is arranged in the grab bucket movement transmission box and measures the lifting height of the grab bucket in the vertical direction; A grab bucket working condition load online intelligent sensing terminal device is arranged on the sling of the pollution cleaning machine and monitors the working condition load of the grab bucket.

Images