Intelligent slag transportation monitoring method and system applied to hydropower station construction project

Abstract

The application provides an intelligent monitoring method and system for a hydropower station construction project, which obtains video monitoring data streams of a slag transportation vehicle in a construction area, and the video monitoring data streams include a plurality of slag transportation vehicle monitoring image frame units arranged according to a collection time sequence. The slag transportation vehicle feature analysis is performed on the image frame units to obtain a loading state feature and a vehicle body space position feature. The loading state change trajectory and the space movement trajectory are generated according to the loading state feature and the vehicle body space position feature of the continuous image frame units, the trajectory synchronous mapping is performed, the corresponding relationship is obtained, and the loading change record is obtained. The loading change record is used to determine a loading point and an unloading point position set, and a slag transportation monitoring report is compiled. The application realizes the intelligent monitoring of the whole process and all aspects of the slag transportation operation, and effectively improves the fine management level of the slag transportation monitoring of the hydropower station construction project.

Description

Technical Field

[0001] This invention relates to the field of slag transport monitoring, and more specifically, to an intelligent slag transport monitoring method and system applied to hydropower station construction projects. Background Technology

[0002] Traditional methods of monitoring waste transportation mainly rely on manual on-site inspections and recordings, which have many drawbacks. Firstly, manual inspections cannot cover all waste transportation vehicles and work sites throughout the construction area, easily creating blind spots. This leads to the failure to detect irregularities in some waste transportation vehicles, such as illegal loading and indiscriminate dumping of waste, which not only damages the construction site environment but may also affect project progress and quality. Secondly, the accuracy of manually recorded data is greatly affected by the subjective factors of the recorder, and the data processing and analysis process is cumbersome, making it difficult to reflect the actual situation of waste transportation operations in a real-time and comprehensive manner, and failing to provide timely and effective decision-making basis for project management. With the development of video surveillance technology, some projects have begun to use video surveillance to assist in waste transportation monitoring. However, most existing video surveillance methods simply record images of waste transportation vehicles, lacking the ability to deeply analyze and process video data. They cannot accurately extract key information such as the loading status and spatial location of waste transportation vehicles from the video, and it is even more difficult to establish a correlation between changes in loading status and spatial location migration, making it difficult to achieve refined and intelligent monitoring of waste transportation operations. Summary of the Invention

[0003] In view of the aforementioned problems, and in conjunction with the first aspect of the present invention, embodiments of the present invention provide an intelligent slag removal monitoring method applied to hydropower station construction projects, the method comprising: Acquire video monitoring data streams of muck-hauling vehicles within the construction area of ​​a hydropower station project. The video monitoring data streams contain multiple muck-hauling vehicle monitoring image frame units arranged in the acquisition sequence. Perform muck vehicle feature parsing on the muck vehicle monitoring image frame unit to obtain the loading status features and vehicle body spatial position features of the muck vehicle in each muck vehicle monitoring image frame unit; Based on the loading status features and vehicle spatial position features corresponding to multiple consecutively arranged monitoring image frames of slag transport vehicles, an association mapping operation between loading changes and position migration is performed to generate the loading status change trajectory and spatial movement trajectory of the slag transport vehicles. Perform trajectory synchronization mapping operation on the loading state change trajectory and the spatial movement trajectory to establish the correspondence between the loading state change nodes and the spatial movement trajectory segments, and obtain the loading change records of the muck truck at each spatial location in the construction area. Based on the loading change records, determine the set of loading points and unloading points of the muck trucks in the construction area, and compile a muck truck monitoring report based on the spatial distance relationship between the loading point set and the unloading point set and the duration parameter of the loading status change trajectory.

[0004] In another aspect, embodiments of the present invention also provide an intelligent slag removal monitoring system for hydropower station construction projects, including a processor and a machine-readable storage medium. The machine-readable storage medium is connected to the processor, the machine-readable storage medium is used to store programs, instructions or code, and the processor is used to execute the programs, instructions or code in the machine-readable storage medium to implement the above-described method.

[0005] Based on the above, this invention acquires video monitoring data streams of muck-hauling vehicles within the construction area and performs in-depth analysis to obtain the loading status features and vehicle spatial position features of each monitoring image frame unit. On this basis, an association mapping operation is performed based on these features of multiple consecutively arranged image frame units to generate the loading status change trajectory and spatial movement trajectory of the muck-hauling vehicles, achieving dynamic tracking of the loading status and spatial position changes of the muck-hauling vehicles. Furthermore, through trajectory synchronization mapping, a correspondence is established between loading status change nodes and spatial movement trajectory segments, obtaining loading change records of muck-hauling vehicles at various spatial positions within the construction area. Based on these loading change records, a set of loading point locations and a set of unloading point locations are determined, and a muck-hauling monitoring report is compiled based on spatial distance relationships and the duration parameters of the loading status change trajectory. This invention achieves intelligent monitoring of the entire muck-hauling operation process in hydropower station construction projects, from data acquisition and analysis to report generation, effectively improving the precision and intelligence level of muck-hauling monitoring. Attached Figure Description

[0006] Figure 1 This is a schematic diagram of the execution flow of the intelligent slag removal monitoring method for hydropower station construction projects provided in this embodiment of the invention.

[0007] Figure 2 This is a schematic diagram of exemplary hardware and software components of an intelligent slag removal monitoring system for hydropower station construction projects provided in an embodiment of the present invention. Detailed Implementation

[0008] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 This is a flowchart illustrating an intelligent slag removal monitoring method for hydropower station construction projects, provided by an embodiment of the present invention. The following is a detailed description of this intelligent slag removal monitoring method for hydropower station construction projects.

[0009] Step S110: Obtain video monitoring data stream of muck-carrying vehicles within the construction area of ​​the hydropower station project. The video monitoring data stream contains multiple muck-carrying vehicle monitoring image frame units arranged in the acquisition sequence.

[0010] In this embodiment, to achieve effective monitoring of muck-hauling vehicles within the construction area of ​​the hydropower station project, it is first necessary to acquire relevant video monitoring data streams. These data streams originate from multiple monitoring devices deployed within the construction area. These devices are distributed at key locations within the construction area according to certain deployment principles, such as road intersections, muck loading areas, and muck unloading areas, to ensure comprehensive coverage of the muck-hauling vehicles' travel paths. The acquired video monitoring data stream is a continuous sequence containing multiple muck-hauling vehicle monitoring image frames collected in chronological order. Each image frame is a static image recording the status information of the muck-hauling vehicle at a specific moment.

[0011] Step S120: Perform muck vehicle feature parsing on the muck vehicle monitoring image frame unit to obtain the loading status features and vehicle spatial position features of the muck vehicle in each muck vehicle monitoring image frame unit.

[0012] After acquiring the video monitoring data stream, feature analysis needs to be performed on each monitoring image frame unit of the muck truck to extract key information about the muck truck. This process involves various image processing and analysis techniques, aiming to accurately identify the muck truck from the image and obtain features related to its loading status and spatial location.

[0013] Step S121: Collect raw video stream data output from multiple fixed monitoring points within the construction area of ​​the hydropower station project. Perform frame sequence decomposition operation on the raw video stream data according to the preset frame extraction interval to obtain multiple monitoring image frame units of muck trucks arranged in the acquisition time sequence. Each monitoring image frame unit of muck trucks is associated with a time identifier and a spatial identifier.

[0014] Specifically, raw video stream data is first collected from multiple fixed monitoring points within the construction area. The selection of these monitoring points takes into account factors such as the terrain of the construction area and vehicle routes to ensure the integrity and validity of the raw video stream data. The collected raw video stream data is a continuous dynamic image, requiring frame sequence decomposition at preset frame extraction intervals. The frame extraction interval can be set according to actual monitoring needs; for example, it can be set to extract one frame at regular intervals or to extract uniformly based on the video's frame rate. Through frame sequence decomposition, the raw video stream data is converted into a series of discrete monitoring image frame units for muck trucks. Each image frame unit is associated with a unique time identifier and a spatial identifier. The time identifier indicates the acquisition time of the image frame, and the spatial identifier indicates which monitoring point acquired the image frame.

[0015] Step S122: Input the muck truck monitoring image frame unit into the muck truck detection network. The muck truck detection network performs a candidate region generation operation on the muck truck monitoring image frame unit to obtain multiple candidate vehicle region boxes. Perform a vehicle category confidence calculation operation on each candidate vehicle region box and select the candidate vehicle region boxes whose category confidence exceeds the judgment threshold as the muck truck detection region boxes.

[0016] After obtaining the image frame units of the muck truck monitoring system, they are input into a pre-trained muck truck detection network. This network, built on deep learning technology, can automatically identify muck trucks in images. First, the network performs candidate region generation on the input image frame units. Candidate region generation involves sliding windows of different sizes and scales across the image to initially determine areas that may contain muck trucks. These generated candidate regions are represented as candidate vehicle bounding boxes, each with its position coordinates and size information in the image.

[0017] Next, a vehicle category confidence score is calculated for each candidate vehicle region box. Category confidence score refers to the probability that the candidate region box contains a muck-hauling vehicle. The muck-hauling vehicle detection network analyzes and judges the image features within the candidate region box to calculate the corresponding category confidence score. Then, the calculated category confidence score is compared with a preset decision threshold. When the category confidence score of a candidate vehicle region box exceeds the decision threshold, it is considered that the region box does indeed contain a muck-hauling vehicle, and it is selected as the muck-hauling vehicle detection region box. The decision threshold setting needs to comprehensively consider the detection accuracy and recall rate to ensure that as many muck-hauling vehicles as possible are detected while minimizing false detections.

[0018] Step S123: Extract the pixel feature matrix within the detection area box of the slag transport vehicle, and perform a loading area segmentation operation on the pixel feature matrix to obtain the cargo box area sub-image and the front area sub-image of the slag transport vehicle.

[0019] After determining the detection area bounding box for the dump truck, it is necessary to further extract the pixel feature matrix within this bounding box. The pixel feature matrix contains information such as the color and brightness of each pixel within the bounding box, and is the basis for subsequent loading area segmentation. Then, the loading area segmentation operation is performed on the pixel feature matrix. The purpose of loading area segmentation is to distinguish between the cargo box area and the cab area of ​​the dump truck so that their features can be analyzed separately.

[0020] Loading area segmentation can be performed using image segmentation algorithms, such as semantic segmentation. This algorithm analyzes the pixel feature matrix and segments the image into different regions based on the differences in pixel features. In images of dump trucks, the cargo box region and the cab region differ in color, texture, and shape. Semantic segmentation algorithms can utilize these differences to separate them, resulting in cargo box region sub-images and cab region sub-images. The cargo box region sub-image mainly contains the part of the dump truck loaded with goods, while the cab region sub-image contains parts such as the driver's cab.

[0021] Step S124: Perform a loading identification operation on the cargo compartment area sub-image to generate a loading occupancy ratio parameter and a loading surface texture feature vector for the cargo compartment area, and combine the loading occupancy ratio parameter and the loading surface texture feature vector into a loading status feature.

[0022] After obtaining the cargo compartment sub-map, a load identification operation needs to be performed to obtain the loading status information of the muck truck. The load identification operation includes generating a load occupancy ratio parameter for the cargo compartment area and a load surface texture feature vector. The load occupancy ratio parameter describes the amount of load in the cargo compartment, while the load surface texture feature vector describes the texture features of the load surface. This information is crucial for determining the type and status of the load. Finally, these two parameters are combined to form the loading status characteristics of the muck truck.

[0023] Step S1241: Perform edge detection operation on the cargo compartment area sub-image to extract the upper edge outline of the cargo compartment and the pixel distribution map of the interior area of ​​the cargo compartment in the cargo compartment area sub-image.

[0024] First, edge detection is performed on the cargo compartment sub-image. Edge detection is a fundamental technique in image processing used to identify the edges of objects in an image. Using an edge detection algorithm, the upper edge contour of the cargo compartment in the sub-image can be detected. The upper edge contour of the cargo compartment is an important feature, defining the opening range of the compartment. Simultaneously, a pixel distribution map of the interior area of ​​the cargo compartment is obtained, showing the distribution of each pixel within the compartment.

[0025] Step S1242: Determine the coordinates of the upper boundary of the cargo compartment opening based on the upper edge contour line of the cargo compartment, identify the distribution range of the surface pixels of the loaded object based on the pixel distribution map of the interior region of the cargo compartment, and calculate the vertical distance difference parameter between the vertex coordinates of the surface pixels of the loaded object and the upper edge contour line of the cargo compartment.

[0026] Based on the extracted upper edge contour line of the cargo compartment, the coordinates of the upper boundary of the cargo compartment opening can be determined. The coordinates of the upper boundary of the opening are crucial for calculating the proportion occupied by the cargo. Then, based on the pixel distribution map of the interior region of the cargo compartment, the distribution range of pixels on the cargo surface is identified. The distribution range of pixels on the cargo surface reflects the position and size of the cargo within the cargo compartment. Next, the difference in vertical distance between the vertex coordinates of the cargo surface pixels and the upper edge contour line of the cargo compartment is calculated. This parameter represents the vertical distance from the vertex of the cargo surface to the upper boundary of the cargo compartment opening; this distance can be used to preliminarily determine the height of the cargo.

[0027] Step S1243: Perform pixel value statistical analysis on the pixel distribution map of the cargo compartment interior area to generate mean and variance parameters of the pixel values ​​on the surface of the load. Based on the vertical distance difference parameter and the mean and variance parameters of the pixel values ​​on the surface of the load, calculate the proportion of the cargo occupying the interior space of the cargo compartment as the cargo occupancy ratio parameter.

[0028] A pixel value statistical analysis is performed on the pixel distribution map of the cargo compartment's interior area. This analysis includes calculating the mean and variance parameters of the pixel values ​​on the cargo surface. The mean parameter reflects the average level of pixel values ​​on the cargo surface, while the variance parameter reflects the dispersion of pixel values. These parameters help determine characteristics such as the color and uniformity of the cargo. Then, based on the vertical distance difference parameter combined with the mean and variance parameters, the proportion of the cargo's interior space occupied by the cargo is calculated. Specifically, the vertical distance difference parameter can be compared with the height of the cargo compartment, and the proportion can be corrected by combining the statistical parameters of the pixel values, thus obtaining a more accurate cargo occupancy ratio.

[0029] Step S1244: Perform texture feature extraction on the pixel distribution map of the interior region of the cargo compartment, calculate the gray-level difference matrix of adjacent pixels in the pixel distribution map of the interior region of the cargo compartment, perform eigenvalue decomposition on the gray-level difference matrix, and generate a cargo surface texture feature vector containing contrast feature components, correlation feature components and entropy feature components.

[0030] Texture features are important indicators for describing the surface features of an object. Texture feature extraction is performed on the pixel distribution map of the cargo compartment's interior area. First, the gray-level difference matrix of adjacent pixels in the pixel distribution map of the cargo compartment's interior area is calculated. The gray-level difference matrix reflects the gray-level differences between adjacent pixels and is the basis for extracting texture features. Then, eigenvalue decomposition is performed on the gray-level difference matrix. Eigenvalue decomposition decomposes the matrix into multiple eigenvalues ​​and eigenvectors. By analyzing these eigenvalues ​​and eigenvectors, texture features can be extracted. The generated cargo surface texture feature vector includes contrast feature components, correlation feature components, and entropy feature components. The contrast feature component reflects the clarity of the texture, the correlation feature component reflects the regularity of the texture, and the entropy feature component reflects the complexity of the texture.

[0031] Step S1245: Perform a dimension concatenation operation on the loading ratio parameter and the loading surface texture feature vector to generate a loading state feature containing numerical parameters and vector parameters.

[0032] Finally, a dimension concatenation operation is performed between the cargo occupancy ratio parameter and the cargo surface texture feature vector. The cargo occupancy ratio parameter is a numerical parameter, and the cargo surface texture feature vector is a vector parameter. Concatenating them together forms a cargo status feature that contains multiple pieces of information.

[0033] Step S125: Perform vehicle orientation recognition operation on the vehicle head area sub-image to generate vehicle head orientation angle parameters. Compile vehicle body spatial position features based on the center point coordinates of the detection area box of the slag transport vehicle and the vehicle head orientation angle parameters. The vehicle body spatial position features include center point coordinate values, orientation angle values, and size parameters of the detection area box.

[0034] A vehicle orientation recognition operation is performed on the front region sub-image to determine the driving direction of the dump truck. Vehicle orientation recognition can determine the orientation of the truck by analyzing features of the front region, such as the shape of the front and the position of the headlights. The generated front orientation angle parameter is an angle value representing the direction of the truck's orientation. Then, the vehicle's spatial position features are compiled based on the coordinates of the center point of the dump truck detection region box and the front orientation angle parameter. The vehicle's spatial position features contain information such as the dump truck's position, orientation, and size in the image, which is of great significance for analyzing the vehicle's trajectory and changes in spatial position.

[0035] Step S1251: Perform key point detection operation on the vehicle front area sub-image to extract the coordinates of the foremost point of the vehicle front, the center point of the top of the vehicle front, and the center point of the bottom of the vehicle front in the vehicle front area sub-image.

[0036] First, keypoint detection is performed on the front of the vehicle sub-image. Keypoint detection is a technique in computer vision used to identify points of significant importance in an image. In the front of the vehicle sub-image, the foremost point, the center point of the top of the front, and the center point of the bottom of the front are key feature points. The coordinates of these points can be accurately extracted using a keypoint detection algorithm. The coordinates of the foremost point indicate the position of the foremost part of the front, while the coordinates of the center points of the top and bottom of the front reflect the height and shape information of the front.

[0037] Step S1252: Determine the initial direction vector of the vehicle's orientation based on the direction of the line connecting the coordinates of the foremost point of the vehicle's front end and the center point of the top of the vehicle's front end. Perform a correction operation on the initial direction vector based on the direction of the line connecting the coordinates of the foremost point of the vehicle's front end and the center point of the bottom of the vehicle's front end to generate the vehicle's orientation direction vector.

[0038] Based on the extracted coordinates of the foremost point of the vehicle's front and the center point of the top of the vehicle's front, a straight line is drawn connecting these two points. The direction of this straight line is the initial direction vector indicating the vehicle's orientation. However, the initial direction vector may contain some errors and needs to be corrected. The correction is based on the direction of the line connecting the coordinates of the foremost point of the vehicle's front and the center point of the bottom of the vehicle's front. By comparing the initial direction vector and the corrected direction vector, the initial direction vector can be adjusted to generate a more accurate vehicle orientation vector.

[0039] Step S1253: Calculate the angle between the vehicle's facing direction vector and the positive direction of the horizontal axis in the pre-established construction area coordinate system, and use the angle as the vehicle's facing angle parameter.

[0040] Within the construction area, a coordinate system is pre-established to describe the spatial position of objects. The angle between the vehicle's heading direction vector and the positive direction of the horizontal axis in this coordinate system is calculated. This angle is the vehicle's heading angle parameter, representing the deflection angle of the vehicle's head relative to the horizontal direction. The heading angle parameter determines the direction of travel for the muck-hauling vehicle.

[0041] Step S1254: Extract the center point coordinates from the detection area frame of the slag transport vehicle, and extract the width and height parameters of the area frame from the detection area frame as the size parameters of the detection area frame.

[0042] The detection region bounding box for the muck truck is a rectangle, and its center point coordinates represent the center position of the muck truck in the image. The x and y coordinates of the center point can be extracted from the detection region bounding box. Simultaneously, the width and height parameters of the region bounding box can also be extracted; these two parameters describe the size of the detection region bounding box, that is, the area occupied by the muck truck in the image.

[0043] Step S1255: The center point coordinates, vehicle front orientation angle parameters, and detection area box size parameters are encapsulated according to a preset data structure to generate vehicle spatial position features containing numerical coordinate components, angle components, and size components.

[0044] The center point coordinates, vehicle heading angle, and detection area bounding box dimensions are encapsulated according to a predefined data structure. This predefined data structure can be a struct or an array, organizing these parameters to form a complete spatial position feature of the vehicle. This feature includes numerical coordinate, angle, and size components, comprehensively describing the spatial position and status of the dump truck in the image.

[0045] Step S130: Based on the loading status features and vehicle spatial position features corresponding to the continuously arranged multiple monitoring image frame units of slag transport vehicles, perform the association mapping operation of loading changes and position migration to generate the loading status change trajectory and spatial movement trajectory of the slag transport vehicles.

[0046] After obtaining the loading status characteristics and vehicle spatial position characteristics of each monitoring image frame unit of the muck truck, it is necessary to analyze multiple consecutively arranged image frame units and perform a correlation mapping operation between loading changes and position migration. The purpose of this operation is to link the loading status changes and position migration of the muck truck, generating a loading status change trajectory and a spatial movement trajectory. The loading status change trajectory reflects the changes in the loading status of the muck truck at different times, while the spatial movement trajectory reflects the driving path of the muck truck within the construction area.

[0047] Step S131: Extract the loading status feature sequence and vehicle body spatial position feature sequence corresponding to the same muck transport vehicle according to the arrangement order of the time identifiers of the monitoring image frame units of the muck transport vehicle. Each loading status feature in the loading status feature sequence has a time correspondence with the vehicle body spatial position feature with the same time identifier in the vehicle body spatial position feature sequence.

[0048] First, based on the time identifier of the image frame units of the muck truck monitoring images, the image frame units belonging to the same muck truck are arranged in chronological order. Then, the corresponding loading status features and vehicle spatial position features are extracted from these arranged image frame units to form a loading status feature sequence and a vehicle spatial position feature sequence. Since each image frame unit has a unique time identifier, each loading status feature in the loading status feature sequence has a temporal correspondence with the vehicle spatial position feature of the same time identifier in the vehicle spatial position feature sequence; that is, they are features of the muck truck acquired at the same time.

[0049] Step S1311: Extract the unique identifier of the muck truck from the detection area box of the muck truck, and based on the unique identifier, filter out the loading status features and vehicle spatial position features belonging to the same muck truck from the muck truck feature analysis results of all muck truck monitoring image frame units.

[0050] To accurately extract the feature sequence of the same muck truck, it is necessary to extract the unique identifier of the muck truck from the detection area bounding box. The unique identifier can be the vehicle's license plate number, vehicle identification number, etc., and it can uniquely identify a muck truck. Based on the unique identifier, the loading status features and vehicle spatial position features belonging to the same muck truck are selected from the muck truck feature analysis results of all muck truck monitoring image frame units.

[0051] Step S1312: Sort the selected loading status features according to the time identifiers of the monitoring image frame units of the slag transport vehicle to generate a loading status feature sequence. Sort the selected vehicle body spatial position features according to the same time order to generate a vehicle body spatial position feature sequence.

[0052] After identifying the loading status and spatial position features of the same muck-hauling vehicle, these features are sorted according to the time identifiers of the monitoring image frame units. A sorting operation is performed on the loading status features to generate a loading status feature sequence; a sorting operation is performed on the vehicle spatial position features to generate a vehicle spatial position feature sequence. The sorted feature sequences, arranged in chronological order, clearly reflect the feature changes of the muck-hauling vehicle at different times.

[0053] Step S132: Perform a loading state difference calculation operation on two loading state features with adjacent time identifiers in the loading state feature sequence, identify the feature point positions in the loading state feature sequence where the loading occupancy ratio parameter changes, and divide the loading state feature sequence into a loading volume stable stage sequence and a loading volume change stage sequence based on the identified feature point positions.

[0054] The loading status feature sequence is compared between two loading status features with adjacent time identifiers, and a loading status difference calculation is performed. This calculation primarily compares the load occupancy ratio parameter between the two features to determine if the loading status has changed. By calculating the difference between two adjacent load occupancy ratio parameters, the locations of significant changes in this parameter can be identified. Based on these locations, the loading status feature sequence can be divided into a stable loading phase sequence and a loading change phase sequence. The load occupancy ratio parameter changes less in the stable loading phase sequence and more significantly in the loading change phase sequence.

[0055] Step S1321: Traverse the two loading status features of each pair of adjacent time identifiers in the loading status feature sequence, extract the loading occupancy ratio parameter of the first loading status feature and the loading occupancy ratio parameter of the second loading status feature, and calculate the absolute value of the difference between the two loading occupancy ratio parameters.

[0056] The loading status feature sequence is iterated through for each pair of adjacent time identifiers, containing two loading status features. For each pair, the loading occupancy ratio parameters for the first and second features are extracted. Then, the absolute value of the difference between these two parameters is calculated. The magnitude of the absolute difference reflects the degree of change in the loading occupancy ratio between two adjacent time points.

[0057] Step S1322: Perform a comparison operation between the absolute value of the difference and a preset change significance threshold. When the absolute value of the difference exceeds the change significance threshold, mark the position index of the second loading state feature in the loading state feature sequence as the change feature point position.

[0058] The calculated absolute value of the difference is compared with a preset threshold for the significance of change. This threshold is a pre-defined value used to determine whether the change in the proportion parameter of the loaded goods is significant. When the absolute value of the difference exceeds the threshold, it indicates a significant change in the loading status, and the position index of the second loading status feature in the loading status feature sequence is marked as the location of the change feature point. These change feature point locations are crucial for segmenting the loading status feature sequence.

[0059] Step S1323: Perform a segmentation operation on the loading state feature sequence based on the marked change feature point positions. Combine the loading state features between two adjacent change feature point positions into a loading volume stable stage sequence, and combine the loading state features corresponding to the change feature point positions into a loading volume change stage sequence. In the loading volume stable stage sequence, the fluctuation range of the loading occupancy ratio parameter of each loading state feature is less than a preset fluctuation threshold, and in the loading volume change stage sequence, the change amplitude of the loading occupancy ratio parameter of each loading state feature exceeds a preset change amplitude threshold.

[0060] Based on the marked locations of the change feature points, the loading state feature sequence is segmented. Loading state features between two adjacent change feature points are combined to form a stable loading phase sequence. In this stable phase sequence, the load occupancy ratio parameter for each loading state feature fluctuates within a small range, below a preset fluctuation threshold. Loading state features corresponding to the change feature points are then combined to form a changing loading phase sequence. In this changing phase sequence, the load occupancy ratio parameter for each loading state feature changes significantly, exceeding a preset change threshold. This segmentation operation clearly distinguishes between stable and changing stages of the loading state of the dump trucks.

[0061] Step S133: Connect the loading state features in the loading quantity stabilization stage sequence in chronological order to form a loading state change trajectory, and mark the loading object occupancy ratio parameter and loading object surface texture feature vector corresponding to each loading state feature in the loading state change trajectory.

[0062] By sequentially connecting the loading state features in the stable loading phase sequence according to time, a loading state change trajectory is formed. In this trajectory, each point corresponds to a loading state feature, and is labeled with the corresponding load occupancy ratio parameter and the load surface texture feature vector. Thus, the loading state change trajectory reflects not only the changes in the load occupancy ratio but also the changes in the load surface texture features.

[0063] Step S134: Perform spatial position migration calculation operation on two vehicle spatial position features with adjacent time identifiers in the vehicle spatial position feature sequence, generate a displacement vector based on the difference in the center point coordinates of the two vehicle spatial position features, and generate a turning angle vector based on the difference in the vehicle head orientation angle parameter of the two vehicle spatial position features.

[0064] Spatial position migration calculation is performed on two vehicle spatial position features with adjacent time identifiers in the vehicle spatial position feature sequence. First, the difference in center point coordinates between the two vehicle spatial position features is calculated, and a displacement vector is generated based on the coordinate difference. The displacement vector reflects the positional change of the muck-hauling vehicle between two adjacent time points. Then, the difference in the vehicle heading angle parameter between the two vehicle spatial position features is calculated, and a turning angle vector is generated. The turning angle vector reflects the orientation change of the muck-hauling vehicle between two adjacent time points.

[0065] Step S1341: Extract the first and second vehicle spatial position features with adjacent time identifiers from the vehicle spatial position feature sequence, extract the first center point coordinate value from the first vehicle spatial position feature, and extract the second center point coordinate value from the second vehicle spatial position feature.

[0066] Two vehicle spatial position features with adjacent time identifiers are selected from the vehicle spatial position feature sequence, referred to as the first vehicle spatial position feature and the second vehicle spatial position feature, respectively. The coordinates of the first center point, including the x-coordinate and y-coordinate, are extracted from the first vehicle spatial position feature; similarly, the coordinates of the second center point, including the x-coordinate and y-coordinate, are extracted from the second vehicle spatial position feature.

[0067] Step S1342: Calculate the difference between the abscissa component of the second center point coordinate value and the abscissa component of the first center point coordinate value; calculate the difference between the ordinate component of the second center point coordinate value and the ordinate component of the first center point coordinate value; combine the abscissa difference and the ordinate difference into a displacement vector.

[0068] Calculate the x-coordinate component of the second center point coordinate value and subtract it from the x-coordinate component of the first center point coordinate value to obtain the x-coordinate difference; calculate the y-coordinate difference by subtracting it from the y-coordinate component of the second center point coordinate value. Combine the x-coordinate difference and y-coordinate difference to form a displacement vector. The direction of the displacement vector indicates the direction of movement of the muck-carrying vehicle, and the magnitude indicates the distance traveled.

[0069] Step S1343: Extract the first vehicle front orientation angle parameter from the spatial position features of the first vehicle body, extract the second vehicle front orientation angle parameter from the spatial position features of the second vehicle body, calculate the angle difference between the second vehicle front orientation angle parameter and the first vehicle front orientation angle parameter, and use the angle difference as the turning angle vector.

[0070] The first vehicle's heading angle parameter is extracted from the spatial position features of the first vehicle body, and the second vehicle's heading angle parameter is extracted from the spatial position features of the second vehicle body. The second vehicle's heading angle parameter is subtracted from the first vehicle's heading angle parameter to obtain the angle difference, which is used as the turning angle vector. The turning angle vector represents the angle of change in orientation of the muck-hauling vehicle between two adjacent moments.

[0071] Step S135: Connect the displacement vector and rotation vector sequentially in chronological order to form a spatial movement trajectory. Mark the center point coordinates, orientation angle, and size parameters of the detection area box corresponding to each trajectory point in the spatial movement trajectory.

[0072] The calculated displacement and rotation vectors are sequentially connected according to time sequence to form the spatial movement trajectory of the muck-hauling vehicle. In this trajectory, each point corresponds to a specific spatial position of the vehicle at a given moment, and the coordinates of its center point, orientation angle, and the dimensions of the detection area are labeled. The spatial movement trajectory visually demonstrates the muck-hauling vehicle's path and status changes within the construction area.

[0073] Step S1351: Use the displacement vector as the line segment vector pointing from the first trajectory point to the second trajectory point in the spatial movement trajectory, and use the rotation angle vector as the change in direction at the second trajectory point in the spatial movement trajectory.

[0074] The displacement vector is represented as a line segment vector pointing from the first trajectory point to the second trajectory point in the spatial movement trajectory. The length and direction of the line segment vector are determined by the displacement vector. The turning angle vector is represented as the change in direction at the second trajectory point in the spatial movement trajectory, indicating the change in orientation of the muck-carrying vehicle when it arrives at the second trajectory point.

[0075] Step S1352: Process all adjacent vehicle spatial position features in the vehicle spatial position feature sequence according to the order of the time identifiers. Connect the displacement vector and rotation vector obtained by the operation in the time order to form a spatial movement trajectory. Mark the center point coordinate value, orientation angle value and detection area box size parameters of the corresponding vehicle spatial position feature at each trajectory point of the spatial movement trajectory.

[0076] Following the order of time identifiers, all adjacent vehicle spatial position features in the vehicle spatial position feature sequence are processed sequentially. For each pair of adjacent features, the corresponding displacement vector and rotation vector are calculated, and they are then connected end-to-end in chronological order to form a complete spatial movement trajectory. At each trajectory point, the center point coordinates, orientation angle, and detection area bounding box dimensions of the corresponding vehicle spatial position feature are labeled to comprehensively understand the spatial position and status of the muck-hauling vehicle at each moment.

[0077] Step S140: Perform trajectory synchronization mapping operation on the loading state change trajectory and the spatial movement trajectory to establish the correspondence between the loading state change nodes and the spatial movement trajectory segments, and obtain the loading change records of the muck truck at each spatial location in the construction area.

[0078] The loading status change trajectory and spatial movement trajectory describe the state changes of the muck-hauling vehicle from different perspectives. To combine the two for analysis, a trajectory synchronization mapping operation needs to be performed. The purpose of the trajectory synchronization mapping operation is to establish the correspondence between loading status change nodes and spatial movement trajectory segments, thereby obtaining the loading change record of the muck-hauling vehicle at each spatial location within the construction area. The loading change record can clearly show when and where the muck-hauling vehicle performed operations such as loading or unloading muck.

[0079] Step S141: Extract the start time point and end time point of the load change phase sequence in the load change trajectory, and determine the time interval of the load change node based on the start time point and end time point of the change.

[0080] First, extract the start and end times of the load change stages from the load status change trajectory. The start time is the moment when the load begins to change significantly, and the end time is the moment when the load change stops. Based on these two times, the time interval of the load status change nodes can be determined, that is, the time period during which the load change occurs.

[0081] Step S142: Extract the starting trajectory point from the spatial movement trajectory that has the same time identifier as the starting time point of the change, and extract the ending trajectory point from the spatial movement trajectory that has the same time identifier as the ending time point of the change.

[0082] In the spatial movement trajectory, each trajectory point has a corresponding time identifier. Based on the start and end times of the loading status change nodes, the starting and ending trajectory points with the same time identifiers are extracted from the spatial movement trajectory. The starting trajectory point is the spatial position of the muck-hauling vehicle at the start time of the change, and the ending trajectory point is the spatial position of the muck-hauling vehicle at the end time of the change.

[0083] Step S143: Mark the trajectory line segment between the starting trajectory point and the ending trajectory point in the spatial movement trajectory as the spatial movement trajectory segment corresponding to the loading state change node.

[0084] Mark the trajectory segment between the starting and ending trajectory points in the spatial movement trajectory. This trajectory segment corresponds to the spatial movement trajectory segment at the node of loading status change. This trajectory segment reflects the travel path of the muck truck during the loading status change.

[0085] Step S144: Determine the change type of the loading status change node based on the change direction of the loading quantity change phase sequence parameter of the loading quantity occupancy ratio parameter. The change type includes loading quantity increase type and loading quantity decrease type.

[0086] Analyze the direction of change of the proportion parameter of loaded goods in the loading volume change sequence. If the proportion parameter of loaded goods gradually increases, it indicates that the muck trucks are loading goods, and the change type is the loading increase type; if the proportion parameter of loaded goods gradually decreases, it indicates that the muck trucks are unloading goods, and the change type is the loading decrease type.

[0087] Step S145: When the change type of the loading status change node is the loading increase type, mark the spatial position corresponding to the starting trajectory point of the spatial movement trajectory segment as the slag loading candidate point, and mark the spatial position corresponding to the ending trajectory point of the spatial movement trajectory segment as the slag loading completion point.

[0088] When the change type of the loading status change node is determined to be the loading increase type, the spatial position corresponding to the starting trajectory point of the spatial movement trajectory segment is marked as the slag loading candidate point, that is, the position where the slag transport vehicle starts loading; the spatial position corresponding to the ending trajectory point is marked as the slag loading completion point, that is, the position where the slag transport vehicle finishes loading.

[0089] Step S146: When the change type of the loading status change node is the load reduction type, mark the spatial position corresponding to the starting trajectory point of the spatial movement trajectory segment as the slag unloading start point, and mark the spatial position corresponding to the ending trajectory point of the spatial movement trajectory segment as the slag unloading completion point.

[0090] When the change type is the load reduction type, the spatial position corresponding to the starting trajectory point of the spatial movement trajectory segment is marked as the unloading start point, that is, the position where the slag transport vehicle begins to unload; the spatial position corresponding to the ending trajectory point is marked as the unloading completion point, that is, the position where the slag transport vehicle completes unloading.

[0091] Step S147: Perform an association storage operation on the candidate loading point, loading completion point, unloading start point and unloading completion point according to the time sequence and spatial position relationship to generate a loading change record of the muck truck in each spatial position within the construction area.

[0092] The marked candidate loading points, loading completion points, unloading start points, and unloading completion points are linked and stored according to time sequence and spatial location. The time sequence ensures the sequential nature of the loading change records, while the spatial location reflects the distribution of these points within the construction area. Through this linked storage, a loading change record of the muck trucks at each spatial location within the construction area is generated, detailing the loading and unloading process of the muck trucks.

[0093] Step S150: Based on the loading change record, determine the set of loading points and unloading points of the muck truck in the construction area, and compile a muck truck monitoring report based on the spatial distance relationship between the loading point set and the unloading point set and the duration parameter of the loading status change trajectory.

[0094] Based on the loading change records, the locations of the loading and unloading points of the muck trucks within the construction area can be determined. The set of loading point locations includes all possible loading locations, and the set of unloading point locations includes all possible unloading locations. Then, based on the spatial distance relationship between the loading and unloading points and the duration parameter of the loading status change trajectory, indicators such as the muck transportation efficiency of the muck trucks are calculated and compiled into a muck transportation monitoring report.

[0095] Step S151: Extract the spatial coordinates of all points marked as slag loading completion points from the loading change record. Perform cluster analysis on the extracted spatial coordinates of the slag loading completion points. Assign slag loading completion points whose spatial distance is less than the preset clustering distance threshold to the same slag loading area. Calculate the average value of the coordinates of all slag loading completion points in each slag loading area as the slag loading point location coordinates. Combine all the calculated slag loading point location coordinates into a slag loading point location set.

[0096] First, the spatial coordinates of all points marked as slag loading completion points are extracted from the loading change records. These coordinates represent the positions of the slag-carrying vehicles when they have finished loading. Then, cluster analysis is performed on these coordinates. Cluster analysis is a method of grouping data points into different clusters, resulting in high similarity among data points within the same cluster and significant differences between data points in different clusters. In this embodiment, slag loading completion points with a spatial distance less than a preset clustering distance threshold are grouped into the same slag loading area. The preset clustering distance threshold is set based on the actual conditions of the construction area and the monitoring accuracy requirements. The average value of the coordinates of all slag loading completion points within each slag loading area is calculated, and this average value is used as the location coordinates of the slag loading points. Finally, all the calculated slag loading point location coordinates are combined to form a set of slag loading point locations.

[0097] Step S152: Extract the spatial coordinates of all points marked as unloading completion points from the loading change record. Perform cluster analysis on the extracted spatial coordinates of unloading completion points. Group unloading completion points with a spatial distance less than the preset clustering distance threshold into the same unloading area. Calculate the average value of the coordinates of all unloading completion points in each unloading area as the unloading point location coordinates. Combine all the calculated unloading point location coordinates into a set of unloading point locations.

[0098] Similarly, the spatial coordinates of all points marked as unloading completion points are extracted from the loading change records. Cluster analysis is performed on these coordinates, grouping unloading completion points whose spatial distance is less than a preset clustering distance threshold into the same unloading region. The average coordinate of all unloading completion points within each unloading region is calculated as the unloading point location coordinates, and all unloading point location coordinates are combined into a set of unloading point locations.

[0099] Step S153: Extract the coordinates of each loading point and the corresponding unloading point from the loading change record, and calculate the Euclidean distance between each loading point and the corresponding unloading point as the slag transport distance parameter.

[0100] Find the coordinates of each loading point and its corresponding unloading point from the loading change records. There is a one-to-one correspondence between loading and unloading points; that is, one loading corresponds to one unloading. Calculate the Euclidean distance between the coordinates of each loading point and its corresponding unloading point, and use this distance as the slag transport distance parameter. Euclidean distance is a commonly used distance calculation method that accurately reflects the straight-line distance between two points.

[0101] Step S154: Extract the difference between the loading ratio parameter corresponding to the location coordinates of each loading point and the loading ratio parameter corresponding to the location coordinates of the unloading point from the loading status change trajectory as the single slag transport volume parameter.

[0102] Extract the load occupancy ratio parameters corresponding to the coordinates of each loading point and the corresponding load occupancy ratio parameters corresponding to the coordinates of each unloading point from the loading status change trajectory. Calculate the difference between these two parameters and use it as the single-trip slag transport volume parameter. The single-trip slag transport volume parameter reflects the amount of cargo transported by the slag transport vehicle each time.

[0103] Step S155: Calculate the slag transportation efficiency index of the slag transportation vehicle in the construction area based on the slag transportation distance parameter and the single slag transportation volume parameter. Organize the operation according to the transportation trip of the slag transportation vehicle by setting the slag loading point location set, the slag unloading point location set, the slag transportation distance parameter, the single slag transportation volume parameter, and the slag transportation efficiency index, and compile the slag transportation monitoring report of the slag transportation vehicle.

[0104] Based on the slag transport distance and single-trip slag transport volume parameters, the slag transport efficiency index of slag transport vehicles can be calculated. The slag transport efficiency index can be the slag transport volume per unit time, the slag transport volume per unit distance, etc. Then, the loading point location set, unloading point location set, slag transport distance parameter, single-trip slag transport volume parameter, and slag transport efficiency index are organized according to the transport trips of the slag transport vehicles. A transport trip refers to the process of a slag transport vehicle completing one transport from the loading point to the unloading point. Organizing this information according to transport trips clearly shows the situation of each transport trip. Finally, the organized information is compiled into a slag transport monitoring report, which includes detailed information such as the transport route, slag transport volume, and slag transport efficiency of the slag transport vehicles.

[0105] For example, in step S160: obtain the slag transport monitoring reports corresponding to multiple slag transport vehicles in the construction area, and extract the frequency parameters of the coordinates of each slag loading point from the slag loading point location set in each slag transport monitoring report.

[0106] To conduct a comprehensive analysis of the muck transportation situation within the construction area, it is necessary to obtain muck transportation monitoring reports for multiple muck transportation vehicles. From each muck transportation monitoring report, the frequency parameter of the coordinates of each muck loading point is extracted from the set of muck loading point locations. The frequency parameter represents the number of times a particular muck loading point's coordinates appear in the muck transportation monitoring report, reflecting the usage frequency of that muck loading point.

[0107] Step S161: Sort the slag loading points in the construction area according to the frequency of occurrence of the slag loading point location coordinates, and identify the slag loading point location coordinates that occur more frequently than the preset frequency threshold as the core slag loading operation area identifier.

[0108] Based on the frequency of occurrence of slag loading point coordinates, the slag loading points within the construction area are sorted. The sorting can be done from highest to lowest frequency. Then, the coordinates of slag loading points whose frequency exceeds a preset threshold are identified, and these coordinates are designated as the core slag loading operation area. The core slag loading operation area is the area within the construction area where slag loading activities are most concentrated.

[0109] Step S162: Extract the frequency parameters of the coordinates of each unloading point from the set of unloading point locations in each slag transportation monitoring report. Sort the unloading points in the construction area according to the frequency parameters of the unloading point coordinates. Identify the coordinates of unloading points whose frequency exceeds the preset frequency threshold as the core unloading operation area identifier.

[0110] Similarly, the frequency parameters of the coordinates of each unloading point in the unloading point location set are extracted from each unloading point monitoring report. The unloading points are sorted according to these parameters, and the coordinates of unloading points whose frequency exceeds a preset frequency threshold are identified as the core unloading operation area markers.

[0111] Step S163: Overlay the core slag loading operation area marker and the core slag unloading operation area marker onto the electronic map of the construction area to generate a slag loading operation distribution map and a slag unloading operation distribution map.

[0112] The identification marks for the core slag loading and unloading operation areas are overlaid onto the electronic map of the construction area. The electronic map is a digital representation of the construction area, providing a clear visual representation of its geographical information. Through this overlay operation, distribution maps for slag loading and unloading operations can be generated, clearly showing the distribution of the core slag loading and unloading operation areas within the construction area.

[0113] Step S164: Calculate the transportation load parameters of each transportation path in the construction area based on the slag loading frequency parameters of different areas in the slag loading operation distribution map and the slag unloading frequency parameters of different areas in the slag unloading operation distribution map. Mark the transportation paths whose transportation load parameters exceed the preset load threshold as warning paths and generate a construction area slag transportation scheduling suggestion report containing warning path identifiers.

[0114] Based on the slag loading frequency parameters of different areas in the slag loading operation distribution map and the slag unloading frequency parameters of different areas in the slag unloading operation distribution map, the transportation load parameters of each transportation route within the construction area are calculated. The transportation load parameters reflect the busyness of the transportation route. The transportation load parameters are compared with preset load thresholds. When the transportation load parameters exceed the preset load thresholds, the transportation route is marked as a warning route. Warning routes indicate that the transportation pressure on that route is high, and there may be congestion or other problems. Finally, a construction area slag transportation scheduling suggestion report containing warning route identifiers is generated to optimize the driving routes of slag transport vehicles and improve transportation efficiency.

[0115] Figure 2 The illustration shows exemplary hardware and software components of an intelligent slag removal monitoring system 100 for hydropower station construction projects, which can implement the ideas of this application, according to some embodiments of this application. For example, a processor 120 can be used in the intelligent slag removal monitoring system 100 for hydropower station construction projects and to perform the functions described in this application.

[0116] The intelligent slag removal monitoring system 100 applied to hydropower station construction projects can be a general-purpose server or a special-purpose server; both can be used to implement the intelligent slag removal monitoring method for hydropower station construction projects described in this application. Although only one server is shown in this application, for convenience, the functions described in this application can be implemented in a distributed manner on multiple similar platforms to balance the load.

[0117] For example, an intelligent slag removal monitoring system 100 applied to a hydropower station construction project may include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and various forms of storage media 140, such as a disk, ROM, or RAM, or any combination thereof. Exemplarily, the intelligent slag removal monitoring system 100 applied to a hydropower station construction project may also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The methods of this application can be implemented according to these program instructions. The intelligent slag removal monitoring system 100 applied to a hydropower station construction project also includes an I / O interface 150 between the computer and other input / output devices.

[0118] For ease of explanation, only one processor is described in the intelligent slag removal monitoring system 100 applied to hydropower station construction projects. However, it should be noted that the intelligent slag removal monitoring system 100 applied to hydropower station construction projects in this application may also include multiple processors. Therefore, the steps performed by one processor described in this application may also be performed jointly or individually by multiple processors. For example, if the processor of the intelligent slag removal monitoring system 100 applied to hydropower station construction projects performs steps A and B, it should be understood that steps A and B may also be performed jointly by two different processors or individually by one processor. For example, the first processor performs step A, the second processor performs step B, or the first processor and the second processor jointly perform steps A and B.

[0119] Furthermore, this embodiment of the invention also provides a readable storage medium, which has computer-executable instructions pre-set in it. When the processor executes the computer-executable instructions, the intelligent slag transport monitoring method applied to hydropower station construction projects as described above is implemented.

[0120] It should be noted that, in order to simplify the description of the present invention and thus help to understand one or more embodiments of the invention, multiple features may sometimes be grouped into one embodiment, drawing or description thereof in the foregoing description of the embodiments of the present invention.

Claims

1. A method for monitoring intelligent slag removal in hydropower station construction projects, characterized in that, The method includes: Acquire video monitoring data streams of muck-hauling vehicles within the construction area of ​​a hydropower station project. The video monitoring data streams contain multiple muck-hauling vehicle monitoring image frame units arranged in the acquisition sequence. Perform muck vehicle feature parsing on the muck vehicle monitoring image frame unit to obtain the loading status features and vehicle body spatial position features of the muck vehicle in each muck vehicle monitoring image frame unit; Based on the loading status features and vehicle spatial position features corresponding to multiple consecutively arranged monitoring image frames of slag transport vehicles, an association mapping operation between loading changes and position migration is performed to generate the loading status change trajectory and spatial movement trajectory of the slag transport vehicles. Perform trajectory synchronization mapping operation on the loading state change trajectory and the spatial movement trajectory to establish the correspondence between the loading state change nodes and the spatial movement trajectory segments, and obtain the loading change records of the muck truck at each spatial location in the construction area. Based on the loading change records, determine the set of loading points and unloading points of the muck trucks in the construction area, and compile a muck truck monitoring report based on the spatial distance relationship between the loading point set and the unloading point set and the duration parameter of the loading status change trajectory.

2. The intelligent slag removal monitoring method applied to hydropower station construction projects according to claim 1, characterized in that, The acquisition of video monitoring data streams of muck-hauling vehicles within the construction area of ​​the hydropower station project includes multiple muck-hauling vehicle monitoring image frame units arranged in the acquisition sequence. Muck-hauling vehicle feature parsing is performed on each monitoring image frame unit to obtain the loading status features and vehicle spatial position features of the muck-hauling vehicles in each monitoring image frame unit, including: The raw video stream data output from multiple fixed monitoring points within the construction area of ​​the hydropower station project is collected. The raw video stream data is subjected to frame sequence decomposition operation according to the preset frame extraction interval to obtain multiple monitoring image frame units of muck trucks arranged in the acquisition time sequence. Each monitoring image frame unit of muck trucks is associated with a time identifier and a spatial identifier. The muck truck monitoring image frame unit is input into the muck truck detection network. The muck truck detection network performs a candidate region generation operation on the muck truck monitoring image frame unit to obtain multiple candidate vehicle region boxes. For each candidate vehicle region box, a vehicle category confidence calculation operation is performed, and candidate vehicle region boxes with a category confidence exceeding the judgment threshold are selected as muck truck detection region boxes. Extract the pixel feature matrix within the detection area of ​​the slag transport vehicle, and perform loading area segmentation operation on the pixel feature matrix to obtain the cargo box area sub-image and the front area sub-image of the slag transport vehicle. Perform a loading identification operation on the cargo compartment area sub-image to generate a loading occupancy ratio parameter and a loading surface texture feature vector for the cargo compartment area. Combine the loading occupancy ratio parameter and the loading surface texture feature vector into a loading status feature. A vehicle orientation recognition operation is performed on the vehicle front area sub-image to generate vehicle front orientation angle parameters. Based on the center point coordinates of the detection area box of the slag transport vehicle and the vehicle front orientation angle parameters, the vehicle body spatial position features are compiled. The vehicle body spatial position features include the center point coordinate value, orientation angle value, and the size parameters of the detection area box.

3. The intelligent slag removal monitoring method applied to hydropower station construction projects according to claim 2, characterized in that, The loading identification operation is performed on the cargo compartment area sub-map to generate a loading occupancy ratio parameter and a loading surface texture feature vector for the cargo compartment area. The loading occupancy ratio parameter and the loading surface texture feature vector are combined into loading status features, including: Perform edge detection on the cargo compartment area sub-image to extract the upper edge outline of the cargo compartment and the pixel distribution map of the cargo compartment interior area in the cargo compartment area sub-image; The coordinates of the upper boundary of the cargo compartment opening are determined based on the upper edge contour line of the cargo compartment. The distribution range of the pixels on the surface of the loaded object is identified based on the pixel distribution map of the interior region of the cargo compartment. The vertical distance difference parameter between the vertex coordinates of the pixels on the surface of the loaded object and the upper edge contour line of the cargo compartment is calculated. Perform pixel value statistical analysis on the pixel distribution map of the cargo compartment interior area to generate mean and variance parameters of the pixel values ​​on the surface of the load. Based on the vertical distance difference parameter and the mean and variance parameters of the pixel values ​​on the surface of the load, calculate the proportion of the cargo occupying the interior space of the cargo compartment as the cargo occupancy ratio parameter. A texture feature extraction operation is performed on the pixel distribution map of the interior region of the cargo compartment, the gray-level difference matrix of adjacent pixels in the pixel distribution map of the interior region of the cargo compartment is calculated, and an eigenvalue decomposition operation is performed on the gray-level difference matrix to generate a cargo surface texture feature vector containing contrast feature components, correlation feature components and entropy feature components. The loading occupancy ratio parameter and the loading surface texture feature vector are concatenated to generate loading state features that include both numerical and vector parameters.

4. The intelligent slag removal monitoring method applied to hydropower station construction projects according to claim 2, characterized in that, The process involves performing vehicle orientation recognition on the vehicle front region sub-image to generate vehicle front orientation angle parameters. Based on the center point coordinates of the muck truck detection area bounding box and the vehicle front orientation angle parameters, the vehicle body spatial position features are compiled. These features include center point coordinates, orientation angle values, and the size parameters of the detection area bounding box, including: Perform key point detection on the vehicle front area sub-image to extract the coordinates of the foremost point of the vehicle front, the center point of the top of the vehicle front, and the center point of the bottom of the vehicle front in the vehicle front area sub-image; The initial direction vector of the vehicle's orientation is determined based on the direction of the line connecting the coordinates of the foremost point of the vehicle's front end and the center point of the top of the vehicle's front end. The initial direction vector is then corrected based on the direction of the line connecting the coordinates of the foremost point of the vehicle's front end and the center point of the bottom of the vehicle's front end, thereby generating the vehicle's orientation direction vector. Calculate the angle between the vehicle's facing direction vector and the positive direction of the horizontal axis in the pre-established construction area coordinate system, and use the angle as the vehicle's facing angle parameter; The center point coordinates are extracted from the detection area box of the slag transport vehicle, and the width and height parameters of the area box are extracted from the detection area box of the slag transport vehicle as the size parameters of the detection area box. The center point coordinates, vehicle heading angle, and detection area frame size are encapsulated according to a preset data structure to generate vehicle spatial position features containing numerical coordinate components, angle components, and size components.

5. The intelligent slag removal monitoring method applied to hydropower station construction projects according to claim 1, characterized in that, The process involves performing a correlation mapping operation between loading changes and position migration based on the loading status features and vehicle spatial position features corresponding to multiple consecutively arranged monitoring image frames of slag transport vehicles, generating the loading status change trajectory and spatial movement trajectory of the slag transport vehicles, including: According to the arrangement order of the time identifiers of the monitoring image frame units of the slag transport vehicle, the loading status feature sequence and the vehicle body spatial position feature sequence corresponding to the same slag transport vehicle are extracted. Each loading status feature in the loading status feature sequence has a time correspondence with the vehicle body spatial position feature with the same time identifier in the vehicle body spatial position feature sequence. For two loading state features with adjacent time identifiers in the loading state feature sequence, a loading state difference calculation operation is performed to identify the feature point positions in the loading state feature sequence where the loading occupancy ratio parameter changes. Based on the identified feature point positions, the loading state feature sequence is divided into a loading volume stable stage sequence and a loading volume change stage sequence. The loading state features in the loading volume stabilization phase sequence are connected in chronological order to form a loading state change trajectory. The loading state change trajectory is marked with the loading volume occupancy ratio parameter and loading volume surface texture feature vector corresponding to each loading state feature. A spatial position migration calculation operation is performed on two vehicle spatial position features with adjacent time identifiers in the vehicle spatial position feature sequence. A displacement vector is generated based on the difference in the center point coordinates of the two vehicle spatial position features, and a turning angle vector is generated based on the difference in the vehicle head orientation angle parameter of the two vehicle spatial position features. The displacement vector and rotation vector are sequentially connected in chronological order to form a spatial movement trajectory. The center point coordinates, orientation angle, and size parameters of the detection area box are marked for each trajectory point in the spatial movement trajectory.

6. The intelligent slag removal monitoring method applied to hydropower station construction projects according to claim 5, characterized in that, The loading status feature sequence and vehicle body spatial position feature sequence corresponding to the same muck-hauling vehicle are extracted according to the arrangement order of the time identifiers of the monitoring image frame units of the muck-hauling vehicle. Each loading status feature in the loading status feature sequence has a time correspondence with the vehicle body spatial position feature of the same time identifier in the vehicle body spatial position feature sequence. A loading status difference calculation operation is performed on two loading status features with adjacent time identifiers in the loading status feature sequence to identify the feature point positions where the load occupancy ratio parameter changes. Based on the identified feature point positions, the loading status feature sequence is divided into a stable loading phase sequence and a loading change phase sequence, including: Extract the unique identifier of the muck truck from the detection area box, and based on the unique identifier, filter the loading status features and vehicle spatial position features belonging to the same muck truck from the muck truck feature analysis results of all muck truck monitoring image frame units. The selected loading status features are sorted according to the time identifiers of the monitoring image frames of the slag transport vehicles to generate a loading status feature sequence. The selected vehicle body spatial position features are sorted according to the same time order to generate a vehicle body spatial position feature sequence. Traverse the two loading status features of each pair of adjacent time identifiers in the loading status feature sequence, extract the loading occupancy ratio parameter of the first loading status feature and the loading occupancy ratio parameter of the second loading status feature, and calculate the absolute value of the difference between the two loading occupancy ratio parameters. The absolute value of the difference is compared with a preset threshold for the significance of change. When the absolute value of the difference exceeds the threshold for the significance of change, the position index of the second loading state feature in the loading state feature sequence is marked as the position of the change feature point. Based on the marked change feature point positions, the loading state feature sequence is segmented and cut. The loading state features between two adjacent change feature point positions are combined into a loading volume stable stage sequence, and the loading state features corresponding to the change feature point positions are combined into a loading volume change stage sequence. In the loading volume stable stage sequence, the fluctuation range of the loading occupancy ratio parameter of each loading state feature is less than a preset fluctuation threshold, and in the loading volume change stage sequence, the change amplitude of the loading occupancy ratio parameter of each loading state feature exceeds a preset change amplitude threshold.

7. The intelligent slag removal monitoring method applied to hydropower station construction projects according to claim 5, characterized in that, The process involves performing a spatial position migration calculation on two vehicle spatial position features with adjacent time identifiers in the vehicle spatial position feature sequence. A displacement vector is generated based on the difference in center point coordinates between the two vehicle spatial position features, and a turning vector is generated based on the difference in the vehicle's frontal orientation angle parameter between the two vehicle spatial position features. The displacement vector and turning vector are then sequentially connected in chronological order to form a spatial movement trajectory. The spatial movement trajectory is then labeled with the center point coordinates, orientation angle, and detection area bounding box dimensions corresponding to each trajectory point, including: Extract the first and second vehicle spatial position features with adjacent time identifiers from the vehicle spatial position feature sequence; extract the first center point coordinate value from the first vehicle spatial position feature; and extract the second center point coordinate value from the second vehicle spatial position feature. Calculate the difference between the abscissa component of the second center point coordinate value and the abscissa component of the first center point coordinate value; calculate the difference between the ordinate component of the second center point coordinate value and the ordinate component of the first center point coordinate value; and combine the abscissa difference and the ordinate difference into a displacement vector. Extract the first vehicle front orientation angle parameter from the spatial position features of the first vehicle body, extract the second vehicle front orientation angle parameter from the spatial position features of the second vehicle body, calculate the angle difference between the second vehicle front orientation angle parameter and the first vehicle front orientation angle parameter, and use the angle difference as the turning angle vector; The displacement vector is used as the line segment vector pointing from the first trajectory point to the second trajectory point in the spatial movement trajectory, and the rotation angle vector is used as the change in direction at the second trajectory point in the spatial movement trajectory. Process all adjacent vehicle spatial position features in the vehicle spatial position feature sequence according to the order of time identifiers. Connect the displacement vector and rotation vector obtained by the operation in time order to form a spatial movement trajectory. Mark the center point coordinate value, orientation angle value and detection area box size parameters of the corresponding vehicle spatial position feature at each trajectory point of the spatial movement trajectory.

8. The intelligent slag removal monitoring method applied to hydropower station construction projects according to claim 1, characterized in that, The process involves performing a trajectory synchronization mapping operation on the loading state change trajectory and the spatial movement trajectory to establish a correspondence between loading state change nodes and spatial movement trajectory segments, thereby obtaining loading change records of the muck-carrying vehicle at each spatial location within the construction area, including: Extract the start and end times of the load change stages from the load change trajectory, and determine the time interval of the load change nodes based on the start and end times of the changes. Extract the starting trajectory point with the same time identifier as the starting time point of the change from the spatial movement trajectory; extract the ending trajectory point with the same time identifier as the ending time point of the change from the spatial movement trajectory. The trajectory line segment between the starting trajectory point and the ending trajectory point in the spatial movement trajectory is marked as the spatial movement trajectory segment corresponding to the loading state change node; The change type of the loading status change node is determined based on the direction of change of the loading quantity change phase sequence parameter of the loading quantity change parameter. The change type includes loading quantity increase type and loading quantity decrease type. When the change type of the loading status change node is the loading increase type, the spatial position corresponding to the starting trajectory point of the spatial movement trajectory segment is marked as the slag loading candidate point, and the spatial position corresponding to the ending trajectory point of the spatial movement trajectory segment is marked as the slag loading completion point. When the change type of the loading status change node is the load reduction type, the spatial position corresponding to the starting trajectory point of the spatial movement trajectory segment is marked as the unloading start point, and the spatial position corresponding to the ending trajectory point of the spatial movement trajectory segment is marked as the unloading completion point. The candidate loading point, loading completion point, unloading start point, and unloading completion point are associated and stored according to time sequence and spatial location relationship to generate a record of loading changes of the muck truck in each spatial location within the construction area.

9. The intelligent slag removal monitoring method for hydropower station construction projects according to claim 8, characterized in that, The process involves determining the set of loading and unloading points for the muck-carrying vehicles within the construction area based on the loading change records, and compiling a muck-carrying monitoring report based on the spatial distance relationship between the loading and unloading point sets and the duration parameter of the loading status change trajectory. This includes: Extract the spatial coordinates of all points marked as slag loading completion points from the loading change record. Perform cluster analysis on the extracted spatial coordinates of slag loading completion points. Group slag loading completion points whose spatial distance is less than the preset clustering distance threshold into the same slag loading area. Calculate the average value of the coordinates of all slag loading completion points in each slag loading area as the slag loading point coordinates. Combine all the calculated slag loading point coordinates into a slag loading point location set. Extract the spatial coordinates of all points marked as unloading completion points from the loading change record. Perform cluster analysis on the extracted spatial coordinates of unloading completion points. Group unloading completion points with a spatial distance less than the preset clustering distance threshold into the same unloading area. Calculate the average value of all unloading completion point coordinates in each unloading area as the unloading point location coordinates. Combine all the calculated unloading point location coordinates into a set of unloading point locations. Extract the coordinates of each loading point and the corresponding unloading point from the loading change record, and calculate the Euclidean distance between each loading point and the corresponding unloading point as the slag transport distance parameter. The difference between the loading ratio parameter corresponding to the location coordinates of each loading point and the loading ratio parameter corresponding to the location coordinates of the unloading point is extracted from the loading status change trajectory and used as the single slag transport volume parameter. Based on the slag transport distance parameters and single slag transport volume parameters, calculate the slag transport efficiency index of the slag transport vehicle in the construction area. Organize the operation according to the transport trip of the slag transport vehicle by setting the slag loading point location set, slag unloading point location set, slag transport distance parameters, single slag transport volume parameters, and slag transport efficiency index, and compile the slag transport monitoring report of the slag transport vehicle.

10. An intelligent slag removal monitoring system applied to hydropower station construction projects, characterized in that, The intelligent slag removal monitoring system applied to hydropower station construction projects includes a processor and a memory. The memory and the processor are connected. The memory is used to store programs, instructions, or code. The processor is used to execute the programs, instructions, or code in the memory to implement the intelligent slag removal monitoring method applied to hydropower station construction projects as described in any one of claims 1-9.

Images