Mine unmanned aerial vehicle danger source image processing early warning method and system

Abstract

The application discloses a mine unmanned aerial vehicle danger source image processing early warning method and system, relates to the technical field of image detection, and comprises the following steps: acquiring a mine unmanned aerial vehicle inspection image, performing coordinate conversion and image group sorting on a shooting position, camera parameters and mine site object registration records, and generating image position conversion records, image group processing records and image participation processing records; aligning the processable image and the comparison image in the mine site coordinates, extracting an image change area, and generating a danger source suspected record, a fixed object verification request record and a collection interference verification request record; combining effective adjacent relationships to perform suspected area connection preprocessing, source verification and continuous confirmation processing, and generating a danger source confirmation record or an intermittent to-be-confirmed record; and processing a danger source site area and a mine object area correspondingly, and generating a danger source influence record, a danger source output result and a warning evidence record. The application improves the accuracy of danger source source differentiation and warning evidence correlation.

Description

Technical Field

[0001] This invention relates to the field of image detection technology, and in particular to a method and system for image processing and early warning of hazardous sources in mining drones. Background Technology

[0002] Open-pit mine areas contain various on-site objects, including slopes, steps, transport roads, drainage facilities, stockpiles, and equipment operating faces. Due to their high mobility and wide field of view, drone inspections are often used to acquire visible light or infrared images of the mine site. These images are then combined with image detection methods to identify hazards such as falling rocks, cracks, water accumulation, personnel intrusion, vehicle stops, and changes in the stockpile structure. This type of method typically relies on inspection images, mine area boundaries, and on-site safety management requirements to detect, locate, and issue early warnings for target areas in the images, representing an interdisciplinary application of image processing and mine safety monitoring.

[0003] However, existing methods still have two limitations: First, the distinction between the sources of image change areas and fixed objects and acquisition interference is not refined enough. Fixed rocks, guardrails, signs, dust, shadows and reflections at the mine site can easily appear in the image at the same time as the hazard source change area, affecting the verification of image detection results. Second, the correspondence between the hazard source confirmation results and mine objects such as transportation roads, drainage ditch entrances, equipment operation ranges, and prohibited entry areas is not clear enough, and the early warning output lacks continuous recording from image change, source verification to object impact. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a method for image processing and early warning of hazardous sources in mining drones, which solves the problems of unclear source differentiation of hazardous source change areas in mining drone inspection images and unclear relationship between early warning output and impact on mining objects.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for image processing and early warning of hazardous sources using a mining drone, comprising: acquiring inspection images from a mining drone; performing coordinate conversion and image group sorting on the shooting location, camera parameters, and mine site object registration records of the inspection images; generating image location conversion records, image group processing records, and image participation records; reading the image group processing records and image location conversion records; aligning the processable images and comparison images in the mine site coordinates; extracting image change areas; and generating suspected hazardous source records, fixed object verification request records, and sampling records based on the correspondence between the change areas and the mine area, fixed objects, and acquisition interference. The system collects interference verification request records; based on suspected hazard source records, fixed object verification request records, and collected interference verification request records, and combined with the effective adjacency relationships in the image group processing records, it performs suspected area continuation preprocessing, source verification, and continuous confirmation processing to generate source verification result records, hazard source continuous records, hazard source confirmation records, or discontinuous pending confirmation records; through hazard source confirmation records and mine site object registration records, it performs object correspondence processing between hazard source site areas and mine object areas, and generates hazard source impact records, hazard source output results, and early warning evidence records based on object entry ratio, distance records, change records, and source verification results.

[0007] As a preferred embodiment of the mine drone hazard source image processing and early warning method of the present invention, the step of acquiring mine drone inspection images and performing coordinate conversion and image group sorting on the shooting position, camera parameters, and mine site object registration records of the inspection images includes: after acquiring the mine drone inspection images, reading the drone's positioning coordinates, flight altitude, shooting direction, gimbal angle, camera intrinsic parameters, and camera extrinsic parameters; reading the object surface data in the mine site object registration records according to the area number; selecting the corresponding local conversion surface; converting the pixel positions in the inspection images into on-site coordinates; and generating the initial image position conversion result; and reading the fixed object registration field in the mine site object registration records. The process involves matching the outline of the current fixed object in the inspection image with the outline range and on-site coordinate range in the fixed object registration field to generate a fixed object number, corresponding outline points, and corresponding point offsets. The initial image position conversion result is then corrected based on the corresponding outline points and corresponding point offsets. When a fixed object is matched, matching results with corresponding point offsets exceeding the allowable range of image position conversion error are discarded. Adjacent image correction object numbers are generated based on the retained matching results. The corrected on-site coordinates, coordinate conversion error, and adjacent image correction object numbers are written into the image position conversion record. If no fixed object is matched or no valid matching result exists after discarding, an uncorrected state is written into the image position conversion record.

[0008] As a preferred embodiment of the mine drone hazard source image processing and early warning method of the present invention, the generation of image position conversion record, image group processing record and image participation processing record includes: sorting the inspection images according to the inspection route number, area number, shooting time and adjacent image number; generating effective adjacent relationships according to the coverage ratio of adjacent images; and determining the processable image number and abnormal image number according to the clarity, exposure, occlusion, edge distortion, shooting position deviation and area corresponding status.

[0009] As a preferred embodiment of the mine drone hazard source image processing and early warning method of the present invention, the step of aligning the processable image and the comparison image in the mine site coordinates includes: after reading the processable image number, selecting the previous image with a valid adjacent relationship as the comparison image; when there is no previous image with a valid adjacent relationship, reading the manually confirmed image before the inspection or the image manually confirmed in the previous inspection cycle as the comparison image, and converting the current inspection image and the comparison image to the same mine site coordinates for regional alignment.

[0010] As a preferred embodiment of the mine drone hazard source image processing and early warning method of the present invention, the step of extracting image change areas and generating hazard source suspected records, fixed object verification request records and acquisition interference verification request records according to the correspondence between the change areas and the mine area, fixed objects and acquisition interference includes: extracting image change areas corresponding to color changes, grayscale changes, edge contour changes and texture direction changes in the aligned comparable area, and corresponding the change areas with the slope range, transportation road range, drainage ditch entrance, equipment operation range, prohibited entry range, fixed object contour and image acquisition anomaly records to generate hazard source suspected records, fixed object verification request records and acquisition interference verification request records.

[0011] As a preferred embodiment of the mine drone hazard source image processing and early warning method of the present invention, the step of combining the effective adjacency relationship in the image group processing record for suspected area continuation preprocessing, source verification and continuous confirmation processing includes: based on the on-site location, suspected hazard source type, main area number and associated area number in the suspected hazard source record, reading the effective adjacency relationship in the image group processing record, calculating the continuation distance and allowable continuation distance value of suspected areas in adjacent images, and generating a suspected area continuation preprocessing record for source verification and continuous confirmation processing.

[0012] As a preferred embodiment of the mine drone hazard source image processing and early warning method of the present invention, the generation of source verification result record, hazard source continuous record, hazard source confirmation record or intermittent pending confirmation record includes: reading fixed object verification request record, acquisition interference verification request record and slope texture reference record, and performing fixed object source verification, acquisition interference source verification and slope texture source verification on the suspected hazard source record respectively; suspected records that have not formed exclusion records continue to generate hazard source continuous record, hazard source confirmation record or intermittent pending confirmation record based on the location continuity result, area change ratio and continuous confirmation ratio.

[0013] As a preferred embodiment of the mine drone hazard source image processing and early warning method of the present invention, the step of performing object correspondence processing between the hazard source site area and the mine object area includes: after reading the hazard source confirmation record, according to the hazard source number, confirmation image number, hazard source continuous record number and site location range in the hazard source confirmation record, back-reading the corresponding hazard source continuous record and image change area record, and obtaining the change area outline, change area area, length record, width record and direction record from the image change area record; generating the hazard source site area according to the change area outline, site location range and image location conversion record, and corresponding the hazard source site area with the records of transportation road range, road intersection range, drainage ditch entrance range, equipment operation range, personnel passage range, prohibited entry range, blasting warning range, vehicle passage range, low-lying working face record or slope rockfall impact range record to generate hazard source object correspondence record.

[0014] As a preferred embodiment of the mine drone hazard source image processing and early warning method of the present invention, the step of generating hazard source impact records, hazard source output results, and early warning evidence records based on object entry ratio, distance records, change records, and source verification results includes: calculating the object entry ratio based on the corresponding records of the hazard source objects, combining the distance records between the hazard source site area and the mine object area, the change records in the continuous hazard source records, and the source verification results to generate hazard source impact records; reading hazard source confirmation records and hazard source impact records, and generating immediate handling early warnings or on-site review early warnings according to the object occupancy status, distance records, change records, and source verification results; reading intermittent pending confirmation records, area change pending confirmation records, acquisition interference pending confirmation records, and slope texture pending confirmation records, and generating a continue acquisition prompt based on the image area to be supplemented, the field to be confirmed, and the reason for the intermittent acquisition; writing the immediate handling early warning, on-site review early warning, or continue acquisition prompt into the hazard source output results, and saving the numbering relationship between the original image, image change area records, suspected hazard source records, verification request records, source verification result records, continuous hazard source records, hazard source confirmation records, hazard source impact records, and hazard source output results.

[0015] Secondly, this invention provides a mine drone hazard source image processing and early warning system, comprising: an image processing module for acquiring mine drone inspection images, reading shooting location, camera parameters, and mine site object registration records for coordinate conversion and image group sorting, generating image location conversion records, image group processing records, and image participation processing records; a change extraction module for reading processable images, aligning the current inspection image with the comparison image in the mine site coordinates, extracting image change areas, and generating hazard source suspected records and verification request records based on the correspondence between change areas and mine areas, fixed objects, and acquisition interference; a source confirmation module for reading hazard source suspected records, fixed object verification request records, and acquisition interference verification request records, performing continuous preprocessing, fixed object source verification, acquisition interference source verification, and continuous confirmation processing on suspected areas, generating hazard source confirmation records or intermittent pending confirmation records; and an early warning output module for reading hazard source confirmation records, corresponding hazard source site areas with mine object areas, and generating hazard source impact records, hazard source output results, and early warning evidence records based on object entry ratio, distance records, change records, and source verification results.

[0016] The beneficial effects of this invention are as follows: By extracting image change areas and generating records of suspected hazards, verification requests for fixed objects, and verification requests for interference, the change areas in the mine drone inspection images can be associated with the mine area, fixed objects, and interference, respectively, providing clear input for source verification and continuous confirmation processing; by processing the correspondence between the hazard confirmation records and the mine site object registration records, hazard impact records, hazard output results, and early warning evidence records are generated, enabling the early warning output to be associated with the object entry ratio, distance records, change records, and source verification results, thereby improving the clarity of hazard source differentiation and the completeness of early warning evidence association. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart for a method of image processing and early warning of hazardous sources using unmanned aerial vehicles (UAVs) in mining.

[0019] Figure 2 This is a schematic diagram of a mine drone hazard source image processing and early warning system.

[0020] Figure 3 This is a flowchart of image position conversion and image group processing.

[0021] Figure 4 Generate a flowchart for image change region extraction and verification requests.

[0022] Figure 5 This chart compares the accuracy of source verification under different interference scenarios.

[0023] Figure 6 A comparison chart showing the completeness of early warning evidence under different hazard source types. Detailed Implementation

[0024] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0025] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0026] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0027] Reference Figures 1-6 As an embodiment of the present invention, this embodiment provides a method for early warning of hazardous sources from mining drones through image processing, comprising the following steps: S1. Acquire images of mine inspection by UAV, perform coordinate conversion and image group sorting on the shooting location, camera parameters and mine site object registration records of the inspection images, and generate image location conversion records, image group processing records and image participation processing records.

[0028] The drone collects images of the mine operation site along the mine inspection route. The collected objects include stope slopes, bench boundaries, transport roads, road intersections, spoil heaps, stockpiles, drainage ditch entrances, the area surrounding the crushing station, equipment operating areas, personnel access areas, prohibited areas, and blasting warning areas. For each inspection image collected, the following information is simultaneously recorded: image number, capture time, inspection route number, capture point number, drone positioning coordinates, flight altitude, shooting direction, gimbal angle, camera intrinsic parameters, camera extrinsic parameters, image resolution, lens focal length, positioning error, image storage path, and adjacent image numbers. The image number distinguishes individual inspection images, the inspection route number distinguishes different inspection tasks, the capture point number determines the acquisition order of images along the inspection route, and adjacent image numbers determine the sequential relationship between consecutive images within the same area. The shooting direction determines the orientation of the inspection image on the mine plan, and the gimbal angle determines the line-of-sight direction when the image is projected onto the mine site plan.

[0029] Inspection images include visible light images. When infrared cameras or dual-light cameras are deployed in the mining area, infrared images are acquired synchronously at the same shooting point, and the visible light and infrared images are written into the same image acquisition record according to the image number or shooting point number. Visible light images are used to extract slope cracks, falling rocks, road obstacles, personnel, vehicles, water accumulation outlines, pile boundaries, and debris outlines; infrared images are used to help distinguish personnel, vehicle heat sources, smoke and dust areas, and low-light areas. The permissible range for image synchronization is determined based on the hardware trigger error of the dual-light camera, the visible light image frame rate, the infrared image frame rate, the UAV flight speed, the shooting altitude, and the test flight images in the mining area, and is written into the image acquisition record. The permissible range for image synchronization is 20 milliseconds to 200 milliseconds; when using dual-light cameras with the same hardware trigger signal, the permissible range for image synchronization is 20 milliseconds to 100 milliseconds; when using visible light cameras and infrared cameras with different acquisition channels but a unified timestamp, the permissible range for image synchronization is 100 milliseconds to 200 milliseconds. The permissible range for image synchronization is confirmed based on the camera calibration results before inspection and the positional offset of the same fixed object in the mine test flight images. When the positional offset of the fixed object exceeds the coordinate conversion error in the image position conversion record, the permissible range for image synchronization is reduced, or the infrared image field is written to an out-of-synchronization state. When the time difference between the acquisition of visible light images and infrared images exceeds the permissible range for image synchronization, the infrared image is not included in the auxiliary judgment of the same image acquisition record, and the visible light image is processed as a single image source. In mine scenarios where only visible light images are acquired, the infrared image field in the image acquisition record is written to an unconfigured state.

[0030] The system reads mine plan, stope design, transport road map, drainage ditch layout, equipment operation area documents, prohibited entry area documents, blasting warning area documents, and manually marked fixed object data to generate a mine site object registration record. The mine site object registration record includes area number, area type, area boundary coordinates, slope range, bench boundary, transport road range, road intersection range, drainage ditch entrance location, equipment operation area, personnel access area, prohibited entry area, blasting warning area, fixed rock location, guardrail location, signage location, equipment foundation location, road edge structure location, and fixed object outline. Area types include slope area, transport road area, spoil heap area, stockpile area, equipment operation area, personnel access area, and prohibited entry area. A correspondence is established between the area number and the shooting point number in the inspection route to determine the mine area corresponding to each inspection image.

[0031] The outline of a fixed object is determined by manually calibrated images, photographs of objects at the mine site, or images confirmed before inspection, and is written into the Fixed Object Registration field of the mine site object registration record. The Fixed Object Registration field records the outline range of the fixed object in the image, the on-site coordinate range of the fixed object, the fixed object type, the fixed object number, and the fixed object registration time. Fixed object types include fixed stones, guardrails, signs, equipment foundations, drainage ditch structures, and road edge structures. The Fixed Object Registration field is not a result of hazard source assessment; it only serves as input data to distinguish between fixed backgrounds and newly added or changed areas during image detection. When the location of a fixed object is adjusted for engineering purposes, the Fixed Object Registration field in the mine site object registration record is updated based on the manually confirmed data, and different versions are distinguished by the record generation time and record version number of the mine site object registration record.

[0032] Before image detection, the drone's positioning coordinates, flight altitude, shooting direction, gimbal angle, camera intrinsic parameters, camera extrinsic parameters, and image resolution are read from the corresponding image acquisition records. The pixel coordinates in the inspection images are converted to mine site coordinates, generating an image location conversion record. The image location conversion record includes the image number, pixel horizontal coordinates, pixel vertical coordinates, site horizontal coordinates, site vertical coordinates, coordinate conversion error, corresponding area number, camera parameter number, and conversion generation time. The site horizontal and vertical coordinates use the same coordinate unit as on the mine plan, and area boundaries, road boundaries, equipment operating areas, prohibited entry areas, and blasting warning areas are registered using the same coordinate unit.

[0033] During image location conversion, the camera's intrinsic and extrinsic parameters, flight altitude, gimbal angle, UAV positioning coordinates, and shooting direction are first read. Then, object surface data from the mine site object registration record is retrieved according to the area number. Object surface data is determined by the mining area design drawing, bench design line, slope surface range, transport road elevation points, drainage ditch layout data, and measured elevation data of the mining area. For slope areas, a local slope conversion surface corresponding to the slope surface range is used; for bench areas, a bench conversion surface corresponding to the bench boundary is used; and for transport roads and equipment operating areas, a local operating conversion surface formed by corresponding elevation points is used. The processor selects a local conversion surface based on the area number corresponding to the inspection image, projects the image pixels along the camera imaging direction onto the corresponding local conversion surface, and generates the site's horizontal coordinates, vertical coordinates, and elevation coordinates. If a local conversion surface cannot be determined, a location conversion confirmation record is generated; this image is not used as the basis for location continuation in continuous hazard source confirmation.

[0034] Image position conversion errors include UAV positioning errors, camera installation errors, gimbal angle recording errors, and fixed object contour matching errors. UAV positioning errors are output by the UAV positioning module; camera installation errors are determined by camera calibration results; gimbal angle recording errors are determined by gimbal control recordings; and fixed object contour matching errors are determined by the offset between the registered contour of the fixed object and the currently recognized contour in the image. Image position conversion errors are written into the image position conversion record and used for the sequential judgment of hazard sources in adjacent images within the same area.

[0035] Based on the inspection route number, area number, shooting time, and adjacent image numbers, continuous inspection images collected in the same area are first grouped into an initial image group. Then, the image position conversion record is read to calculate the coverage ratio of adjacent images. Image pairs that meet the adjacent image coverage requirements are written into the valid adjacency relationship, generating an image group processing record. The image group processing record includes the image group number, inspection route number, area number, number of images, image order, adjacent image correspondence, adjacent image coverage relationship, valid adjacency relationship, processable image number, abnormal image number, and image group generation time. The adjacent image correspondence is used to record the sequential order of image numbers, and the adjacent image coverage relationship is used to record the overlap between the corresponding on-site areas of two images.

[0036] Each image, after location calculation, forms a field coverage area, which represents the area of ​​the mine site plane that the current inspection image corresponds to. The coverage ratio of adjacent images is expressed as follows: ; in, Indicates the first Zhang Image and the First The coverage ratio of adjacent images between images; Indicates the sequence number of the currently processed image in the image group; This indicates the sequence number of the previous adjacent image in the image group that is currently being processed; Indicates the first The on-site coverage area is formed by converting the location of the image; Indicates the first The on-site coverage area is formed by converting the location of the image; Indicates the first The image shows the on-site coverage area and the first Overlapping areas between the areas covered by the images; Indicates the first The image shows the on-site coverage area and the first The overlap area between the areas covered by the images on site; Indicates the first The area covered by the image at the scene; Indicates the first The area covered by the image at the scene; Indicates the first The area covered by the image and the first The smaller value among the areas covered by the image on site; This indicates calculating the area of ​​the region within the parentheses; This represents a non-zero area correction value, used to avoid calculation errors caused by a zero denominator or an excessively small on-site coverage area. The area correction is calculated based on the area corresponding to one pixel after image location conversion, or the minimum effective coverage area preset by the image processing program. Non-zero area correction values ​​are determined according to the image location conversion records within the same inspection task. When the area corresponding to a single pixel in the current image is consistent, the non-zero area correction value is the area corresponding to one pixel. When the area corresponding to a single pixel in different locations in the current image is inconsistent due to local conversion surfaces, flight altitude, or shooting angle, the non-zero area correction value is the average area corresponding to a single pixel within the calculation area. The minimum effective area preset by the image processing program is used when image location conversion records are missing, the area corresponding to a single pixel cannot be stably obtained, or the area corresponding to a single pixel is lower than the recordable area; its value ranges from one to nine pixels. The non-zero area correction value is re-determined at the start of each inspection task based on image resolution, flight altitude, camera parameters, and local conversion surfaces, and written into the image location conversion record. The coverage ratio of adjacent images, the overlap ratio of fixed objects, the area change ratio, and the object entry ratio within the same image group are determined by reading the non-zero area correction value from the corresponding image location conversion record.

[0037] When the coverage ratio of adjacent images meets the requirements, the two images are written into the same image group as a valid adjacent relationship. When the coverage ratio of adjacent images does not meet the requirements, the image order of the two images is retained, but they are not considered as valid adjacent relationships in the continuous confirmation process. The requirements for adjacent image coverage are determined by the inspection route design document, UAV flight altitude, camera field of view, and test flight images of the mining area. Single images in an image group that do not have valid adjacent relationships participate in the extraction of suspected hazard areas, source verification, and object matching. When a suspected hazard record in a single image forms an object occupancy state with a prohibited entry area, equipment operation area, blasting warning area, transportation road area, or drainage ditch entrance area, a single image on-site review record is generated, but a continuous hazard record is not directly generated. The single image on-site review record records the image number, suspected hazard type, object number, object entry ratio, source verification result, and image area to be supplemented, and is used to generate on-site review warnings or continue collection prompts. For images that have formed a valid adjacency relationship, if the suspected area meets the location continuation condition but the area change ratio exceeds the allowable range of area change for the corresponding hazard source type, a rapid area change pending confirmation record is generated. The rapid area change pending confirmation record is not used as an exclusion result, and continues to enter the object corresponding processing. When the object entry ratio or distance record meets the requirements of the corresponding object type, an on-site review warning is generated.

[0038] Each inspection image undergoes a processing decision, generating an image processing record. This record includes image number, image group number, region number, sharpness record, exposure record, occlusion record, edge distortion record, shooting position deviation record, adjacent image coverage record, region correspondence status, processing status, and reason for non-participation. The processing decision also reads the image participation condition record. This record includes sharpness requirements, allowable exposure range, allowable occlusion range, allowable edge distortion range, allowable shooting position deviation, adjacent image coverage requirements, and region correspondence requirements. The image participation condition record is derived from UAV camera calibration results, inspection route design documents, mine test flight images, dusty weather samples, low-light samples, strong light samples, and manually identifiable image samples. The value ranges in the image participation condition record are determined by the UAV camera calibration results, inspection route design documents, mine test flight images, and manually identifiable image samples, and are written into the corresponding record before the inspection task begins. Sharpness requirements are determined based on the normalized edge grayscale variation range and texture grayscale variation range, with values ​​ranging from 0.35 to 0.65. Exposure tolerance ranges are determined based on the average image brightness and the proportion of overexposed and underexposed pixels, with the average brightness ranging from 30 to 225 and the proportion of overexposed or underexposed pixels not exceeding 0.3%. Occlusion tolerance ranges are determined based on the proportion of the occluded area to the effective image area, with values ​​ranging from 0 to 0.35. Edge distortion tolerance ranges are determined based on the proportion of the distorted edge area and the offset of the fixed object boundary, with the distorted edge area... The percentage should not exceed 0.2; the offset of the fixed object boundary should be one to five pixels; the allowable deviation of the shooting position should be 0.5 meters to five meters; the coverage requirement of adjacent images should be 0.25 to 0.7; the minimum number of recorded pixels should be four to twenty-five consecutive pixels; the position overlap ratio in the verification of fixed objects should be 0.5 to 0.8, the outline overlap ratio should be 0.4 to 0.7, and the color difference should be 0 to 0.25; in the judgment of vehicle stationary status, the change in the position of the vehicle center point should be 0.5 meters to three meters, and the number of consecutive images should be two to five.

[0039] Sharpness records are generated from the number of edge pixels, the magnitude of edge grayscale variation, and the magnitude of grayscale variation in textured areas. These records are used to determine whether crack edges, personnel outlines, vehicle outlines, rockfall outlines, and pile boundaries are suitable for extraction. Exposure records are generated from the image's average brightness, brightness distribution range, overexposed pixel areas, and underexposed pixel areas. These records are used to record the impact of strong light, backlighting, and low illumination on image region extraction. Occlusion records are generated from continuous low-texture areas, smoke and dust color areas, dust color areas, and equipment outline occlusion areas. Edge distortion records are generated from the stretching of fixed object outlines in image edge regions, straight line boundary offsets, and the difference in outlines of the same fixed object in adjacent images. Shooting position deviation records are generated from the offset between the current image position conversion record and the shooting point designed for the inspection route. Adjacent image coverage records are generated from the coverage ratio of adjacent images.

[0040] When an inspection image, after location conversion, cannot be mapped to an area number in the mine site object registration record, an area correspondence anomaly record is generated. This record includes the image number, image group number, converted site coverage area, reason for non-correspondence, and reason for exclusion from hazard source extraction. The inspection image corresponding to this area correspondence anomaly record is assigned an anomaly image number and is not included in the extraction of suspected hazard source areas.

[0041] For inspection images that meet the image participation criteria, a participation status is written to the image participation processing record, along with the processable image number from the image group processing record. For inspection images that do not meet the image participation criteria, an acquisition anomaly status is written to the image participation processing record, along with the anomaly image number from the image group processing record, and an image acquisition anomaly record is generated. The image acquisition anomaly record includes the anomaly image number, anomaly source, anomaly area location, anomaly area outline, reason for not participating in hazard source extraction, and the corresponding fields for the anomaly source. Anomaly sources include image blur, overexposure, underexposure, dust obstruction, smoke obstruction, equipment obstruction, edge distortion, shooting position deviation, insufficient coverage of adjacent images, and area correspondence failure.

[0042] Abnormal images are not used for hazard source suspected area extraction, nor are they used as comparison images in image difference processing. The fields for occlusion areas, exposure anomaly areas, edge distortion areas, and anomaly source in abnormal images are only used as input for the acquisition interference verification request record. Acquisition interference verification processing only reads the location, outline, and anomaly source fields of abnormal areas in abnormal images; it does not extract new hazard source suspected areas from abnormal images, nor does it use abnormal images as current inspection images or comparison images for change area extraction.

[0043] S2. Read the image group processing record and image position conversion record, align the processable image and the comparison image in the mine site coordinates, extract the image change area, and generate the hazard source suspected record, fixed object verification request record and acquisition interference verification request record according to the correspondence between the change area and the mine area, fixed object and acquisition interference.

[0044] The system reads the processable image numbers from the image group processing record and then reads the mine site object registration record, image location conversion record, image processing record, and image acquisition anomaly record according to the area number. Anomaly image numbers in the image group processing record are not used for hazard source suspected area extraction; the anomaly area location, anomaly area outline, and anomaly source fields corresponding to the anomaly image number are only used to generate an acquisition interference verification request record. The inspection image corresponding to the processable image number enters the image change extraction processing.

[0045] Before image change extraction and processing, comparison images are determined. If a previous image with a valid adjacency relationship exists and is recorded as a processable image number in the image group processing record, that previous image is read as the comparison image. If a previous image with a valid adjacency relationship exists but is recorded as an abnormal image number, it is not used as the comparison image; instead, the image manually confirmed before the inspection or the image manually confirmed in the previous inspection cycle is read as the comparison image. If no previous image with a valid adjacency relationship exists, the image manually confirmed before the inspection or the image manually confirmed in the previous inspection cycle is read as the comparison image. The image number, source type, corresponding area number, and confirmation time of the comparison image are written into the image comparison source record.

[0046] The system reads the image location conversion record, converts the current inspection image and the comparison image to the same mine site coordinates, and extracts the pixel range corresponding to the same site area from the two images according to the area number. Areas in the current inspection image that can form a site coordinate correspondence with the comparison image are marked as comparable areas; areas that cannot form a site coordinate correspondence are not included in the image change area extraction and are written into the non-comparable area record. The non-comparable area record includes the image number, area number, non-comparable area location, reason for non-comparability, and comparison image number. Reasons for non-comparability include insufficient coverage of adjacent images, missing location conversion, inconsistent area numbers, missing comparison image, and abnormal area correspondence.

[0047] Within the comparable area, color, grayscale, edge contour, and texture direction differences are processed between the current inspection image and the comparison image to generate an image difference record. Color differences are formed by the color channel differences between the current inspection image and the comparison image within the same on-site location range; grayscale differences are formed by the pixel grayscale value differences within the same on-site location range; edge contour differences are formed by differences in edge line position, edge continuity length, and closed contour range; texture direction differences are formed by differences in texture line direction and texture distribution position within the same on-site location range. The image difference record includes the image number, comparison image number, region number, pixel range of the difference region, on-site range of the difference region, color change value, grayscale change value, edge contour change value, and texture direction change value. Before color difference processing, the current inspection image and the comparison image are uniformly converted to the HSV color space, and the correspondence between image size, pixel position, and on-site coordinates is aligned according to the image position conversion record. Color change values ​​are generated by the normalized average of the H channel differences, S channel differences, and V channel differences within the same on-site location range; grayscale change values ​​are generated by the grayscale mean difference and grayscale variance difference within the same on-site location range. Edge contour variation values ​​are generated by extracting edge lines using Canny edge extraction, and then comparing edge line position offset, edge continuity length difference, and closed contour overlap ratio. Texture direction variation values ​​are generated by calculating the principal direction of each texture region using Sobel gradient, and then comparing the principal direction difference within the same location range in the current inspection image and the comparison image.

[0048] When extracting image change regions, an image change region record is generated when any one of the following conditions is met: color change value, grayscale change value, edge contour change value, or texture direction change value. Adjacent pixels form a continuous region. Personnel contours are extracted based on the vertically circumscribed rectangle, human body edge line, reflective clothing color region, or infrared heat source region. Vehicle contours are extracted based on the vehicle body rectangular contour, wheel position, vehicle color region, and the correspondence with the road area. Rockfall contours are extracted based on blocky closed edges, grayscale texture changes, and the occupancy record of the transport road area. Water accumulation contours are extracted based on continuous bright areas, low-texture areas, boundary closure records, and the correspondence with the drainage ditch inlet. When multiple image difference fields simultaneously meet the requirements, the hazard source extraction condition record is read according to the region type, and the corresponding contour record, area record, length record, width record, site location, and change source fields are jointly written into the hazard source suspected record.

[0049] The image variation region consists of adjacent pixels that meet the difference recording conditions. The difference recording conditions include at least one of the following: color change value meets color change recording requirements, grayscale change value meets grayscale change recording requirements, edge contour change value meets contour change recording requirements, or texture direction change value meets texture change recording requirements. The color change recording requirements, grayscale change recording requirements, contour change recording requirements, and texture change recording requirements are determined from mine test flight images, manually annotated variation regions, and normal image samples under different lighting conditions.

[0050] When extracting image change regions, scattered noise points are first removed, and then adjacent difference pixels are connected to form one or more change regions. Scattered noise points are difference pixels that do not form connected regions with adjacent difference pixels, or whose number of connected pixels does not reach the minimum number of recorded pixels. The minimum number of recorded pixels is determined by image resolution, UAV flight altitude, and manually labeled samples. Each change region generates an image change region record. The image change region record includes change region number, image number, image group number, region number, change region pixel position, change region location, change region outline, change region area, change region length, change region width, color change value, grayscale change value, edge contour change value, texture direction change value, comparison image number, and change source field. The change source field is marked according to color change, grayscale change, contour change, texture change, or a combination of multiple changes.

[0051] When the area of ​​a single pixel within the changing region is the same, or when the average area of ​​a single pixel is used, the area of ​​the changing region is expressed as: ; in, Indicates the area of ​​the changed region; This indicates the number of pixels involved in the area calculation within the changed region; This represents the area of ​​the scene corresponding to a single pixel after the image position is converted. It is obtained by statistically analyzing the pixel positions of the changed regions in the image change region record; It is determined by the on-site coordinate conversion results in the image location conversion record.

[0052] When the area of ​​the scene corresponding to different pixels within the changing region differs, the area of ​​the changing region is represented as: ; in, Indicates the pixel number that participated in the area calculation within the changed region; Indicates the first term within the changed region The area of ​​the site corresponding to each pixel after image position conversion. Write image change area records for the generation of suspected hazard source records, fixed object verification request records, and acquisition interference verification request records.

[0053] After the image change area is generated, the mine site object registration record is read, and the change area's location is mapped to the slope range, bench boundary, transport road range, road intersection range, drainage ditch entrance location, equipment operation range, personnel access range, prohibited entry range, blasting warning range, and fixed object outline. This mapping process does not generate an early warning result; instead, it generates a suspected hazard source record, a fixed object verification request record, and a data acquisition interference verification request record, respectively. A change area can generate both a suspected hazard source record and a verification request record simultaneously. When a change area simultaneously meets both the suspected hazard source condition and the fixed object mapping condition, the suspected hazard source record is retained, and a fixed object verification request record is generated concurrently.

[0054] When a changed area spans multiple area numbers, multiple corresponding area records are generated based on the overlap area between the changed area and each of the existing areas. The area with the largest overlap area is written to the main area number, and the remaining areas are written to the associated area number. The suspected hazard source record stores both the main area number and the associated area number. When a changed area simultaneously meets multiple suspected hazard source types, multiple suspected type tags are generated under the same changed area number, and these multiple suspected types are jointly written to the suspected hazard source record for retrieval during source verification processing.

[0055] Before generating a suspected hazard source record, the processor generates a hazard source extraction condition record according to the area type. The field combinations and value ranges in the hazard source extraction condition record are determined by the mine test flight images, manually labeled hazard source samples, UAV flight altitude, image resolution, image location conversion records, mine safety management requirements, and historical manual review results. The slope range and step boundary correspond to linear contours, slope gaps, newly added crushed stone ranges, or changes in slope texture direction; the transport road range corresponds to block contours, vehicle contours, personnel contours, or continuous bright areas; the drainage ditch entrance corresponds to continuous bright areas, scattered object contours, or drainage ditch entrance obstruction areas; the equipment operation range, prohibited entry range, and blasting warning range correspond to personnel contours, vehicle contours, or equipment operation range occupancy contours.

[0056] A suspected hazard source record is generated when the changed area is located within the slope range, step boundary, transport road range, drainage ditch inlet, equipment operation range, prohibited entry range, or blasting warning range, and the changed area's outline record, area record, length record, width record, color change value, grayscale change value, edge outline change value, or texture direction change value meet the field combination and value range of the corresponding area type in the hazard source extraction condition record. The suspected hazard source record includes the suspected record number, changed area number, image group number, image number, main area number, associated area number, area type, suspected hazard source type, suspected type marker, site location, image outline, area record, length record, width record, direction record, change source field, associated object number, and entry confirmation processing marker. The associated object number is the slope range number, transport road range number, drainage ditch inlet number, equipment operation range number, prohibited entry range number, or blasting warning range number corresponding to the changed area.

[0057] When the changed area is located within the slope, the system reads the linear contour record, slope gap record, newly added gravel range record, slope texture direction change record, and step boundary position to generate suspected crack records, suspected spalling records, or suspected slope slippage records. The linear contour record is determined by the length and width of the bounding rectangle of the changed area, the skeleton length record, and the edge continuity record. The slope gap record is formed by the location of the slope surface contour interruption, the gap range, and the gap edge line. The newly added gravel range record is formed by the newly added blocky changed area within the slope toe or step platform. The slope texture direction change record is formed by the difference in texture line direction within the same slope range in the current inspection image and the comparison image. Suspected crack records include the linear contour number, length record, width record, direction record, and the corresponding step boundary; suspected spalling records include the slope gap location, gap range, newly added gravel range, and slope color change fields; suspected slope slippage records include the slope texture offset range, slope toe change location, slippage direction record, and the corresponding step boundary.

[0058] When the changed area is located within the transport road area, the system reads block contour records, road occupancy records, vehicle contour records, personnel contour records, continuous bright area records, and road intersection locations to generate suspected rockfall occupancy records, suspected vehicle stoppage records, suspected personnel trespassing records, or suspected water accumulation records. Block contour records are determined by the bounding rectangle of the changed area, contour closure records, and area records. Road occupancy records are formed by the overlap between the changed area and the transport road area. Vehicle contour records are formed by the vehicle body rectangle, wheel positions, or vehicle color areas. Personnel contour records are formed by the human body shape, reflective clothing color areas, or infrared heat source areas; when no infrared image is configured in the mining area, the infrared heat source area field is not read in the personnel contour records. Continuous bright area records are formed by brightness change records, texture change records, and boundary closure records. Records of suspected rockfalls obstructing the road are written into the block outline, road area occupancy record, and vehicle passage range correspondence; records of suspected vehicle stoppages are written into the vehicle outline, vehicle center position, and adjacent image position change record; records of suspected personnel accidentally entering the area are written into the personnel outline, personnel on-site position, and the correspondence between the prohibited entry area or equipment operation area; records of suspected water accumulation are written into the continuous bright area, boundary closure record, texture change record, and drainage ditch inlet correspondence.

[0059] When the changed area is located in a spoil heap or stockpile area, the records of changes in the stockpile boundary, the range of newly added debris, and the changes in the slope toe position are read to generate a suspected stockpile slippage record or a suspected debris record. The stockpile boundary change record is formed by the difference between the stockpile boundary line in the current image and the stockpile boundary line in the comparison image; the range of newly added debris is formed by new blocky changes on the outside of the stockpile or at the slope toe position; the changes in the slope toe position are formed by the difference in the position of the slope toe line in the on-site coordinates. The suspected stockpile slippage record includes the range of changes in the stockpile boundary, the changes in the slope toe position, and the slippage direction; the suspected debris record includes the on-site location of the debris, the outline of the debris, and the corresponding transport road range or drainage ditch inlet number.

[0060] When the changed area is located within the equipment's operating range or a prohibited area, the system reads personnel outline records, vehicle outline records, equipment outline records, and area boundary relationships to generate suspected personnel entry records, suspected vehicle occupancy records, or suspected equipment operating range occupancy records. The suspected personnel entry record includes the personnel's on-site location, personnel outline, entered area number, and image number; the suspected vehicle occupancy record includes the vehicle outline, vehicle on-site location, occupied area number, and the corresponding road intersection; the suspected equipment operating range occupancy record includes the changed area outline, equipment operating range number, overlapping location, and occupancy source field.

[0061] The fixed object verification request record is generated when there is a positional, contour, or color correspondence between the changed area and the fixed object registration field in the mine site object registration record. The positional correspondence result is determined based on the overlap ratio between the changed area's site coordinates and the fixed object's site coordinates; the contour correspondence result is determined based on the contour overlap ratio between the changed area's contour and the fixed object's registered contour, the boundary offset, and the difference in the circumscribed rectangle's direction; the color correspondence result is determined based on the color difference between the changed area's color change location and the fixed object's contour edge or surface area. The ranges for the positional overlap ratio, contour overlap ratio, boundary offset, circumscribed rectangle's direction difference, and color difference are determined by the fixed object's manually calibrated image, pre-inspection confirmation image, camera calibration results, image position conversion errors, and historical manual verification results.

[0062] When at least one of the location correspondence result, contour correspondence result, or color correspondence result reaches the corresponding pass range, a fixed object verification request record is generated. The fixed object verification request record records the verification request number, suspected record number, changed area number, fixed object number, fixed object type, location correspondence result, contour correspondence result, color correspondence result, fixed object overlap ratio, partial correspondence status, and entry source verification mark. If only fixed object location correspondence exists, but the changed area contour, color, or edge range is inconsistent with the fixed object registration fields, the fixed object verification request record writes a partial correspondence status and retains the suspected hazard source record.

[0063] Fixed object overlap is represented as: ; in, Indicates the overlap ratio between the changing area and the fixed object; This indicates the current area of ​​the site corresponding to the area of ​​change. This indicates the on-site area of ​​the fixed object in the fixed object registration record; This indicates the overlapping area between the currently changing area and the fixed object's on-site area; This indicates the overlapping area between the currently changing area and the fixed object's on-site area; Indicates the area of ​​the currently changing region; This indicates calculating the area of ​​the region within the parentheses; This represents the non-zero area correction value in the calculation of the overlap ratio of fixed objects. It is determined according to the rules for determining non-zero area correction values ​​and is used to avoid... Zero values ​​or excessively small areas of change can cause calculation errors. Take the area of ​​the scene corresponding to one pixel after the image position is converted, or take the area of ​​the minimum effective change area preset by the image processing program. Write a fixed object verification request record to distinguish between fixed objects and newly added / changed areas during source verification processing. This is not considered a direct result of ruling out suspected sources of danger.

[0064] An acquisition interference verification request record is generated when a changing area corresponds to an image acquisition anomaly record or interference behavior within an image group. Sources of acquisition interference include dust, smoke, shadows, reflections, occlusion, exposure anomalies, and edge distortion. An acquisition interference verification request record is generated when the changing area is located within an occlusion area, exposure anomaly area, or edge distortion area in an image acquisition anomaly record. An acquisition interference verification request record is also generated when the changing area does not fall within an image acquisition anomaly record, but forms a concentrated brightness change, boundary diffusion, discontinuous boundary, low-texture occlusion, or brightness change consistent with the projection direction of a fixed object within the same image group. The acquisition interference verification request record includes the verification request number, suspected record number, changing area number, anomaly image number, acquisition interference source, interference area location, boundary change record, brightness change record, visible persistence record, and entry source verification marker.

[0065] Dust and smoke are recorded for boundary changes, low-texture occlusion range, and color change range; shadows are recorded for brightness change direction, fixed object projection direction, and brightness region position changes in adjacent images; reflections are recorded for concentrated brightness changes, bright area position, and visible continuity; edge distortion is recorded for image edge region contour stretching and fixed object contour offset. Interference verification request records are only used as entry points for source verification and are not directly deleted as suspected hazard records. Whether a suspected hazard record is excluded is determined jointly by source verification processing and continuous processing.

[0066] If an image change area does not meet any suspected hazard conditions, but corresponds to a fixed object or acquisition interference, only a fixed object verification request record or acquisition interference verification request record is generated; no suspected hazard record is generated. If an image change area does not meet any suspected hazard conditions and does not correspond to a fixed object or acquisition interference, a non-early warning change retention record is generated. The non-early warning change retention record includes the change area number, image number, area number, change area location, change source field, and the reason for not proceeding to the confirmation process. Non-early warning change retention records are not processed for hazard confirmation.

[0067] S3. Based on the suspected hazard source record, fixed object verification request record, and acquisition interference verification request record, and combined with the effective adjacency relationship in the image group processing record, perform suspected area continuation preprocessing, source verification, and continuous confirmation processing to generate source verification result record, hazard source continuous record, hazard source confirmation record, or intermittent pending confirmation record.

[0068] The processor reads suspected hazard source records, fixed object verification request records, data acquisition interference verification request records, image acquisition anomaly records, image group processing records, image location conversion records, and image change area records. It reads suspected hazard source records according to the suspected record number, and also reads fixed object verification request records and data acquisition interference verification request records under the same change area number. When multiple suspected type markers exist for the same change area, a suspected type processing list is generated. This list includes suspected cracks, suspected spalling, suspected slope slippage, suspected rockfall obstruction, suspected vehicle parking, suspected personnel trespassing, suspected water accumulation, suspected pile slippage, suspected scattered objects, suspected vehicle occupation, and suspected equipment operation area occupation. Each item in the suspected type processing list stores the source verification result. When a suspected type is excluded, it does not affect the continued processing of other suspected types under the same change area number.

[0069] The processor reads the current and previous images with valid adjacency relationships from the image group processing records, and generates a suspected area continuation preprocessing record based on the area number, main area number, associated area number, suspected hazard type, site location, and image time sequence in the suspected hazard source record. The suspected area continuation preprocessing record records the current suspected record number, the previous suspected record number, the current image number, the previous image number, the suspected hazard type, the site center point coordinates, the continuation distance, the allowable continuation distance value, the location continuation condition result, and the corresponding image group number.

[0070] Whether suspected regions in adjacent images meet the location continuation condition is determined based on the continuation distance: ; in, Indicates the first The suspected area in the image is related to the first... The distance between suspected areas in the image; Indicates the sequence number of the currently processed image in the image group; This indicates the sequence number of the previous adjacent image in the image group that is currently being processed; Indicates the first The converted horizontal coordinates of the suspected area center point in the image; Indicates the first The converted longitudinal coordinates of the suspected area center point in the image; Indicates the first The converted horizontal coordinates of the suspected area center point in the image; Indicates the first The converted longitudinal coordinates of the center point of the suspected area in the image. , , and All coordinates are obtained from image location conversion records, and the coordinate units are consistent with the on-site coordinate units in the mine site object registration records. When the changed area spans multiple area numbers, the center point is calculated according to the changed area portion within the main area number, and the associated area number is written into the suspected area continuation preprocessing record.

[0071] The allowable value for connection distance is expressed as follows: ; in, This indicates the allowable distance for suspected areas in adjacent images to be consecutively identified as the same hazard source. Indicates the first Coordinate conversion error of the image; Indicates the first Coordinate conversion error of the image; This indicates the permissible on-site displacement velocity between adjacent images for the corresponding hazard type; Indicates the first Zhang Image and the First The time difference between the capture of the images. and All data is read from the image position conversion record, and the coordinate conversion error in the image position conversion record already includes the UAV positioning error, camera installation error, gimbal angle recording error, local conversion surface error, and fixed object contour matching error. Therefore, the UAV positioning error will not be added separately. Calculated from the capture time of the two images. Corresponding to cracks, spalling, falling rocks, water accumulation, and mass slippage. Determined based on the range of location changes in historical review samples; personnel and vehicles corresponding to... The speed limits for personnel movement, vehicle speed limits, and inspection image samples are determined based on the mine operation management rules. It is only used for connecting suspected records of the same type in adjacent images, and is not used to directly generate early warning results.

[0072] Before calculating the allowable value for the continuity distance, the processor generates a displacement velocity record for each hazard type. Suspected cracks, suspected spalling, suspected slope slippage, and suspected mass slippage are all fixed-position changes, with an allowable on-site displacement velocity of 0 to 0.1 meters per second. Suspected rockfall obstruction, suspected scattered objects, and suspected water accumulation are on-site targets with minimal positional changes, with an allowable on-site displacement velocity of 0 to 0.3 meters per second. Suspected personnel intrusion is a moving target, with an allowable on-site displacement velocity of 0.3 to 1.8 meters per second. Suspected vehicle occupancy is a low-speed or stationary vehicle, with an allowable on-site displacement velocity of 0 to 1.5 meters per second. Suspected vehicle occupancy corresponds to vehicles entering road intersections, equipment operating areas, or prohibited areas, with an allowable on-site displacement velocity of 0 to 5.6 meters per second. Suspected equipment operating area occupancy corresponds to changes in the position of equipment or occupants, with an allowable on-site displacement velocity of 0 to 1 meter per second. All values ​​are determined by mine operation management rules, road speed limits, the time difference between adjacent UAV image captures, historical verification samples, and manually annotated continuous image samples. When the speed limit given in the mine operation management rules is lower than the upper limit of the corresponding range, the speed limit in the mine operation management rules shall be used as the upper limit of the allowable on-site displacement speed. This allowable on-site displacement speed shall be written into the suspected area continuation preprocessing record for calculation of the allowable continuation distance.

[0073] when Not greater than When the current suspected area and the previous suspected area are both recorded in the suspected area continuation preprocessing record to meet the location continuation conditions; when Greater than When the location connection condition is not met, write the current connection distance and the allowed connection distance value.

[0074] The processor performs fixed object source verification based on the fixed object verification request record. Fixed object source verification reads the fixed object number, fixed object type, location correspondence result, outline correspondence result, color correspondence result, and fixed object overlap ratio. When the location correspondence result, outline correspondence result, and color correspondence result of the same suspected record all point to the same fixed object number, and the fixed object overlap ratio meets the fixed object exclusion ratio requirement, a fixed object exclusion record is generated, and the corresponding suspected type does not enter continuous confirmation processing. The fixed object exclusion ratio requirement is determined by the manually calibrated fixed object image, the image confirmed before inspection, image position conversion error, and historical manual review results. When the location correspondence result points to the fixed object number, but the outline correspondence result or color correspondence result does not reach the corresponding acceptable range, or the changed area exceeds the fixed object's on-site coordinate range and forms a new area, a fixed object pending confirmation record is generated, and the corresponding suspected type continues to enter continuous confirmation processing.

[0075] The processor performs acquisition interference source verification based on the acquisition interference verification request record and image acquisition anomaly record. The acquisition interference source verification reads the fields for acquisition interference source, interference area location, boundary change record, brightness change record, visibility continuity record, abnormal area outline, and abnormal source. If the change area of ​​a suspected hazard source record falls within the abnormal area location in the image acquisition anomaly record, the acquisition interference source and abnormal source fields are consistent, and the suspected area continuation preprocessing record does not meet the location continuation condition, an acquisition interference exclusion record is generated, and the corresponding suspected type does not enter continuous confirmation processing. If a suspected hazard source record has an acquisition interference verification request record, but the suspected area meets the location continuation condition, or the change area outline, area, length, and width records are retained in adjacent images, an acquisition interference pending confirmation record is generated, and the corresponding suspected type continues to enter continuous confirmation processing.

[0076] When a suspected hazard source has both a fixed object verification request record and a data acquisition interference verification request record, a fixed object verification result and a data acquisition interference verification result are generated separately. If either verification result results in an exclusion record, and the suspected area does not meet the location continuation condition, the corresponding suspected type will not enter continuous confirmation processing; if both verification results are pending confirmation, or if either verification result is pending confirmation and the suspected area meets the location continuation condition, the corresponding suspected type will enter continuous confirmation processing.

[0077] For suspected crack and spalling records, the processor performs slope texture source verification. This verification reads the texture direction change values, length records, width records, and contour records from the slope texture reference record and image change area record. The slope texture reference record is generated from manually confirmed images before the inspection or from the previous inspection cycle. If a suspected crack or spalling record corresponds to the rock layer texture direction and slope color band position in the slope texture reference record, and the suspected area does not meet the positional continuity condition, or if the length, width, and contour records are not retained in adjacent images, a slope texture exclusion record is generated. If a suspected crack or spalling record corresponds to the rock layer texture direction, but the suspected area meets the positional continuity condition, and the length, width, or contour records are retained in adjacent images, a slope texture pending confirmation record is generated.

[0078] Source verification results are written into the source verification result record. The source verification result record includes the suspected record number, change area number, suspected hazard source type, fixed object verification result, data collection interference verification result, slope texture verification result, exclusion record number, pending confirmation record number, and whether it has entered continuous confirmation processing. Suspected records for which no fixed object exclusion record, data collection interference exclusion record, or slope texture exclusion record has been generated are entered into continuous confirmation processing.

[0079] The continuous verification process reads the valid adjacency relationships in the image group processing records and pairs suspected records in adjacent images within the same image group according to the area number, suspected hazard source type, on-site location, image time sequence, and change area number. During pairing, the current suspected record and the suspected record in the previous image must have the same area number or a related area number, the same suspected hazard source type, or belong to the same suspected type processing list under the same change area number, and neither must have formed an exclusion record.

[0080] For suspected areas that meet the location continuation criteria, continue to calculate the area change ratio: ; in, Indicates the first The suspected area in the first image is relative to the first... The area change ratio of the suspected region in the image; Indicates the first Area of ​​the suspected region in the image; Indicates the first Area of ​​the suspected region in the image; Indicates the first The area of ​​the suspected region in the image is related to the first... The absolute value of the difference between the areas of suspected regions in the images; This represents the non-zero area correction value used in the calculation of the area change ratio. It is determined according to the rules for identifying non-zero area correction values ​​and is used to avoid... Zero values ​​or excessively small area records can cause calculation errors. and All are obtained from the area of ​​the changed region recorded in the image change region record. Take the area of ​​the scene corresponding to one pixel after the image position is converted, or take the area of ​​the minimum effective change area preset by the image processing program. It is used to represent the area change of the current image relative to the previous image, but not for comparing non-adjacent images.

[0081] when When the area change falls within the allowable range for the corresponding hazard source type, the current suspected area and the previous suspected area are recorded in the same hazard source continuous record. The allowable area change range is determined based on continuous image samples of similar hazard sources, UAV flight altitude, image resolution, and manual verification results. Cracks, spalling, and pile slippage are read according to the area change range of fixed-location hazard sources; personnel and vehicles are read according to the area change range of moving objects; the area change range corresponding to dust, smoke, reflection, and shadows is not used as the basis for hazard source confirmation, but only as a reference field for verifying the source of interference.

[0082] when Not greater than ,but If the area change does not fall within the allowable range for the corresponding hazard type, an area abrupt change to be confirmed record is generated. This record includes the suspected record number, current image number, previous image number, current area, previous area, area change percentage, hazard type, and reason for confirmation. This record is not included in the hazard confirmation record but can be used as input for prompting continued data collection.

[0083] The continuous hazard source record includes hazard source number, hazard source type, area number, main area number, associated area number, first image number, last image number, number of consecutive images, site location range, area change record, length change record, width change record, direction change record, source verification result, and fields to be confirmed. The number of consecutive images is calculated based on the number of images forming valid connections within the same hazard source continuous record.

[0084] The continuous verification ratio for continuous recording of hazard sources is expressed as follows: ; in, This indicates the proportion of consecutively confirmed images of the same hazard source recorded consecutively within a group of images. This indicates the number of images in continuous records of the same hazard source that meet the conditions of location continuity and area change, and for which no fixed object exclusion record, acquisition interference exclusion record, or slope texture exclusion record has been formed. This indicates the number of images within the same image group that cover the location range of the hazard source and can be processed. Images covering the location range of the hazard source are determined by the valid adjacency relationships and image location conversion records in the image group processing log. When the value is zero, the corresponding image group does not enter the continuous confirmation process; when When it is not zero, calculate . It is obtained by counting the number of valid consecutive images in the continuous record of the hazard source.

[0085] Before generating a hazard source confirmation record, the processor generates a continuous confirmation condition record. The continuous confirmation condition record is written with the allowable area change range and continuous confirmation ratio requirements according to the suspected hazard source type. Suspected cracks, suspected spalling, and suspected pile slippage are fixed-location hazard sources, with an allowable area change range of 0 to 0.4 and a continuous confirmation ratio requirement of 0.6 to 0.85. Suspected rockfall obstruction, suspected scattered objects, and suspected water accumulation are on-site targets with minor location changes, with an allowable area change range of 0 to 0.5 and a continuous confirmation ratio requirement of 0.55 to 0.8. Suspected personnel intrusion is a moving object, with an allowable area change range of 0 to 0.8 and a continuous confirmation ratio requirement of 0.4 to 0.7. Suspected vehicle parking and suspected vehicle occupation are vehicle targets, with an allowable area change range of 0 to 0.7 and a continuous confirmation ratio requirement of 0.45 to 0.75. Suspected equipment operation area occupation is a change in equipment or occupants, with an allowable area change range of 0 to 0.6 and a continuous confirmation ratio requirement of 0.5 to 0.8. The allowable range of area change and the continuous confirmation ratio requirements are determined by continuous image samples of similar hazardous sources, UAV flight altitude, image resolution, image coverage relationship of inspection route, and manual review results, and are recorded in the continuous confirmation condition record.

[0086] when The area change falls within the allowable range for the corresponding suspected hazard source type in the continuous confirmation condition record, and When the continuous confirmation ratio requirement for the corresponding suspected hazard source type is met, and no fixed object elimination record, data collection interference elimination record, or slope texture elimination record is formed in the continuous hazard source record, a hazard source confirmation record is generated; when When the area change exceeds the allowable range, a record of area abrupt change to be confirmed is generated; when If the required continuous verification ratio is not met, an intermittent pending verification record is generated. This clarifies the allowable range of area changes and the specific value range of the continuous verification ratio requirement, enabling the continuous verification process for hazard sources to be reproduced.

[0087] The hazard source confirmation record includes the hazard source number, hazard source type, area number, main area number, associated area number, site location range, confirmed image number, number of consecutive images, consecutive confirmation ratio, source verification results, associated object number, and confirmation status. The confirmed image number includes the image numbers that contributed to the formation of the hazard source's continuous record. The associated object number corresponds to the slope range number, transport road range number, drainage ditch inlet number, equipment operation range number, prohibited entry range number, or blasting warning range number in the hazard source confirmation record.

[0088] For hazard sources that do not meet the continuous confirmation ratio requirement, continuous records are generated as intermittent pending confirmation records. Intermittent pending confirmation records include the suspected record number, hazard source type, area number, reason for interruption, area to be supplemented with images, object to be reviewed, and suggested re-collection requirements. Reasons for interruption include insufficient number of valid adjacent relationships, area abrupt changes to be confirmed, fixed objects to be confirmed, collection interference to be confirmed, slope texture to be confirmed, and insufficient area correspondence. Intermittent pending confirmation records are not included in the early warning output but are used to generate prompts for continued collection or manual review.

[0089] S4. By using the hazard source confirmation record and the mine site object registration record, the hazard source site area and the mine object area are matched. Based on the object entry ratio, distance record, change record and source verification results, hazard source impact record, hazard source output result and early warning evidence record are generated.

[0090] The system reads hazard source confirmation records, intermittent pending confirmation records, mine site object registration records, image location conversion records, hazard source continuous records, and source verification result records. These records are sorted by hazard source number, hazard source type, area number, main area number, associated area number, and confirmed image number to generate a hazard source early warning processing record. This record includes the hazard source number, hazard source type, confirmed image number, image group number, main area number, associated area number, site location range, continuous confirmation ratio, source verification result, associated object number, and corresponding processing marker for the entering object. Hazard source confirmation records are processed for immediate handling early warning or on-site review early warning; intermittent pending confirmation records are not processed for immediate handling early warning or on-site review early warning, but only generate a continue acquisition prompt. The continue acquisition prompt includes the area of ​​image to be supplemented, suggested shooting direction, suggested shooting height, hazard source type requiring review, reason for intermittent status, and corresponding suspected record number.

[0091] The system reads the scope of transportation roads, road intersections, drainage ditch entrances, equipment operating areas, personnel access areas, prohibited areas, blasting warning areas, slope areas, and bench boundaries from the mine site object registration record and generates a mine object processing record for this stage of the assessment. The mine object processing record includes object type, object number, object site area, object boundary, object source, and corresponding area number. The scope of transportation roads, road intersections, equipment operating areas, personnel access areas, prohibited areas, blasting warning areas, and drainage ditch entrances are treated as regional objects; bench boundaries and slope boundaries are treated as linear objects. Regional objects are used to calculate the object entry ratio, while linear objects are used to generate distance records and do not participate in the object entry ratio calculation.

[0092] Vehicle passage range records are generated based on the transport road range, road edge lines, and mine vehicle passage management requirements. These records include road number, vehicle passage boundary, road intersection number, direction of travel, and source of generation. Low-lying work area records are generated based on area boundaries, drainage ditch layout data, on-site elevation data, or manually marked low-lying areas. These records include area number, low-lying area boundary, associated drainage ditch inlet number, and source of generation. Slope rockfall impact range records are generated based on slope range, bench boundary, transport road range, and manually marked rockfall impact area. These records include slope range number, bench boundary number, transport road range number, impact range boundary, and source of generation. Vehicle passage range records, low-lying work area records, and slope rockfall impact range records are only used as input data for location mapping between hazard sources and mine objects.

[0093] Hazardous source site areas are generated based on hazard confirmation records. The areas occupied by falling rocks, accumulated water, vehicles, personnel, scattered objects, and equipment operation ranges are formed by converting the site location range and image outline in the hazard confirmation record into an image location conversion record. Linear hazard sources corresponding to cracks, spalling boundaries, and changes in pile boundaries are generated based on length, width, and direction records; if the width record is missing, the site width corresponding to one pixel in the image location conversion record is read as the minimum width record. The minimum width record is only used to form a calculable hazard source site area and is not used as the result of hazard source size determination. Hazardous source site areas are not re-extracted from the original image.

[0094] The hazard source site area is mapped to the mining object processing records to generate a hazard source object mapping record. This record includes the hazard source number, hazard source type, object type, object number, hazard source site area, object site area, object boundary, object entry ratio, distance record, overlap record, continuity record, and change record. When the same hazard source confirmation record corresponds to multiple mining objects, multiple hazard source object mapping records are generated, and multiple object numbers and corresponding judgment criteria are written into the hazard source output results.

[0095] The entry of hazardous sources into the area is calculated based on the proportion of objects entering: ; in, This indicates the proportion of areas where hazards have been identified that have entered the mining area. This indicates the hazardous source site area corresponding to the hazardous source confirmation record; This refers to the mining object area in the mining site object registration record or mining object processing record. The mining object area includes the transportation road area, road intersection area, equipment operation area, personnel passage area, prohibited entry area, drainage ditch entrance area, blasting warning area, vehicle passage area, and the impact area of ​​low-lying working face or slope rockfall. This indicates the overlapping area between the hazardous source site and the mining target area; This indicates the overlapping area between the hazardous source site area and the mining target area; Indicates the area of ​​the hazardous source site; This indicates calculating the area of ​​the region within the parentheses; This represents the non-zero area correction value for the object in the scaling calculation. The value is determined according to the rules for determining non-zero area correction values ​​and is used to avoid... The calculation is abnormal if the value is zero or the area of ​​the hazardous source site is too small. Take the area of ​​the scene corresponding to one pixel after the image position is converted, or take the minimum effective hazard source area preset by the image processing program. Write it to the record corresponding to the hazard source object, but do not output it as a separate hazard source result.

[0096] Before generating the corresponding record for the hazardous source object, the processor generates an object occupancy judgment condition record. The object occupancy judgment condition record is written with the object entry ratio requirement and distance requirement according to the object type. For transport road areas, the object entry ratio requirement is 0.1 to 0.3, and the distance requirement is 0 to 2 meters; for road intersection areas, the object entry ratio requirement is 0.05 to 0.2, and the distance requirement is 0 to 3 meters; for equipment operation areas, the object entry ratio requirement is 0.05 to 0.25, and the distance requirement is 0 to 2 meters; for personnel passage areas, the object entry ratio requirement is 0 to 0.05 to 0.2, and the distance requirement is 0 to 1.5 meters; the object entry ratio requirement for prohibited entry areas and blasting warning areas is also specified. The following parameters are required: a value greater than zero, and a distance requirement of 0 to 5 meters; the entry ratio of objects corresponding to the drainage ditch inlet area should be 0.1 to 0.4, and the distance requirement of 0 to 1.5 meters; the entry ratio of objects corresponding to the vehicle passage area record should be 0.1 to 0.3, and the distance requirement of 0 to 2 meters; the entry ratio of objects corresponding to the low-lying work surface record should be 0.1 to 0.5, and the distance requirement of 0 to 2 meters; the entry ratio of objects corresponding to the slope rockfall impact area record should be 0.05 to 0.25, and the distance requirement of 0 to 3 meters.

[0097] The object entry ratio requirement is determined by the mine site object registration records, mine road traffic management requirements, vehicle passage width, equipment operation boundaries, drainage ditch design width, mine safety boundary documents, and manual review samples. The distance requirement represents the maximum distance between the boundary of the hazardous source site area and the boundary of the mine object that is allowed to trigger an on-site review warning. When the distance record is less than or equal to the distance requirement for the corresponding object type, a distance-met status is written; when the distance record is greater than the distance requirement for the corresponding object type, a distance-not-met status is written. When the object entry ratio reaches the object entry ratio requirement for the corresponding object type, the corresponding record for the hazardous source object is written as "object occupied"; when the object entry ratio does not reach the object entry ratio requirement for the corresponding object type, but the distance record is less than or equal to the distance requirement for the corresponding object type, the corresponding record for the hazardous source object is written as "on-site review"; when the object entry ratio does not reach the object entry ratio requirement for the corresponding object type, and the distance record is greater than the distance requirement for the corresponding object type, the corresponding record for the hazardous source object is written as "unoccupied," and the distance record and change record are retained.

[0098] Distance records are generated based on the shortest on-site distance between the hazard source area and the mining object. The distance between the crack endpoint and the bench boundary, slope boundary, or drainage ditch inlet is calculated by reading the on-site coordinates of the crack endpoint and the object boundary coordinates. The distance between falling rocks, vehicles, personnel, accumulated water, and debris and the transport road area, equipment operating area, prohibited entry area, drainage ditch inlet, or slope rockfall impact area is calculated by reading the boundary of the hazard source area and the boundary of the object area. Distance records include the hazard source number, object number, distance value, distance calculation object, calculation image number, and coordinate source. Distance requirements are determined by mine road access management requirements, equipment operating area boundaries, prohibited entry area boundaries, drainage ditch inlet location errors, and slope inspection requirements, and are written into the object distance requirement record according to object type.

[0099] The change records are generated based on the initial and final confirmation images from the hazard source confirmation records. Crack change records include length differences, width differences, direction records, and endpoint position changes; water accumulation change records include area differences, boundary position differences, and the correspondence between drainage ditch inlets; shovel slip change records include shovel boundary position differences, changes in the location of scattered debris, and changes in the slope toe position; vehicle dwelling change records include changes in the vehicle center point position and the number of consecutive images; and personnel intrusion change records include changes in the personnel's location, the number of consecutive images, and the proportion of personnel entering the site. Change records are read from the hazard source confirmation records, hazard source continuous records, and image position conversion records; image change extraction is not performed again.

[0100] Hazard source impact records are generated based on the corresponding records, distance records, and change records of the hazard source objects. The hazard source impact record includes the hazard source number, hazard source type, object type, object number, object entry ratio, object occupancy status, distance value, change field, number of consecutive images, associated mine objects, and entry / exit classification markers. Associated mine objects include the transportation road area, road intersection area, equipment operation area, personnel access area, prohibited entry area, blasting warning area, drainage ditch entrance, slope area, and bench boundary. When the source verification result record corresponding to the hazard source confirmation record contains fields requiring confirmation, the hazard source impact record includes the review requirement field; suspected types that have already formed exclusion records are not included in either the hazard source confirmation record or the hazard source impact record.

[0101] Hazard source output results are generated based on hazard source impact records. These output results are categorized into immediate response warnings, on-site review warnings, and continued data collection prompts. When the object entry ratio meets the requirement, it is processed according to the object occupancy status. If the object entry ratio does not meet the requirement, but the distance record meets the distance requirement corresponding to the object type, an on-site review warning is generated. If only intermittent pending confirmation records, area abrupt change pending confirmation records, data collection interference pending confirmation records, or slope texture pending confirmation records exist, a continued data collection prompt is generated. When the same hazard source confirmation record simultaneously meets multiple output categories, the output category is determined in the order of immediate response warning, on-site review warning, and continued data collection prompt, and all object numbers, triggering criteria, and corresponding record numbers are written into the hazard source output results.

[0102] When a personnel trespassing confirmation record creates an object occupancy status with a prohibited entry area, equipment operation area, or blasting warning area, a personnel trespassing warning result is generated. The personnel trespassing warning result includes the personnel hazard source number, personnel location, entrant type, entrant number, entrant percentage, confirmed image number, number of consecutive images, and handling prompts. Handling prompts include one or more of the following: personnel evacuation, on-site confirmation, or re-collection. If a personnel trespassing confirmation record does not create an object occupancy status, but the source verification result contains a field requiring confirmation, an on-site review warning is generated.

[0103] When a rockfall obstruction confirmation record forms an object occupancy status with records of the transport road area, road intersection area, or vehicle passage area, a rockfall obstruction warning result is generated. The rockfall obstruction warning result includes the rockfall hazard source number, road object number, object entry ratio, rockfall site area, confirmation image number, number of consecutive images, and handling prompts. Handling prompts include one or more of the following: road clearing, vehicle detour, or on-site confirmation. If a rockfall obstruction confirmation record does not form an object occupancy status, but the distance record meets the distance requirements corresponding to the transport road, an on-site review warning is generated.

[0104] When a crack confirmation record forms distance and change records with the boundary of a step, the boundary of a slope range, or the entrance of a drainage ditch, a slope crack early warning result is generated. The slope crack early warning result includes the crack hazard source number, the crack's location, the distance record to the crack endpoint, the crack length difference, the crack width difference, the confirmed image number, the number of consecutive images, and a handling prompt. The handling prompt includes one or more of the following: slope inspection, personnel avoidance, re-acquisition, or on-site confirmation. If the crack confirmation record only forms a field for unconfirmed slope texture and does not form a length or width difference, a prompt to continue acquisition is generated. When a slope slippage confirmation record forms distance, change, or object occupancy records with the boundary of a slope range, the boundary of a step, the range of a transport road, or the impact range of a rockfall on a slope, a slope slippage early warning result is generated. The slope slippage early warning result includes the slope slippage hazard source number, the slope texture offset range, the change location of the slope toe, the slippage direction record, the associated slope range number, the confirmed image number, the number of consecutive images, and a handling prompt. The action prompts should be written into one or more of the following: slope inspection, personnel avoidance, road traffic restriction, re-collection, or on-site confirmation.

[0105] When a vehicle occupancy confirmation record, along with records of road intersections, drainage ditch inlets, equipment operation areas, or slope rockfall impact areas, forms an object occupancy status, a vehicle occupancy warning result is generated. The vehicle occupancy warning result includes the vehicle hazard source number, vehicle location, occupant object number, change in vehicle center point position, number of consecutive images, object entry ratio, and handling prompts. Handling prompts include one or more of the following: vehicle removal, road clearing, or on-site confirmation. If a vehicle occupancy confirmation record does not form an object occupancy status, but the number of consecutive images meets the vehicle occupancy judgment requirements, an on-site review warning is generated.

[0106] When the water accumulation confirmation record, along with records of the transportation road area, drainage ditch inlet area, or low-lying work surface, forms an object occupancy status, a water accumulation impact warning result is generated. The water accumulation impact warning result includes the water accumulation hazard source number, water accumulation site area, object entry ratio, area difference, drainage ditch inlet correspondence, confirmation image number, and handling prompts. Handling prompts include one or more of the following: drainage treatment, road traffic restriction, re-collection, or on-site confirmation. If the water accumulation confirmation record does not form an object occupancy status but does form an area difference or boundary position difference, an on-site review warning is generated.

[0107] When a stockpile slip confirmation record or a debris impact confirmation record creates an object occupancy status with the transport road area, drainage ditch inlet area, or stockpile area boundary, a stockpile slip warning result or a debris impact warning result is generated. The stockpile slip warning result includes the hazard source number, stockpile boundary position difference, slope toe position change, object entry ratio, confirmation image number, and handling prompts. The debris impact warning result includes the debris hazard source number, debris site area, associated transport road number, associated drainage ditch inlet number, object entry ratio, and handling prompts. Handling prompts include one or more of the following: clearing debris, inspecting stockpile boundaries, restricting access, or on-site confirmation.

[0108] When the equipment work area occupancy confirmation record and the equipment work area form an object occupancy status, an equipment work area occupancy warning result is generated. The equipment work area occupancy warning result includes the hazard source number, occupant object number, object entry ratio, confirmed image number, number of consecutive images, and handling prompts. Handling prompts include one or more of the following: equipment pause, personnel confirmation, area cleanup, or re-collection. If the equipment work area occupancy confirmation record contains a field requiring confirmation due to data acquisition interference, a field review warning is generated; an immediate handling warning is not directly generated.

[0109] After generating the hazard source output result, a warning evidence record is generated simultaneously. The warning evidence record is stored in the following order of numbering: original image, image change area, suspected hazard source record, verification request record, source verification result, continuous hazard source record, confirmed hazard source record, hazard source impact record, and hazard source output result. The warning evidence record includes the evidence record number, original image number, hazard source area screenshot number, image change area record number, suspected hazard source record number, fixed object verification request record number, data acquisition interference verification request record number, source verification result record number, continuous hazard source record number, confirmed hazard source record number, hazard source impact record number, and hazard source output result number. The hazard source area screenshot is taken from the inspection image corresponding to the confirmed image number, and the screenshot range corresponds to the on-site location range in the hazard source confirmation record.

[0110] The early warning evidence record also preserves the entry ratio of the objects involved in the judgment, distance records, change records, continuous confirmation ratio, and source verification results. When mine safety personnel conduct a review, they can read the original image, image change area record, source verification result record, hazard source confirmation record, and hazard source impact record according to the numbering relationship in the early warning evidence record. The manual review results generate a handling retention record. The handling retention record includes the hazard source output result number, review personnel, review time, on-site handling result, image review conclusion, and remarks field. The handling retention record is saved separately and is not part of the automatic update process in this method.

[0111] To verify the effectiveness of source verification processing and early warning evidence association processing in this invention, simulation experiments were conducted using images from mine drone inspections. The experimental images were configured according to the mine site object registration records, including slope range, transport road range, drainage ditch entrance range, equipment operation range, prohibited entry range, fixed object outline, and image acquisition anomaly areas. Five types of interference scenarios were also set: near fixed objects, dust and smoke, shadows and reflections, edge distortion, and mixed interference. During processing, image change area records were first generated, followed by suspected hazard source records, fixed object verification request records, and acquisition interference verification request records. Source verification result records were then generated after source verification processing. These source verification result records were correlated with manually verified sources to obtain the source verification accuracy rate under different interference scenarios, forming a... Figure 5 . Figure 5 The present invention processing in the text represents the result of source verification after reading the fixed object verification request record and the interference verification request record, while direct change classification processing represents the result of classifying directly based on the image change region without performing source verification. Figure 5 As can be seen, the present invention maintains a high source verification accuracy rate under various interference scenarios, indicating that the present invention can distinguish the source of image changes caused by fixed objects and acquisition interference.

[0112] Simultaneously, five hazard source confirmation scenarios were selected: rockfall obstructing roads, slope cracks, accidental personnel entry, water accumulation, and landslide. Hazard source confirmation records and mine site object registration records were read, and the hazard source site areas were matched with the mine object areas to generate hazard source object matching records, hazard source impact records, hazard source output results, and early warning evidence records. In the experiment, the completeness of the early warning evidence association was calculated based on whether the original image number, image change area record number, suspected hazard source record number, source verification result record number, hazard source confirmation record number, hazard source impact record number, and hazard source output result number were completely associated. Figure 6 . Figure 6 The record chain in this invention represents the processing result of saving complete early warning evidence records, while outputting only early warning records represents the processing result of saving only the final early warning result. Figure 6 It is evident that the record chain of this invention has a high completeness rate of early warning evidence association under different types of hazard sources, indicating that this invention can form a traceable record relationship between hazard source confirmation results, mine object impact, and early warning output results.

[0113] This embodiment also provides a mine drone hazard source image processing and early warning system, including: an image processing module, used to acquire mine drone inspection images, read the shooting location, camera parameters, and mine site object registration records for coordinate conversion and image group sorting, and generate image location conversion records, image group processing records, and image participation processing records; a change extraction module, used to read processable images, align the current inspection image with the comparison image in the mine site coordinates, extract image change areas, and generate hazard source suspected records and verification request records based on the correspondence between change areas and mine areas, fixed objects, and acquisition interference; a source confirmation module, used to read hazard source suspected records, fixed object verification request records, and acquisition interference verification request records, perform continuous preprocessing, fixed object source verification, acquisition interference source verification, and continuous confirmation processing on suspected areas, and generate hazard source confirmation records or intermittent pending confirmation records; and an early warning output module, used to read hazard source confirmation records, correspond hazard source site areas with mine object areas, and generate hazard source impact records, hazard source output results, and early warning evidence records based on object entry ratio, distance records, change records, and source verification results.

[0114] In summary, this invention, through the extraction of image change areas and the generation of suspected hazard source records, fixed object verification request records, and data acquisition interference verification request records, enables change areas in mine drone inspection images to establish corresponding relationships with mine areas, fixed objects, and data acquisition interference, respectively, providing clear input for source verification and continuous confirmation processing. Furthermore, by processing the object correspondence between hazard source confirmation records and mine site object registration records, hazard source impact records, hazard source output results, and early warning evidence records are generated. This allows early warning outputs to be associated with object entry ratios, distance records, change records, and source verification results, improving the clarity of hazard source differentiation and the completeness of early warning evidence association.

[0115] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for image processing and early warning of hazardous sources in mining using unmanned aerial vehicles (UAVs), characterized in that, include: The system acquires images from mine inspections using drones, performs coordinate conversion and image group sorting based on the image capture location, camera parameters, and mine site object registration records, and generates image location conversion records, image group processing records, and image participation processing records. Read the image group processing record and image position conversion record, align the processable image and the comparison image in the mine site coordinates, extract the image change area, and generate the hazard source suspected record, fixed object verification request record and acquisition interference verification request record based on the correspondence between the change area and the mine area, fixed object and acquisition interference. Based on suspected hazard source records, fixed object verification request records, and data acquisition interference verification request records, combined with the effective adjacency relationships in the image group processing records, suspected area continuation preprocessing, source verification, and continuous confirmation processing are performed to generate source verification result records, hazard source continuous records, hazard source confirmation records, or intermittent pending confirmation records. By using hazard source confirmation records and mine site object registration records, the hazard source site area is matched with the mine object area. Based on the object entry ratio, distance records, change records and source verification results, hazard source impact records, hazard source output results and early warning evidence records are generated.

2. The mining drone hazard source image processing and early warning method as described in claim 1, characterized in that, The process of acquiring images from a mine drone inspection, and performing coordinate conversion and image group sorting based on the image capture location, camera parameters, and mine site object registration records, includes: After acquiring images of the mine inspection drone, the drone's positioning coordinates, flight altitude, shooting direction, gimbal angle, camera intrinsic parameters, and camera extrinsic parameters are read. The object surface data in the mine site object registration record is read according to the area number. The corresponding local conversion surface is selected, and the pixel position in the inspection image is converted into the site coordinates to generate the initial image position conversion result. Read the fixed object registration field from the mine site object registration record, match the current fixed object outline in the inspection image with the outline range and site coordinate range in the fixed object registration field, and generate the fixed object number, the outline corresponding point, and the corresponding point offset; The initial image position conversion result is corrected based on the corresponding points of the contour and the offset of the corresponding points. When a fixed object is matched, the matching result with the offset of the corresponding point exceeding the allowable range of image position conversion error is removed. The adjacent image correction object number is generated based on the retained matching result. Write the corrected on-site coordinates, coordinate conversion error, and adjacent image correction object numbers into the image position conversion record; If no fixed object is matched or no valid matching result is found after elimination, write an uncorrected state in the image position conversion record.

3. The mining drone hazard source image processing and early warning method as described in claim 1 or 2, characterized in that, The generated image location conversion record, image group processing record, and image participation processing record include: The inspection images are sorted according to the inspection route number, area number, shooting time, and adjacent image number. Valid adjacent relationships are generated based on the coverage ratio of adjacent images. Processable image numbers and abnormal image numbers are determined based on clarity, exposure, occlusion, edge distortion, shooting position deviation, and area correspondence status.

4. The mining drone hazard source image processing and early warning method as described in claim 3, characterized in that, The step of aligning the processable image and the comparison image in the coordinates of the mine site includes: After reading the processable image number, select the previous image with a valid adjacent relationship as the comparison image; When there is no previous image with a valid adjacency relationship, the image that was manually confirmed before the inspection or the image that was manually confirmed in the previous inspection cycle is read as the comparison image. The current inspection image and the comparison image are converted to the same mine site coordinates for area alignment.

5. The mining drone hazard source image processing and early warning method as described in claim 4, characterized in that, The process of extracting image change areas and generating suspected hazard source records, fixed object verification request records, and data acquisition interference verification request records based on the correspondence between the change areas and the mining area, fixed objects, and acquisition interference includes: Extract the image change areas corresponding to color changes, grayscale changes, edge contour changes, and texture direction changes within the aligned comparable area. Correspond the change areas with the slope range, transportation road range, drainage ditch inlet, equipment operation range, prohibited entry range, fixed object contour, and image acquisition anomaly records to generate suspected hazard source records, fixed object verification request records, and acquisition interference verification request records.

6. The mining drone hazard source image processing and early warning method as described in claim 5, characterized in that, The process of combining effective adjacency relationships in image group processing records to perform preprocessing for suspected region continuation, source verification, and continuous confirmation includes: Based on the on-site location, suspected hazard type, main area number, and associated area number in the suspected hazard source record, the effective adjacency relationship in the image group processing record is read, the continuity distance and allowable value of the continuity distance of suspected areas in adjacent images are calculated, and a suspected area continuity preprocessing record for source verification and continuous confirmation processing is generated.

7. The mining drone hazard source image processing and early warning method as described in claim 6, characterized in that, The generated source verification result record, hazard source continuous record, hazard source confirmation record, or intermittent pending confirmation record includes: Read the fixed object verification request record, the data collection interference verification request record, and the slope texture reference record, and perform fixed object source verification, data collection interference source verification, and slope texture source verification on the suspected hazard source records respectively; For suspected records that have not yet been excluded, continuous hazard source records, hazard source confirmed records, or intermittent pending confirmation records will continue to be generated based on the location continuation results, area change ratio, and continuous confirmation ratio.

8. The mining drone hazard source image processing and early warning method as described in claim 7, characterized in that, The process of mapping the hazardous source site area to the mining target area includes: After reading the hazard source confirmation record, based on the hazard source number, confirmation image number, hazard source continuous record number and on-site location range in the hazard source confirmation record, backtrack to read the corresponding hazard source continuous record and image change area record, and obtain the change area outline, change area area, length record, width record and direction record from the image change area record; Based on the outline of the changed area, the range of the site location, and the image location conversion record, a hazardous source site area is generated. The hazardous source site area is then matched with records of the range of transportation roads, road intersections, drainage ditch entrances, equipment operation areas, personnel access areas, prohibited areas, blasting warning areas, vehicle access areas, low-lying work surfaces, or slope rockfall impact areas to generate a corresponding record for the hazardous source object.

9. The mining drone hazard source image processing and early warning method as described in claim 8, characterized in that, The process of generating hazard source impact records, hazard source output results, and early warning evidence records based on object entry ratio, distance records, change records, and source verification results includes: Based on the corresponding records of hazardous source objects, the entry ratio of objects is calculated. Combined with the distance records between the hazardous source site area and the mine object area, the change records in the continuous records of hazardous sources, and the source verification results, a hazardous source impact record is generated. Read the hazard source confirmation record and hazard source impact record, and generate an immediate response warning or on-site review warning according to the object occupancy status, distance record, change record and source verification results; Read the records of intermittent data to be confirmed, records of abrupt changes in area to be confirmed, records of data acquisition interference to be confirmed, and records of slope texture to be confirmed. Generate a prompt to continue data acquisition based on the image area to be supplemented, the field to be confirmed, and the reason for the intermittency. Immediate handling warnings, on-site review warnings, or continued data collection prompts will be written into the hazard source output results. The original images, records of image change areas, suspected hazard source records, verification request records, source verification result records, continuous hazard source records, hazard source confirmation records, hazard source impact records, and the numbering relationship between the hazard source output results will be saved.

10. A mine unmanned aerial vehicle (UAV) hazard source image processing and early warning system, based on the mine UAV hazard source image processing and early warning method according to any one of claims 1 to 9, characterized in that, include: The image processing module is used to acquire images of mine drone inspections, read the shooting location, camera parameters and mine site object registration records, perform coordinate conversion and image group sorting, and generate image location conversion records, image group processing records and image participation processing records. The change extraction module is used to read the processable image, align the current inspection image with the comparison image in the coordinates of the mine site, extract the changed area of ​​the image, and generate a suspected hazard source record and a verification request record based on the correspondence between the changed area and the mine area, fixed objects, and acquisition interference. The source confirmation module is used to read suspected hazard source records, fixed object verification request records, and data collection interference verification request records. It performs continuous preprocessing, fixed object source verification, data collection interference source verification, and continuous confirmation processing on suspected areas to generate hazard source confirmation records or intermittent pending confirmation records. The early warning output module is used to read the hazard source confirmation record, match the hazard source site area with the mine object area, and generate hazard source impact record, hazard source output result and early warning evidence record based on the object entry ratio, distance record, change record and source verification result.

Images