Multi-sensor fusion discrimination coal gangue detection and classification method and system

Abstract

The invention relates to the technical field of multi-sensor identification, in particular to a coal gangue detection and classification method and system based on multi-sensor fusion discrimination, and the method comprises the following steps: obtaining visible light and infrared images, calculating brightness and intensity judgment feature conditions, executing edge detection to extract gray segments, and fusing textures and a thermal field to generate a vector set. And performing clustering analysis to finish classification judgment, and outputting a coal gangue detection classification result. According to the method, a precise trigger mechanism is established through brightness and thermal radiation double-feature screening, boundary recognition sensitivity is enhanced through gray abrupt change point division, salient region extraction capacity is enhanced through weighted fusion of texture energy and gray gradient, and a cross-modal consistency feature group is constructed through combination of two-dimensional vector construction and similarity screening. The recognition expression integrity is improved, static threshold classification is replaced by vector distribution clustering, accurate mapping and classification decision making of material attributes in a complex scene are achieved, and the stability and the recognition rate of a coal gangue detection result are guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multi-sensor identification, and in particular to a multi-sensor fusion discrimination coal gangue detection and classification method and system. BACKGROUND

[0002] The technical field of multi-sensor identification includes realizing more comprehensive and accurate identification and classification of target objects by integrating data from multiple types of sensors. The core content is to synchronously collect data obtained by sensors of different physical mechanisms, perform information fusion and joint analysis, thereby improving the identification accuracy and anti-interference ability of the system. It is widely used in industrial detection, security monitoring, intelligent transportation, environmental monitoring and other fields, often involving image sensors, infrared sensors, laser radars, ultrasonic devices, etc. The focus is on how to realize spatial and temporal correlation processing of multi-source data to ensure the stability and reliability of the identification process.

[0003] Among them, the multi-sensor fusion discrimination coal gangue detection and classification method refers to using multiple sensors to collect material image and physical property data in the process of coal transportation or mining, and automatically distinguishing and classifying coal and gangue by combining image texture analysis and spectral characteristic recognition. For the identification problem of mixed coal and gangue mixture in coal, the joint processing of image light intensity information and near-infrared reflectivity is carried out. By extracting texture features in visible light images and comparing and analyzing coal quality response characteristics in the near-infrared band, the material attribute discrimination and classification identification are completed. The identification process usually carries out fusion discrimination according to image region segmentation criteria and spectral response determination threshold, and completes the detection and classification of coal gangue by comprehensively considering the physical differences between multiple sensor data.

[0004] In the existing coal gangue detection and classification process, the linkage processing of image and physical property data lacks stable screening basis, and invalid data is easily introduced into the process when multiple types of information are concurrent, causing system processing redundancy. The boundary recognition method in the image relies on single light intensity change or contour model, and is not sensitive in the presence of complex background interference or boundary blur phenomenon, making it difficult to accurately extract key regions and reducing the quality basis of subsequent recognition data. In the feature extraction process, most of them adopt independent calculation method, and lack of systematic measurement of the inherent weight relationship between features, resulting in scattered expression ability and affecting the accurate positioning of high recognition regions. The multi-modal data fusion process mainly uses channel stacking or synchronous extraction, and does not form a mechanism to establish consistency reference in feature space, so the correlation between modalities is weak, and classification interference may occur when the information structure is inconsistent. The feature clustering method generally relies on setting class labels or static threshold judgment, which is difficult to adapt to the situation of fuzzy class boundary or uneven distribution density in actual collected data. In the application scene where the material state changes frequently or the transition between classes is obvious, it is easy to cause the identification deviation to expand, affecting the reliability of the identification result in the industrial practical environment. SUMMARY

[0005] The present application aims to solve the problems existing in the prior art and proposes a coal gangue detection classification method based on multi-sensor fusion discrimination.

[0006] In order to achieve the above-mentioned purpose, the present application adopts the following technical scheme: a coal gangue detection classification method based on multi-sensor fusion discrimination, comprising the following steps: S1: Obtain the visible light image and the infrared image of the coal gangue conveying belt, respectively calculate the brightness mean value and the intensity standard deviation, judge whether the brightness mean value is greater than the gray threshold value and whether the intensity standard deviation is lower than the infrared radiation stability threshold value, and generate a multi-source data trigger signal; S2: Based on the multi-source data trigger signal, perform edge detection scanning on the fusion atlas, calculate the continuous coordinate gray difference value, identify the mutation point to divide the gray section, and generate an edge feature section set; S3: According to the edge feature section set, calculate the texture energy value of each section, weight and merge the corresponding gray gradient value, sort and filter the gray section with the maximum weight value, and generate a main texture feature area; S4: Based on the main texture feature area, extract the corresponding area visible light image gray co-occurrence matrix feature and infrared image thermal radiation gradient field, construct a two-dimensional vector and compare the cosine similarity between the vector groups, and generate a multi-modal feature vector set; S5: According to the modal feature vector set, perform clustering analysis on all vector groups, count the proportion of each group and identify the maximum class attribution relationship, map the corresponding class attribution to the current atlas area, and obtain the coal gangue detection classification result.

[0007] As a further scheme of the present application, the multi-source data trigger signal includes frame number, activation state mark, trigger signal type, the edge feature section set includes gray mutation point, coordinate interval, gray section number, the main texture feature area includes gray section index, atlas area coordinates, texture energy level, the multi-modal feature vector set includes texture co-occurrence matrix feature vector, thermal radiation gradient field vector, feature similarity value, and the coal gangue detection classification result includes atlas area class label, identification frame number and corresponding coordinate position.

[0008] As a further scheme of the present application, the acquisition step of the multi-source data trigger signal is specifically: S111: Extract the gray value based on the visible light image line by line, and form a brightness sequence according to the pixel order, obtain the gray difference value of every two adjacent pixels in the brightness sequence, calculate the gray difference value set mean value by traversing the entire sequence, and generate the brightness difference value mean value; S112: Obtain the infrared image specified region heat radiation intensity value, form the intensity sequence according to the pixel acquisition order, combine the brightness difference value mean and execute the standard deviation calculation operation on the intensity sequence, judge the coupling degree between the response amplitude and stability of the heat radiation under the brightness change, and obtain the infrared bright-heat composite difference intensity value; S113: Based on the brightness difference value mean and the infrared bright-heat composite difference intensity value, respectively judge the gray scale threshold value and the infrared radiation stability threshold value, if the brightness difference value mean is greater than the set gray scale threshold value and the infrared bright-heat composite difference intensity value is less than the infrared standard deviation threshold value, record the corresponding frame number and activate the data marking state, and generate a multi-source data trigger signal.

[0009] As a further scheme of the present application, the edge feature section set acquisition step is specifically: S211: Based on the multi-source data trigger signal, execute the edge detection scanning operation on the corresponding fusion atlas, set the edge detection standard value, detect the gradient value of each pixel point and compare it with the set threshold value, and obtain the edge gradient mask graph; S212: According to the edge gradient mask graph, extract the corresponding gray scale sequence according to the row and column order of the marked edge points in the atlas, record the gray value change on the continuous coordinates, calculate the coordinate gradient response, determine the change trend of the gradient response intensity on the space continuous position, identify the position where the gray difference value is greater than the edge detection threshold value as the mutation point, and obtain the mutation point coordinate sequence; S213: According to the mutation point coordinate sequence, segment and divide the image according to the interval distribution between adjacent mutation points, record the starting point and ending point coordinates of each segment, and generate the edge feature section set.

[0010] As a further scheme of the present application, the main texture feature area acquisition step is specifically: S311: According to the edge feature section set, obtain the pixel coordinates covered in each gray scale segment, and construct a gray level co-occurrence matrix in the corresponding atlas area, sequentially count the co-occurrence frequency of each gray pair combination, and calculate the texture energy value, and obtain the texture energy value sequence; S312: Based on the texture energy value sequence and the gray scale gradient value of each segment in the edge feature section set, calculate the absolute value mean of the gray scale gradient value of each segment as the gray scale change rate of adjacent pixels in the segment, weight the fusion of each segment texture energy value and the corresponding gray scale gradient value, calculate the fusion texture gradient weight value of the gray scale segment, and obtain the fusion texture gradient weight value sequence; S313: According to the fusion texture gradient weight value sequence, sort all gray scale segments from high to low according to the weight value, select the gray scale segment with the maximum weight value, extract the corresponding atlas area coordinate range, and record the pixel coordinate index of the corresponding segment, and generate the main texture feature area.

[0011] As a further scheme of the present application, the step of obtaining the multi-modal feature vector set is specifically: S411: Based on the main texture feature region, the corresponding position of the visible light image is extracted, all pixel gray values in the region are intercepted and a gray level co-occurrence matrix is constructed, the occurrence frequency of adjacent pixel gray combination is counted pair by pair to fill the co-occurrence matrix, and a gray level texture feature vector group is obtained; S412: According to the gray level texture feature vector group and the corresponding infrared image region, a thermal radiation gradient field is constructed, a thermal vector field sequence is constructed, each pair of gray level texture feature vectors and thermal radiation gradient vectors are combined to form a two-dimensional feature vector, the cosine similarity between the two-dimensional feature vectors is calculated, all vector pairs with a similarity greater than a feature consistency threshold are retained, and a high-similarity vector combination set is obtained; S413: Based on the high-similarity vector combination set, the vector content in each pair of vector combinations that meet the feature consistency condition is integrated to establish a multi-modal feature vector set.

[0012] As a further scheme of the present application, the step of obtaining the coal gangue detection and classification result is specifically: S511: According to the multi-modal feature vector set, the feature point coordinates of each vector are extracted, the corresponding frame number and atlas coordinates are constructed to build an index structure, the modal feature dimensions are uniformly mapped, and the vectors are sorted and arranged according to the vector number, and a multi-modal feature distribution point set is obtained; S512: According to the multi-modal feature distribution point set, the density characteristics of all vector points are judged, the core points are determined, the core points are taken as the starting point to expand their neighborhood to form a class cluster structure, the number of members of each class vector is counted, and the proportion in the overall vector set is calculated, the class with the maximum number of vectors is identified, and a clustering result number set is obtained; S513: According to the vector number marked as the maximum class in the clustering result number set and the corresponding frame number and atlas coordinate position, the atlas coordinate points are extracted item by item, the corresponding class is marked as the coal gangue type, and is uniformly recorded to establish a coal gangue detection and classification result.

[0013] A coal gangue detection and classification system based on multi-sensor fusion discrimination, comprising: A fusion signal extraction module acquires visible light and infrared image frames, extracts brightness and intensity sequences, calculates gray mean and intensity standard deviation, judges double threshold conditions, and acquires multi-source data trigger signals; An edge region identification module extracts gray difference values and judges mutation conditions based on the multi-source data trigger signal frames, divides gray sections and records coordinates, and acquires an edge feature section set; The main texture feature extraction module calculates texture energy and gray gradient based on the edge feature section set coordinates, screens the maximum value section after weighted fusion, extracts coordinates, and obtains a main texture feature area; The multi-modal feature construction module extracts a gray matrix feature and a thermal radiation gradient based on the main texture feature area, constructs a two-dimensional vector, and screens a combination with a similarity greater than a feature consistency threshold to obtain a multi-modal feature vector set. The clustering recognition attribution module performs clustering analysis according to the multi-modal feature vector set coordinates, counts the proportion of each group, maps the maximum group frame number and coordinates, and obtains a coal gangue detection classification result.

[0014] Compared with the prior art, the advantages and positive effects of the present application are that: In the present application, a precise triggering mechanism is established by screening the dual features of brightness and thermal radiation, the initial data filtering accuracy is improved, the boundary recognition sensitivity is enhanced by dividing the gray mutation points, the image segmentation has a more physical difference basis, the weighted fusion of texture energy and gray gradient enhances the extraction ability of the prominent area, the two-dimensional vector construction combines the similarity screening to construct a cross-modal consistency feature group, the recognition expression integrity is improved, the vector distribution clustering replaces the static threshold classification, has self-adaptive aggregation ability, realizes accurate mapping and classification decision of material properties in a complex scene, and each link from key frame screening to feature attribution output is optimized in cooperation, so that the stability and recognition rate of the coal gangue detection result are ensured. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 The main step flowchart of the present application is shown in the figure; Figure 2 The multi-source data trigger signal acquisition flowchart of the present application is shown in the figure; Figure 3 The edge feature section set acquisition flowchart of the present application is shown in the figure; Figure 4 The main texture feature area acquisition flowchart of the present application is shown in the figure; Figure 5 The multi-modal feature vector set acquisition flowchart of the present application is shown in the figure; Figure 6 The coal gangue detection classification result acquisition flowchart of the present application is shown in the figure. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0017] In the description of the present application, it should be understood that the terms "length", "width", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, in the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0018] Please refer to Figure 1 A coal gangue detection and classification method based on multi-sensor fusion discrimination includes the following steps: S1: Obtain the visible light image and infrared image of the coal gangue conveying belt, extract the gray value of each row of the visible light image to form a brightness sequence, extract the thermal radiation intensity value of the infrared image region to form an intensity sequence, calculate the mean value of the difference set of adjacent pixels of the brightness sequence, and calculate the standard deviation of the intensity sequence, judge whether the brightness mean value is greater than the gray threshold value (the gray boundary value set according to the reflectivity difference between coal and gangue (usually coal <100, gangue >150)) and the infrared standard deviation is lower than the infrared radiation stability threshold value (the standard deviation threshold value set based on the thermal radiation characteristics of minerals (usually <5℃)), if both conditions are met, record the corresponding frame number and mark the active state, and generate a multi-source data trigger signal; S2: Based on the multi-source data trigger signal, perform edge detection scanning on the fusion graph (by spatially registering the visible light image and the infrared image at the pixel level, integrating the brightness information of the visible light image and the thermal radiation intensity data of the infrared image according to the same coordinates to form a unified graph data structure, and generating a fusion graph containing gray and thermal information), extract the gray sequence and calculate the gray difference value of the continuous coordinates, identify the positions greater than the edge detection threshold value (the gradient threshold value set by the Canny algorithm (high and low threshold value ratio 1:3)) as the mutation points, divide the gray sections according to the mutation points and record the coordinate intervals of each section, and generate an edge feature section set; S3: According to the edge feature section set, calculate the texture energy value of each section (the texture feature quantity calculated by the gray level co-occurrence matrix), combine the texture energy value of each section with the corresponding gray gradient value (the absolute value mean of the gray change rate of adjacent pixels), and sort the weighted values, select the gray section with the largest weight value and extract the corresponding graph region coordinates, and generate a main texture feature region; S4: Based on the main texture feature area, extract the corresponding area visible light image gray level co-occurrence matrix features (including contrast, correlation, energy, homogeneity, etc. Standard texture features) and infrared image thermal radiation gradient field (vector field representing temperature changes in the infrared image), construct a two-dimensional vector and compare the cosine similarity between the vector groups, screen the vector combinations with similarity higher than the feature consistency threshold (similarity judgment standard (usually > 0.85) trained according to experimental data), and integrate to generate a multi-modal feature vector set; S5: According to the distribution structure of each feature vector in the multi-modal feature vector set, perform DBSCAN clustering analysis (density-based spatial clustering algorithm, conforming to ISO / IEC2382 standard) on all vector groups, count the proportion of each group and identify the maximum category attribution, map the corresponding category attribution to the current atlas area, simultaneously record the corresponding identification area corresponding frame number and coordinate position, and obtain the coal gangue detection classification result.

[0019] The multi-source data trigger signal includes frame number, activation state marker, trigger signal type, the edge feature section set includes gray level mutation point, coordinate interval, gray level section number, the main texture feature area includes gray level section index, atlas area coordinate, texture energy level, the multi-modal feature vector set includes texture co-occurrence matrix feature vector, thermal radiation gradient field vector, feature similarity value, and the coal gangue detection classification result includes atlas area category label, identification frame number, and corresponding coordinate position.

[0020] Please refer to Figure 2 , S1 step is: S111: Based on the visible light image, extract the gray level value row by row, and form a brightness sequence according to the pixel order, obtain the gray level difference value of every two adjacent pixels in the brightness sequence, calculate the mean value of the gray level difference value set by traversing the entire sequence, and generate the brightness difference value mean; In the process of extracting the gray level value row by row based on the visible light image, first, the scanning mode of the image needs to be selected. In actual sampling, the gray level value of each pixel point is obtained by setting the image processing device to scan horizontally row by row, and the pixels in each row are sequentially arranged to generate a brightness sequence. Taking a row of pixels in a single frame image as an example, if the gray level values are 85, 90, 87, 93, and 88 in sequence, the brightness sequence L=[85, 90, 87, 93, 88] is formed. Then, the difference value calculation operation is performed on the gray level values of every two adjacent pixels in the sequence, and the gray level difference value sequence [5, -3, 6, -5] is obtained, which is calculated by subtracting the gray level values bit by bit. The gray level difference value calculation reflects the micro-motion characteristics of the pixel gray level, and the negative value is retained to record the trend of brightness decrease, without absolute value processing, so as to maintain the brightness trend information. Then, the arithmetic mean of all difference values is calculated, that is, the brightness difference value mean= (5+ (-3)+6+ (-5)) / 4=0.75. The sampling data is shown in the following table: Table 1 coal gangue image sample data table pixel number gray value gray difference value infrared intensity value (°C) 1 85 5 42.1 2 90 -3 41.9 3 87 6 42.0 4 93 -5 42.3 5 88 — 41.8 As shown in Table 1, the table lists the gray value of the continuous 5 pixel points and the difference value sequence, which can be used for subsequent brightness change trend judgment and amplitude classification. According to the empirical data, the jump within ±10 range belongs to the moderate area, ±10 to ±25 is the neutral interval, and more than ±25 is the severe change area. In actual application, the average of most jumps in the sampled image is concentrated between-10 and 10, so the average of the brightness difference value can be used as the reference data of the gray dynamic change degree, and finally the average of the brightness difference value is obtained.

[0021] As shown in Table 1, the table lists the gray value of the continuous 5 pixel points and the difference value sequence, which can be used for subsequent brightness change trend judgment and amplitude classification. According to the empirical data, the jump within ±10 range belongs to the moderate area, ±10 to ±25 is the neutral interval, and more than ±25 is the severe change area. In actual application, the average of most jumps in the sampled image is concentrated between-10 and 10, so the average of the brightness difference value can be used as the reference data of the gray dynamic change degree, and finally the average of the brightness difference value is obtained.

[0022] S112: Obtain the infrared image specified area thermal radiation intensity value, form the intensity sequence according to the pixel acquisition order, combine the brightness difference value average and execute the standard deviation calculation operation on the intensity sequence, judge the coupling degree between the response amplitude and stability of the thermal radiation under the brightness change, and obtain the infrared brightness-heat composite difference intensity value; When acquiring the thermal radiation intensity value of the infrared image region, the region overlapping with the gray scale extraction region in the visible light image is selected, and the infrared intensity value is collected point by point in pixel order. As shown in Table 1, the thermal radiation temperature of each corresponding pixel is 42.1℃, 41.9℃, 42.0℃, 42.3℃, and 41.8℃, respectively, forming an infrared intensity sequence I = [42.1, 41.9, 42.0, 42.3, 41.8]. Then, standard deviation calculation is performed on the intensity sequence. In the specific operation, the sequence mean is first calculated as (42.1 + 41.9 + 42.0 + 42.3 + 41.8) / 5 = 42.02℃, then the square of the difference between each term and the mean is calculated, resulting in [ (0.08) 2, (-0.12) 2, (-0.02) 2, (0.28) 2, (-0.22) 2] = [0.0064, 0.0144, 0.0004, 0.0784, 0.0484]. The sum is 0.148, and after dividing by 5, the standard deviation is √(0.0296) ≈ 0.172℃. This standard deviation reflects the fluctuation degree of the infrared thermal radiation intensity. If the value is within 0.3℃, it can be determined as a stable thermal distribution interval. If the fluctuation is higher than 0.7℃, it can be determined as an abnormal distribution interval, which needs to be further removed. The judgment reference value is determined according to the statistical thermal reflection fluctuation of the coal gangue sample. Usually, the stable threshold is set to 5℃ as the safety upper limit. The data at this point is far below the threshold, so it can be directly processed as a qualified frame in the subsequent process, and finally the infrared bright-thermal composite difference intensity quantity is obtained.

[0023] S113: Based on the brightness difference value mean and the infrared bright-thermal composite difference intensity quantity, respectively, the gray scale threshold and the infrared radiation stability threshold are judged. If the brightness difference value mean is greater than the set gray scale threshold and the infrared bright-thermal composite difference intensity quantity is less than the infrared standard deviation threshold, the corresponding frame number is recorded and the data marking state is activated, and a multi-source data trigger signal is generated. According to the brightness difference value mean 0.75 and the infrared bright-thermal composite difference intensity quantity 0.172℃ obtained in the foregoing, the condition judgment operation is performed. The judgment conditions are that the brightness difference value mean needs to be greater than the set gray scale threshold, and the infrared bright-thermal composite difference intensity quantity needs to be less than the infrared stability threshold. The set gray scale threshold is 120 (derived from the interval mean of coal <100, gangue >150), and the thermal radiation stability threshold is set to 5℃. The brightness difference value mean 0.75 in the above sample is far below the gray scale threshold 120, and does not meet the brightness catastrophe condition, so this frame does not activate the state. If the brightness difference value mean of another sample frame is 135 and the thermal radiation composite difference intensity quantity is 4.2℃, the dual conditions are met. The system will automatically record the current frame image number, such as frame 36, and mark the active state as "1". The number-state mapping relationship is established in the image state table, and the trigger record array is written. The image frame number, activation state, timestamp, and other meta information are combined to construct a trigger data packet, and finally a multi-source data trigger signal is generated.

[0024] Referring to Figure 3 , S2 is: S211: Based on the multi-source data trigger signal, an edge detection scanning operation is performed on the corresponding fusion graph, an edge detection threshold value is set, the gradient value of each pixel point is detected and compared with the set threshold value, and an edge gradient mask graph is obtained; After selecting the current image frame based on the multi-source data trigger signal, the image region corresponding to the current frame needs to be extracted from the fusion graph, and an edge gradient scanning operation is performed according to the preset scanning path. Each time the scanning needs to call the gray value of the current pixel point and the gray value change between its adjacent pixels to construct a gradient image. The system presets the high and low thresholds of edge detection as 90 and 30 respectively, which meets the 1:3 threshold setting ratio in Canny edge detection. By comparing each pixel gradient value with the set threshold value, if the current pixel gradient value is greater than 90, it is marked as a strong edge point, if it is less than 30, it is directly excluded and not recorded, and the gray value change point between the two needs to be judged whether it is connected with the strong edge point. If it is connected, it is set as a weak edge point, otherwise it is regarded as a background region and discarded. This logic needs to be executed once on the entire image. Taking the horizontal scanning data of an image as an example, the corresponding pixel gray value and edge gradient sample can be seen in the following table: Table 2: Fusion graph edge gradient extraction sample table pixel point number image coordinate gray value edge gradient value 1 （12，45） 132 15 2 （13，45） 134 18 3 （14，45） 185 72 4 （15，45） 182 65 5 （16，45） 130 12 6 （17，45） 129 10 As shown in Table 2, the edge gradient values of all pixel points in the current sample are less than the high threshold 90. Among them, the gradient values of pixel points 3 and 4 are between 30 and 90, but they are not connected with the strong edge point and are not retained. The edge gradient values of pixel points 1, 2, 5 and 6 are all less than 30, which are directly excluded. The final edge gradient mask graph has no effective edge point marker in the horizontal region, and the edge gradient mask graph is obtained after the execution process is completed.

[0025] S212: According to the edge gradient mask graph, the corresponding gray sequence of the marked edge points in the graph is extracted in row and column order, the gray value change on the continuous coordinates is recorded, and the difference value calculation is performed on the gray value change between adjacent coordinates to obtain the gray difference value sequence ΔG. Then the absolute value of each difference value item in the difference value sequence is processed and compared with the edge detection threshold value, using the formula: ; Calculate the coordinate gradient response Determine the trend of the change of the gradient response intensity at the spatial continuous position, identify the position where the gray difference value is greater than the edge detection threshold value as the mutation point, and obtain the mutation point coordinate sequence, wherein, represents the gray value of the jth edge point, represents the corresponding image coordinate position, and m is the number of edge points; The sample data from the foregoing Table 2 is as follows: , , , , , , Participating pixel pair: First pair of pixels: ; ; ; Second pair of pixels: ; ; ; First pair of pixels: ; Second pair of pixels: ; Comprehensive response degree calculation m = 2: ; The results show that, in the selected pixel pair, the coordinate gradient response degree , if the current mutation judgment threshold is set to 40, then this response degree does not constitute the basis for the mutation point, and the image in this region does not have the signal of the mutation structure, and is not marked as a mutation point. In the actual processing process, this value will be filtered and not included in the mutation point coordinate sequence. The calculation result directly determines whether the current scanning section meets the mutation positioning requirements and promotes the subsequent gray section division.

[0026] The coordinate gradient response degree represents the comprehensive mutation degree of the image between adjacent pixels under the joint action of three dimensions of gray change, spatial position interval, and brightness deviation. Its essence is to measure whether the gray jump intensity of each pair of edge pixels in the spatial structure has a mutation feature. By taking the gray difference value as the basic change quantity, combining the physical distance between adjacent pixels as the scale standard, and introducing the deviation between brightness and neutral gray as the texture control factor, the response degree not only reflects the degree of change in pixel gray, but also reflects whether the change occurs under a stable brightness background and a reasonable spatial interval. Therefore, the larger the value of the coordinate gradient response degree, the more likely it is to constitute an edge mutation or a morphological boundary in the image structure, which is an important basic index for determining the edge mutation point in the atlas.

[0027] The operation logic of the formula aims to integrate multiple key factors affecting mutation recognition to construct an enhanced gradient response degree index, where the numerator represents the change amplitude between the gray levels of adjacent pixels, which is the basic factor for judging the edge strength of the image, and The physical interval on the spatial coordinates is represented, and the introduction of the item can avoid misjudgment of single-point gray level jump caused by noise, that is, the influence needs to be appropriately weakened when the gray level change occurs in a distance of a very small area, and further combined with the root form of the brightness deviation The physical meaning is the deviation degree of the current pixel brightness relative to the neutral brightness, the root processing can smooth the response change amplitude of the high brightness or low brightness area, so that the judgment of the gray level change of the image in the high contrast area is more stable, and the combination of the spatial interval and the brightness deviation by addition forms a dynamic normalization item, in order to balance the mutation suppression ability in the spatial and brightness directions at the same time, so that the change of the gradient response degree is not amplified or weakened by local fluctuation in the low brightness dense area or the high frequency detail area. Finally, through summation and averaging, the balanced mutation response degree of the overall image is obtained, and the response ability of the image edge recognition to the three factors of gray level gradient, position interval and brightness characteristics is enhanced.

[0028] S213: According to the mutation point coordinate sequence, the image is segmented and divided according to the interval distribution between adjacent mutation points, the starting point and the ending point coordinates of each segment are recorded, and the edge feature segment set is generated; According to the mutation point coordinate sequence, the continuous image coordinates are split into multiple gray level paragraphs according to the mutation points, each two mutation points form an independent segment, and the original image data is called to extract the corresponding gray level sequence of each segment, the total gray level change and the segment length value are calculated, and the average gray level increase rate in the segment is calculated. According to the set rule, the effective gray level segment is screened, and the screening rule is that the segment length is not less than 3 pixels, and the total gray level difference is greater than 20. Assuming that the identified mutation point coordinates are (13, 45), (17, 45) and (21, 45), two paragraphs [13-17] and [17-21] are formed, the gray level difference in the segment is calculated as 26 and 14 respectively, and only the first segment meets the difference threshold requirement, and the second segment is discarded. The segment number, segment length, starting and ending coordinates, average gray level increment and other fields are reserved and an edge paragraph set table is constructed for subsequent atlas feature mining processing. Finally, the interval structure index corresponding to all effective paragraphs is established, and the edge feature segment set is generated.

[0029] Please refer to Figure 4 , the S3 step is: S311: According to the edge feature segment set, the pixel coordinates covered in each gray level segment are obtained, and a gray level co-occurrence matrix is constructed in the corresponding atlas area. The co-occurrence frequency of each gray level pair combination is sequentially counted, and the texture energy value is calculated. The texture energy value sequence is obtained. According to the start and end coordinate range stored in the edge feature section set, the pixel set covered by each gray scale section is located in the atlas, all adjacent pixel pairs in each section are obtained, the number of gray scale pair combinations with a distance of 1 pixel in the horizontal direction (0°) is counted, and a gray scale co-occurrence matrix is constructed. The matrix dimension is set to 256x256, corresponding to a 256-level gray scale image. In the matrix, the value corresponding to each position is the frequency of the gray scale pair combination in the section. After normalizing the gray scale pair frequency, square all elements in the matrix and sum them up to get the texture energy value of the gray scale section. For example, if the gray scale pair (130, 132) appears 8 times, (132, 132) appears 12 times, and (132, 133) appears 5 times in gray scale section D1, a total of 25 gray scale pairs are counted, then the normalized frequencies of the three items are 0.32, 0.48, and 0.2 respectively. After squaring, they are 0.1024, 0.2304, and 0.04 respectively. The total is 0.3728. After adding the square values of other gray scale pairs, let the final sum be 0.85, i.e. the texture energy value is 0.85. The same operation is performed on other gray scale sections D2, D3, etc. to get the complete texture energy sequence, as shown in the following table: Table 3 Gray scale section texture and gradient data table gray section number texture energy value gray gradient value D1 0.85 12.0 D2 0.65 18.0 D3 0.72 9.0 As shown in Table 3, the texture energy values corresponding to D1, D2 and D3 are 0.85, 0.65 and 0.72 respectively, which provide the basis for subsequent fusion weight calculation. The final texture energy value sequence is obtained.

[0030] S312: Based on the texture energy value sequence and the gray scale gradient value of each section in the edge feature section set, the absolute value mean of the gray scale gradient value of each section is calculated as the absolute value mean of the gray scale gradient value of each section. The texture energy value of each section is weighted and fused with the corresponding gray scale gradient value. The formula is: ; The fusion texture gradient weight value of the kth gray scale section is calculated , and the fusion texture gradient weight value sequence is obtained, where represents the texture energy value of the kth gray scale section, represents the gray scale gradient value of the kth gray scale section, and n represents the total number of all gray scale sections. The denominator part is used for normalization and constraint of the deviation between energy and gradient to avoid abnormal high results of gradient; Read the texture energy value sequence and the gray gradient value column in Table 3, respectively, for each segment of the gray gradient, the joint operation is carried out, the gray gradient value is the mean value of the absolute value of the gray difference of each segment of the adjacent pixel, if the D1 gray sequence is [130, 140, 154, 166, 178], the adjacent difference value is [10, 14, 12, 12], the mean value is 12.0, then the D1 texture energy value E1=0.85 and the gradient value G1=12.0 are substituted into the formula: ; The expansion calculation is that the numerator is , the denominator is , and finally is obtained. , , the results are written into the fusion weight column to form a complete response set, and the larger the response is, the stronger the coordination of the segment texture energy and the gradient change rate is, and finally the fusion texture gradient weight value sequence is obtained.

[0031] The fusion texture gradient weight value is a comprehensive measurement index, which is used to reflect the joint expression ability of a gray segment in the image in the two dimensions of texture structure definition and gray change intensity, wherein the texture energy reflects the arrangement order and concentration of the pixel gray pairs in the segment, and the gray gradient measures the gray change rate between adjacent pixels in the segment, the weight value measures whether the region has stable texture structure and prominent gray boundary features at the same time by fusing the two types of features, and the larger the value is, the higher the texture significance and gray jump intensity of the gray segment in space at the same time, which can be used as the preferred screening basis of the main texture feature region in the atlas region identification, and has strong discrimination and selection representativeness.

[0032] The operation logic design of the formula is based on the coupling characteristics of texture energy and gray gradient in image representation, the numerator part adopts the square root of the sum of the texture energy value and the gray gradient plus one to multiply, because the texture energy value reflects the concentration of the internal gray distribution of the image region, and the gradient value represents the gray change amplitude, the product of the two can constitute a joint representation intensity, and the gradient value plus one and the square root processing are to retain its contribution when the gradient value is small, and to suppress the linear amplification effect of the product caused by the large gradient, and to keep the value smooth growth; the denominator part is the sum of the absolute value of the difference between the texture energy value and the gradient value in all gray segments plus one, which is intended to build a normalization factor to balance the feature deviation degree between different segments, the plus one operation avoids the denominator being zero, and at the same time introduces the overall amplitude of the difference between segments to stretch or compress the output of the molecule, forming a relative score structure, which overall pushes the gray segments with high texture concentration and high gray mutation amplitude to the front of the priority weight, ensuring the discrimination of the screening structure and the representativeness of the region selection.

[0033] S313: According to the fusion texture gradient weight value sequence, all gray scale segments are sorted from high to low according to the weight value, the gray scale segment with the maximum weight value is selected, the corresponding atlas region coordinate range is extracted, and the pixel coordinate index of the corresponding segment is recorded to generate the main texture feature region; According to the fusion texture gradient weight value sequence, all gray scale segments are sorted from large to small, the sorting results are recorded, and the segment with the maximum weight value is marked as the main segment. For example, the sorting results are D2>D1>D3, and the corresponding weight values are 0.0984, 0.0811 and 0.0616. Then, the first D2 segment is extracted, the start and end coordinates corresponding to it in the atlas are located back, the start and end pixel coordinate positions are recorded, the coverage area range is obtained, all pixel positions in the area are constructed as the main texture structure region index, the structure record table of the paragraph number, coordinate range, texture gradient fusion value and other contents is established, and the main texture region table is written. Finally, the main texture feature region is generated.

[0034] Please refer to Figure 5 , the S4 step is: S411: Based on the main texture feature region, the corresponding position of the visible light image is extracted, all pixel gray scale values in the region are intercepted and a gray scale co-occurrence matrix is constructed, the appearance frequency of adjacent pixel gray scale combinations is counted pair by pair to fill the co-occurrence matrix, and a gray scale texture feature vector group is obtained; Based on the image region extracted from the main texture feature region, all gray scale pixels in the spatial coordinate range are first determined, the adjacent pixel gray scale pairs in the horizontal direction are extracted in the range, the gray scale co-occurrence matrix is constructed by counting the frequency of their appearance in the image, the matrix size is set to 256x256, corresponding to 256 gray levels, the direction angle is 0°, and the pixel distance is 1. Then, texture feature calculation is performed on each gray scale segment region to obtain contrast, correlation, energy, and homogeneity statistics values in turn. The contrast is obtained by summing the frequency of the square of the gray scale difference, the correlation is obtained by calculating the covariance of the joint distribution, the energy is obtained by summing the square of the frequency, and the homogeneity is obtained by summing the frequency divided by 1 plus the square of the gray scale difference. Assuming that the gray scale pair combinations of a region are concentrated on the main diagonal, the gray scale contrast is 2.1, the correlation is 0.82, the energy is 0.31, and the homogeneity is 0.78. Record this group of values as V1 vector. Process other regions in the same way to form a set of gray scale texture vectors, as shown in the following table: Table 4 Gray scale texture and infrared gradient two-dimensional vector data table vector number gray contrast gray correlation thermal gradient direction component thermal gradient amplitude component V1 2.1 0.82 0.58 0.69 V2 2.4 0.79 0.61 0.72 V3 2.0 0.85 0.57 0.66 As shown in Table 4, V1, V2, and V3 record the standard gray scale texture indicators and thermal gradient features of the corresponding regions, providing a structural basis for subsequent multi-modal joint, and finally obtaining a set of gray scale texture feature vectors.

[0035] S412: Construct a thermal radiation gradient field according to the gray texture feature vector set and the corresponding infrared image region, and construct a thermal vector field sequence based on the directional gradient formed by the temperature difference of adjacent pixels in the image. Combine each pair of gray texture feature vectors and thermal radiation gradient vectors to form a two-dimensional feature vector, and use the formula: ; Calculate the cosine similarity between the gth and hth two-dimensional feature vectors , and retain all vector pairs with similarity greater than the feature consistency threshold to obtain a high-similarity vector combination set, where 、 represent the gray texture feature dimension and the infrared thermal gradient dimension in the first vector, respectively. The denominator is used to normalize the vector length. The filtering criterion is obtained from the experimental training threshold setting (generally set to 0.85 or higher); According to the data recorded in each vector in Table 4, the gray texture contrast and the infrared thermal gradient direction or amplitude are combined to form a two-dimensional feature vector. The similarity is calculated by combining all vectors in pairs. Using cosine similarity as the calculation standard, V1= (2.1, 0.58) and V2= (2.4, 0.61), then ; Comparing V1 and V3, and assuming V3 is (2.0, 0.57), the calculation method is the same, and ≈0.9984, which is higher than the filtering threshold of 0.85. The two vector pairs are retained. Finally, the structure relationship items that meet the conditions are selected from all combinations to obtain a high-similarity vector combination set.

[0036] Cosine similarity is an index that measures the degree of similarity between two vectors in a multi-dimensional space. Its core meaning is to calculate the cosine of the angle between two vectors to determine whether their directions are consistent in space. The value range is between -1 and 1. The closer the value is to 1, the closer the direction of the two vectors, i.e. the higher the similarity. A value of 0 indicates that the two vectors are orthogonal and unrelated. A value close to -1 indicates that the directions are completely opposite. In actual image processing or feature analysis, cosine similarity can be used to compare whether the feature vectors of different regions (such as texture intensity and thermal gradient response) show the same trend, thereby identifying target regions that show consistency or correlation under different modalities, and achieving cross-modal feature matching and fusion analysis.

[0037] The operation logic of the formula is based on the geometric meaning of cosine similarity. Its purpose is to evaluate the cosine of the angle formed by two two-dimensional vectors in vector space, reflecting their similarity. The numerator part of the formula is the sum of the corresponding products of the values in each dimension of the two vectors, i.e. , represents the dot product of the two vectors, used to measure the degree of their coincidence in direction; the denominator part is composed of the product of the modulus of each vector, the modulus is the square root of the square sum of each component of the vector, in the form of , for normalization of the dot product result, so that the final similarity result is limited in the interval of-1 to 1, so as to exclude the influence of the size of the vector itself on the result, and only the direction information is reserved for comparison, so the multiplication embodies the superposition effect of the consistency between vectors, and the square and square root operations are used to calculate the modulus of the vector and complete the standardization, the overall structure ensures that the closer the similarity value is to 1, the more consistent the direction of the two vectors is, that is, the more similar the structure characteristics of the gray texture and the thermal gradient are in the atlas.

[0038] S413: Based on the high-similarity vector combination set, the vector contents in each pair of vector combination meeting the feature consistency condition are integrated to establish a multi-modal feature vector set; According to the selected high-similarity vector combination set, the vectors in each pair of combination are integrated, the gray feature and the thermal gradient feature are classified uniformly, the repeated items are removed, and a unique coding identifier is set for each group of combinations, such as V1-V2 combination is set as F1, V1-V3 combination is set as F2, the four original values corresponding to each group of vectors are written into the vector set data structure, the content field includes combination number, gray contrast, gray correlation, thermal direction component, thermal amplitude component, and the combination source number and region coordinate index position are expanded and stored, forming a complex structure body with positioning and discrimination ability, finally these structures are sequentially written into a unified set, constructing the information pool required for cross-modal feature matching, and finally a multi-modal feature vector set is established.

[0039] Please refer to Figure 6 , the S5 step is: S511: According to the multi-modal feature vector set, the feature point coordinates of each vector are extracted, the corresponding frame number and atlas coordinates are constructed to form an index structure, the modal feature dimensions are uniformly mapped, and the vectors are sorted and arranged according to the vector number, and a multi-modal feature distribution point set is obtained; According to the structured vector information in the multi-modal feature vector set, it needs to be converted into standard two-dimensional feature points in order to recognize the spatial distribution of the subsequent density clustering. In the specific implementation, the modal feature dimension 1 and the modal feature dimension 2 are extracted from each vector as two-dimensional coordinates, the gray texture feature and the infrared thermal gradient feature are normalized respectively and are uniformly mapped to the interval [0, 1], then a joint structure record containing coordinate data, vector number, belonging frame number and atlas region coordinates is constructed, and a feature point set is formed. For example, there are three groups of vectors: V1 contains feature dimensions (0.72, 0.61), frame number 101, coordinates (12, 34); V2 feature is (0.70, 0.63), frame number 101, coordinates (13, 35); V3 is (0.74, 0.60), frame number 102, coordinates (14, 33), which are merged into standard input vectors, and the number and position information are labeled in the subsequent supporting input table data structure. In this way, the feature distribution point set composed of all vectors is constructed, and the table content is as follows: Table 5 Multi-modal feature vector clustering sample table vector number modal feature dimension 1 modal feature dimension 2 frame number atlas coordinate V1 0.72 0.61 101 （12，34） V2 0.70 0.63 101 （13，35） V3 0.74 0.60 102 （14，33） V4 0.32 0.22 103 （45，67） V5 0.35 0.20 103 （46，66） V6 0.33 0.21 104 （47，65） As shown in Table 5, the modal indicators of all feature vectors are in the standard interval, and cover multiple frame numbers and atlas coordinate regions, which meet the input requirement of spatial aggregation trend, and finally a multi-modal feature distribution point set is obtained.

[0040] S512: According to the multi-modal feature distribution point set, the density characteristics of all vector points are judged, the core points are determined, the class cluster structure is formed from the core points to their neighborhood, the number of members of each class is counted, the proportion in the whole vector set is calculated, the class with the maximum number of vectors is identified, and the clustering result number set is obtained; According to the multi-modal feature distribution point set constructed, all vectors are input into the spatial clustering logic as independent points, the upper limit of the neighborhood distance is set to 0.05, and the minimum number of neighborhood samples is set to 2 to determine the core point condition. For each vector point, the two-dimensional spatial distance between it and the other points is calculated, and it is identified whether it meets the core point condition in its neighborhood. If it meets the condition, it is marked as a core and a density connected cluster is constructed from the point. In the above data, the distances between vectors V1, V2 and V3 are less than the set neighborhood threshold, which meets the clustering connectivity, so they are classified into the same class C1; V4, V5 and V6 also meet the same condition to construct class cluster C2. After the clustering labels of all points are completed, the number of vectors contained in each class is counted, the number of C1 class is 3, and the number of C2 class is 3. If C1 is taken as the maximum class according to the number priority, all member numbers and associated frames and coordinates of C1 are extracted, and the corresponding frames 101, 102 and coordinates (12, 34), (13, 35) and (14, 33) of V1-V3 vectors are recorded, and a complete class mapping table and region positioning table are established for subsequent mapping output, and finally the clustering result number set is obtained.

[0041] S513: According to the vector number marked as the maximum category in the clustering result number set, the frame number and the atlas coordinate position corresponding to the vector number are extracted one by one, the corresponding category is marked as the coal gangue type, and is uniformly recorded to establish the coal gangue detection classification result; According to all vector numbers classified as the maximum category in the clustering result number set, the frame number and the atlas coordinate associated with the vector numbers are extracted one by one, the category information of the set is marked on the original atlas area, and the atlas annotation structure item is formed. Each structure contains vector number, category number, frame number and coordinate point content, all information is written into the classification result identification table, at the same time, the coordinate layer is constructed and the current category label is given, the area atlas coverage is classified as the coal gangue detection result, the recognition area coordinate layer is generated, and the time dimension synchronization annotation is completed combined with the frame serial number, and the structured recognition result file is output, and finally the coal gangue detection classification result is established.

[0042] A coal gangue detection classification system based on multi-sensor fusion discrimination, comprising: The fusion signal extraction module acquires visible light and infrared image frames, extracts brightness and intensity sequences, calculates gray mean and intensity standard deviation, judges double threshold condition, and acquires multi-source data trigger signal; The edge area recognition module extracts gray difference value and judges mutation condition based on the multi-source data trigger signal frame, divides gray section and records coordinates, and acquires edge feature section set; The main texture feature extraction module calculates texture energy and gray gradient based on the edge feature section set coordinates, filters the maximum value section after weighted fusion, extracts coordinates, and acquires the main texture feature area; The multi-modal feature construction module extracts gray matrix features and thermal radiation gradient based on the main texture feature area, constructs two-dimensional vectors and filters combinations with similarity greater than feature consistency threshold, and acquires multi-modal feature vector set; The clustering recognition attribution module performs clustering analysis according to the coordinates of the multi-modal feature vector set, counts the proportion of each group, maps the frame number and coordinates of the largest group, and acquires the coal gangue detection classification result.

[0043] The above is only a preferred embodiment of the present application, and does not limit the form of the present application, any skilled person in the art can use the disclosed technical content to make changes or modifications to equivalent embodiments applied to other fields, but any simple modification, equivalent change and modification made to the above embodiments without departing from the technical solution content of the present application, according to the technical essence of the present application, still belongs to the protection scope of the technical solution of the present application.

Claims

1. A coal gangue detection and classification method based on multi-sensor fusion discrimination, characterized in that, The method comprises the following steps: S1: Obtain the visible light image and the infrared image of the coal gangue conveying belt, calculate the brightness mean value and the intensity standard deviation respectively, judge whether the brightness mean value is greater than the gray threshold value and whether the intensity standard deviation is lower than the infrared radiation stability threshold value, and generate a multi-source data trigger signal; S2: Based on the multi-source data trigger signal, perform edge detection scanning on the fusion atlas, calculate the continuous coordinate gray difference value, identify the mutation point to divide the gray section, and generate an edge feature section set; S3: According to the edge feature section set, calculate the texture energy value of each section, weight and merge the corresponding gray gradient value, sort and filter the gray section with the maximum weight value, and generate a main texture feature area; S4: Based on the main texture feature area, extract the corresponding region visible light image gray co-occurrence matrix feature and infrared image thermal radiation gradient field, construct a two-dimensional vector and compare the cosine similarity between the vector groups, and generate a multi-modal feature vector set; S5: According to the modal feature vector set, perform clustering analysis on all vector groups, count the proportion of each group and identify the maximum class attribution relationship, map the corresponding class attribution to the current fusion atlas area, and obtain the coal gangue detection classification result.

2. The coal gangue detection classification method of multi-sensor fusion discrimination according to claim 1, characterized in that, The multi-source data trigger signal includes frame number, activation state mark, trigger signal type, the edge feature section set includes gray mutation point, coordinate interval, gray section number, the main texture feature area includes gray section index, atlas region coordinates, texture energy level, the multi-modal feature vector set includes texture co-occurrence matrix feature vector, thermal radiation gradient field vector, feature similarity value, and the coal gangue detection classification result includes atlas region class label, identification frame number and corresponding coordinate position.

3. The coal gangue detection classification method of multi-sensor fusion discrimination according to claim 1, characterized in that, The acquisition step of the multi-source data trigger signal is specifically: S111: Based on the visible light image, extract the gray value row by row, and form a brightness sequence according to the pixel order, obtain the gray difference value of every two adjacent pixels in the brightness sequence, calculate the gray difference value set mean value by traversing the entire sequence, and generate the brightness difference value mean value; S112: Obtain the infrared image specified region thermal radiation intensity value, form an intensity sequence according to the pixel acquisition order, combine the brightness difference value mean value and perform standard deviation calculation operation on the intensity sequence, judge the coupling degree between the response amplitude and stability of thermal radiation under brightness change, and obtain the infrared brightness-heat composite difference intensity value; S113: Based on the brightness difference value mean value and the infrared brightness-heat composite difference intensity value, judge the brightness difference value mean value and the infrared brightness-heat composite difference intensity value respectively with respect to the gray threshold value and the infrared radiation stability threshold value, if the brightness difference value mean value is greater than the set gray threshold value and the infrared brightness-heat composite difference intensity value is less than the infrared standard deviation threshold value, record the corresponding frame number and activate the data mark state, and generate a multi-source data trigger signal.

4. The coal and gangue detection and classification method of multi-sensor fusion discrimination according to claim 1, characterized in that, The acquisition step of the edge feature section set is specifically: S211: Based on the multi-source data trigger signal, perform edge detection scanning operation on the corresponding fusion atlas, set the edge detection standard value, detect the gradient value of each pixel point and compare it with the set threshold value, and obtain the edge gradient mask image; S212: According to the edge gradient mask map, the corresponding gray sequence of the labeled edge points in the fusion atlas is extracted in row and column order, the gray value change on the continuous coordinates is recorded, the coordinate gradient response degree is calculated, the change trend of the gradient response strength on the spatial continuous position is determined, the position where the gray difference value is greater than the edge detection threshold is identified as the mutation point, and the mutation point coordinate sequence is obtained; S213: According to the mutation point coordinate sequence, the image is segmented and divided according to the interval distribution between adjacent mutation points, the starting point and ending point coordinates of each segment are recorded, and an edge feature segment set is generated.

5. The coal and gangue detection and classification method of multi-sensor fusion discrimination according to claim 1, characterized in that, The acquisition step of the main texture feature region is specifically: S311: According to the edge feature segment set, the pixel coordinates covered in each gray segment are obtained, and a gray co-occurrence matrix is constructed in the corresponding atlas region, the co-occurrence frequency of each gray pair combination is sequentially counted, and the texture energy value is calculated, and a texture energy value sequence is obtained; S312: Based on the texture energy value sequence and the gray gradient value of each segment in the edge feature segment set, the absolute value mean of the gray gradient value of each segment is calculated, the texture energy value of each segment is weighted and fused with the corresponding gray gradient value, the fusion texture gradient weight value of the gray segment is calculated, and a fusion texture gradient weight value sequence is obtained; S313: According to the fusion texture gradient weight value sequence, all gray segments are sorted from high to low according to the weight value, the gray segment with the maximum weight value is selected, the corresponding atlas region coordinate range is extracted, and the pixel coordinate index of the corresponding segment is recorded, and a main texture feature region is generated.

6. The coal and gangue detection and classification method of multi-sensor fusion discrimination according to claim 1, characterized in that, The acquisition step of the multi-modal feature vector set is specifically: S411: Based on the main texture feature region, the corresponding visible light image is extracted, all pixel gray values in the region are intercepted and a gray co-occurrence matrix is constructed, the appearance frequency of adjacent pixel gray combinations is counted pair by pair to fill the gray co-occurrence matrix, and a gray texture feature vector group is obtained; S412: According to the gray texture feature vector group and the corresponding infrared image region, a thermal radiation gradient field is constructed, a thermal vector field sequence is constructed, each pair of gray texture feature vectors and thermal radiation gradient vectors are combined to form a two-dimensional feature vector, the cosine similarity between the two-dimensional feature vectors is calculated, all vector pairs with a similarity greater than a feature consistency threshold are retained, and a high-similarity vector combination set is obtained; S413: Based on the high-similarity vector combination set, the vector content in each vector combination that meets the feature consistency condition is integrated to establish a multi-modal feature vector set.

7. The coal gangue detection classification method of multi-sensing fusion discrimination according to claim 1, characterized in that, The acquisition step of the coal gangue detection classification result is specifically: S511: According to the multi-modal feature vector set, the feature point coordinates of each vector are extracted, the corresponding frame number and atlas coordinates are constructed to build an index structure, the modal feature dimensions are uniformly mapped, and the vectors are sorted and arranged according to the vector number, and a multi-modal feature distribution point set is obtained; S512: According to the multi-modal feature distribution point set, the density characteristics of all vector points are judged, the core points are determined, the core points are taken as the starting point to expand their neighborhood to form a class cluster structure, the number of members of each class vector is counted, and the proportion in the whole vector set is calculated, the class with the maximum vector number is identified, and the clustering result number set is obtained; S513: According to the vector number marked as the maximum class in the clustering result number set, the corresponding frame number and atlas coordinate position are extracted one by one, the atlas coordinate points are extracted, the corresponding class is marked as the coal gangue type, and is recorded uniformly, and the coal gangue detection classification result is established.

8. A coal gangue detection and classification system based on multi-sensor fusion discrimination, characterized in that, The system is used to realize the coal gangue detection classification method of multi-sensing fusion discrimination according to any one of claims 1-7, comprising: The fusion signal extraction module acquires visible light and infrared image frames, extracts brightness and intensity sequences, calculates gray mean and intensity standard deviation, judges double threshold condition, and acquires multi-source data trigger signal; The edge region identification module extracts gray difference value and judges mutation condition based on the multi-source data trigger signal frame, divides gray section and records coordinates, and acquires edge feature section set; The main texture feature extraction module calculates texture energy and gray gradient based on the edge feature section set coordinates, filters the maximum value section after weighted fusion, extracts coordinates, and acquires main texture feature area; The multi-modal feature construction module extracts gray matrix features and thermal radiation gradient based on the main texture feature area, constructs two-dimensional vector and filters the combination with similarity greater than feature consistency threshold, and acquires multi-modal feature vector set; The clustering recognition attribution module performs clustering analysis according to the multi-modal feature vector set coordinates, counts the proportion of each group, maps the frame number and coordinates of the largest group, and acquires the coal gangue detection classification result.

Images