System for detecting fire coal residues in carriage based on intelligent image recognition

Abstract

The invention provides a carriage internal fire coal residue detection system based on intelligent image recognition, and relates to the technical field of fire coal residue detection, and the system comprises a video monitoring module which is used for obtaining collected videos of all monitoring cameras in a carriage, and obtaining a plurality of key frame images based on the collected videos; the carriage three-dimensional point cloud model construction module is used for constructing a carriage three-dimensional point cloud model of the corresponding carriage based on the plurality of key frame images; the fire coal residue preliminary identification module is used for obtaining fire coal residue identification results of the plurality of key frame images; the fire coal residual area locking evaluation module is used for evaluating the current fire coal accumulation severity of the fire coal residual area; the key monitoring area marking module is used for confirming whether key monitoring area marking is carried out in the corresponding area of the carriage three-dimensional point cloud model or not; and the monitoring control module is used for controlling the monitoring camera to work. The technical problem that fire coal residues cannot be found in time due to the lack of a fire coal residue monitoring assembly in the prior art is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of coal residue detection, and particularly relates to a coal residue detection system inside a carriage based on intelligent image recognition. Background Art

[0002] During the acceptance process of coal entering a thermal power plant, after the coal enters the plant, it needs to be unloaded on a car dumper. Due to low temperature or high moisture content in the coal quality, a large amount of coal adheres to the carriage and cannot be completely unloaded. Due to the lack of a coal residue monitoring component, the coal residue cannot be detected in time, resulting in a reduction in the coal acceptance quantity. The traditional solution is to manually turn over the carriage regularly to check for residues, but this method is time-consuming and laborious and also increases safety risks. Summary of the Invention

[0003] The present invention provides a coal residue detection system inside a carriage based on intelligent image recognition to solve at least one of the above-mentioned technical problems.

[0004] To solve the above technical problems, the present invention discloses a coal residue detection system inside a carriage based on intelligent image recognition, including: A video monitoring module, configured to obtain the acquisition videos of all monitoring cameras inside the carriage and obtain a plurality of key frame images based on the acquisition videos; A carriage three-dimensional point cloud model construction module, configured to construct a carriage three-dimensional point cloud model corresponding to the carriage based on a plurality of key frame images; A coal residue preliminary recognition module, configured to input a plurality of key frame images into a trained coal residue recognition model to obtain the coal residue recognition results of the plurality of key frame images; A coal residue area locking and evaluation module, configured to obtain the coal residue area corresponding to the key frame images with the recognition result of coal residue, confirm the position of the coal residue area in the carriage, and evaluate the current severity of coal accumulation in the coal residue area; A key monitoring area marking module, configured to confirm whether to mark the key monitoring area in the corresponding area of the carriage three-dimensional point cloud model based on the current severity of coal accumulation in the coal residue area and a preset severity of coal accumulation; A monitoring control module, configured to control the operation of the monitoring cameras based on the key monitoring area marking result of the carriage three-dimensional point cloud model, and perform an artificial intervention alarm prompt based on the multiple acquisition feedback results of the monitoring cameras.

[0005] Preferably, the carriage three-dimensional point cloud model construction module includes: An image preprocessing unit, configured to perform image preprocessing on a plurality of key frame images; A feature extraction unit, configured to extract features from the preprocessed key frame images; A feature matching unit for matching feature points in a number of key frame images; A 3D reconstruction unit for obtaining the coordinate positions of feature points in 3D space based on the position information of feature point pairs in different key frame images, and constructing a preliminary 3D point cloud model; A point cloud optimization unit for optimizing the preliminary 3D point cloud model; A texture mapping unit for mapping the texture information of key frame images onto the optimized 3D point cloud model to obtain a 3D point cloud model of the corresponding carriage.

[0006] Preferably, the coal residue area locking and evaluation module includes: A coal residue area locking sub-module for locking the coal residue area based on the key frame images identified as having coal residues; A coal residue area position recognition module for inputting the key frame images identified as having coal residues into a trained coal residue area position recognition model to obtain the position information of the current coal residue area in the carriage, where the position information includes the carriage bottom, carriage wall, and carriage corner; A coal residue area evaluation sub-module for evaluating the current severity of coal accumulation in the coal residue area based on the pixel information of the coal residue area, the thickness of the coal accumulated in the coal residue area, and the current environmental temperature and humidity of the carriage.

[0007] Preferably, the coal residue area locking sub-module includes: A coal background pixel acquisition unit for acquiring the pixel value of each pixel point in the key frame image identified as having a coal residue, and calculating the pixel mean of the adjacent pixel points corresponding to each pixel point in each key frame image, and taking it as the background pixel of the pixel point in the key frame image; A coal background acquisition unit for calculating the sum of the background pixels of the corresponding pixel points on a number of key frame images identified as having coal residues, and taking the quotient of the sum of the background pixels of the corresponding pixel points on the key frame images identified as having coal residues and the number of key frame images identified as having coal residues as the generalized background pixel value of the pixel point, and taking the image composed of the generalized background pixel values of each pixel point as the coal background image; A coal residue area locking unit for calculating the absolute value of the difference between the pixel value of each pixel point in each key frame image identified as having a coal residue and its generalized background pixel value, and when the absolute value of the difference between the pixel value of the pixel point and its generalized background pixel value is greater than a preset pixel difference, marking the pixel point as a pixel point in the coal residue area, and the area composed of all the pixel points in the coal residue area is the coal residue area.

[0008] Preferably, the coal residue area evaluation sub-module includes: A pixel count unit for counting the number of pixels in the coal residue area of the coal residue region; A laser detection unit for detecting the thickness of the coal pile in the coal residue area; An environmental temperature and humidity detection unit for detecting the current temperature value and humidity value of the carriage; A coal pile severity calculation unit for calculating the current coal pile severity coefficient in the coal residue area.

[0009] Preferably, the current coal pile severity coefficient in the coal residue area: (1); where is the current coal pile severity coefficient of the xth coal residue area, is the coal residue area weight value, is the number of pixels in the coal residue area of the xth coal residue area, is the reference number of pixels in the coal residue area corresponding to the maximum acceptable coal residue area, is the coal thickness weight value, is the coal pile thickness of the xth coal residue area, is the acceptable reference coal thickness, e is a natural number, and its value is 2.71, is the current temperature and humidity weight value of the carriage, is the preset reference temperature value of the carriage, is the current humidity value of the carriage, is the current temperature value of the carriage, is the preset humidity value of the coal.

[0010] Preferably, the key monitoring area marking module includes: A marking judgment unit for comparing the current coal pile severity coefficient in the coal residue area with the reference coal pile severity coefficient. If the current coal pile severity coefficient in the coal residue area is greater than the reference coal pile severity coefficient, the coal residue area is marked; A marking execution unit for corresponding the pixels in the coal residue area to be marked with the coordinates of the carriage three-dimensional point cloud model one by one, taking the corresponding coordinate points as marking coordinate points, and performing a color marking on the marking coordinate points. If there are overlapping marking coordinate points in the remaining coal residue areas and the current coal residue area, repeated color overlay marking is performed on the marking coordinate points, and the saturation of each marking overlay is the same; The marker elimination unit is used to count the number of times each marker coordinate point is marked. If the number of times the current marker coordinate point is marked is zero in y consecutive detection cycles, the marker coordinate point is eliminated once, reducing the saturation by one. The area composed of all marker coordinate points is the key monitoring area.

[0011] Preferably, the monitoring control module includes: The monitoring area zoning unit is used to zone the key monitoring area, take the area composed of marker coordinate points with the same saturation as the same type of area to be sorted, sort the color saturations corresponding to several areas to be sorted, and rank the areas to be sorted in descending order of color saturation; The area to be sorted segmentation unit is used to segment each area to be sorted to obtain several discrete key monitoring areas corresponding to each area to be sorted; The monitoring camera acquisition times determination unit is used to determine the number of video acquisitions of the monitoring camera in the detection cycle for each discrete key monitoring area according to the ranking result corresponding to each discrete key monitoring area: (2); where, is the number of video acquisitions of the monitoring camera in the discrete key monitoring area with the ranking result of j in the detection cycle, is the natural logarithm, is the number of types of ranking results, is the ranking number of the current discrete key monitoring area, is the maximum ranking number among all discrete key monitoring areas, is the minimum ranking number among all discrete key monitoring areas.

[0012] Preferably, the area to be sorted segmentation unit includes: The segmentation contour determination sub-unit is used to count the number of marker coordinate points of the same type within a preset radius spherical space centered on each marker coordinate point in the area to be sorted. If the number of marker coordinate points with the same saturation is less than the preset number, the marker coordinate point is used as a contour coordinate point, and the closed line formed by all adjacent contour coordinate points is used as the segmentation contour line of each discrete key monitoring area; The discrete key monitoring area determination unit is used to take the area composed of the marker coordinate points in the area to be sorted corresponding to each segmentation contour line as the discrete key monitoring area.

[0013] Preferably, it further includes an artificial intervention alarm prompt unit, and the artificial intervention alarm prompt unit includes: The coordinate point quantity statistics unit is used to count the total number of marker coordinate points corresponding to the saturation of each discrete key monitoring area in each detection cycle; An alarm judgment unit is used to count the increase in the total number of marked coordinate points of the saturation corresponding to the same discrete key monitoring area in two adjacent detection cycles. If the increase in the total number of marked coordinate points of the saturation corresponding to the same discrete key monitoring area in two adjacent detection cycles is greater than a preset increase, an artificial intervention alarm prompt for the corresponding discrete key monitoring area is triggered.

[0014] Compared with the prior art, the present invention has the following beneficial effects: The present invention obtains key frame images inside the carriage through a video monitoring module, and uses a coal residue recognition model to analyze these images to identify whether there is coal residue. Combining with the three-dimensional point cloud model of the carriage, the position and severity of the coal residue area are further locked and evaluated through a coal residue area locking and evaluation module. The key monitoring area marking module adjusts the monitoring strategy according to the evaluation results, increases the number of times the monitoring camera collects images, and ensures more attention to the serious area. When the feedback results of the monitoring camera collected multiple times show that the expansion degree of the corresponding key monitoring area in two adjacent detection cycles exceeds the preset degree, an artificial intervention alarm prompt is given. In this way, the coal residue situation can be monitored in real time, measures can be taken in time, and potential safety hazards can be avoided. It solves the technical problem in the prior art that due to the lack of a coal residue monitoring component, the coal residue cannot be discovered in time. Description of the Drawings

[0015] The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the present invention and do not constitute a limitation to the present invention. In the drawings: Figure 1 It is a schematic diagram of the in-carriage coal residue detection system based on intelligent image recognition of the present invention. Detailed Embodiments

[0016] The following describes the preferred embodiments of the present invention with reference to the drawings. It should be understood that the preferred embodiments described herein are only used to explain and illustrate the present invention and are not used to limit the present invention.

[0017] In addition, in the present invention, descriptions such as "first" and "second" are for descriptive purposes only, and do not particularly refer to the order or sequence. Nor are they used to limit the present invention. They are merely used to distinguish components or operations described with the same technical terms, and should not be construed as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions and technical features between various embodiments can be combined with each other, but it must be based on the ability of those of ordinary skill in the art to implement. When the combination of technical solutions results in contradictions or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0018] The present invention provides the following embodiments Embodiment 1 The embodiment of the present invention provides a coal residue detection system for the interior of a carriage based on intelligent image recognition, as Figure 1 shown, including: A video monitoring module, configured to obtain the captured videos of all monitoring cameras in the carriage, and obtain a plurality of key frame images based on the captured videos; A carriage three-dimensional point cloud model construction module, configured to construct a three-dimensional point cloud model of the corresponding carriage based on a plurality of key frame images; A preliminary coal residue recognition module, configured to input a plurality of key frame images into a trained coal residue recognition model to obtain the coal residue recognition results of the plurality of key frame images; A coal residue area locking and assessment module, configured to obtain the coal residue area corresponding to the key frame images with the recognition result of coal residue, confirm the position of the coal residue area in the carriage, and evaluate the current severity of coal accumulation in the coal residue area; A key monitoring area marking module, configured to confirm whether to mark the key monitoring area in the corresponding area of the carriage three-dimensional point cloud model based on the current severity of coal accumulation in the coal residue area and a preset severity of coal accumulation; A monitoring control module, configured to control the operation of the monitoring cameras based on the key monitoring area marking result of the carriage three-dimensional point cloud model, and perform manual intervention alarm prompts based on the multiple captured feedback results of the monitoring cameras.

[0019] In this embodiment, the monitoring cameras in the carriage are respectively installed at different positions, and each monitoring camera is equipped with a lighting component.

[0020] In this embodiment, the carriage three-dimensional point cloud model is a digital model of the current carriage composed of a large number of three-dimensional coordinate points.

[0021] In this embodiment, the coal residue identification model is a model obtained by training a neural network model with a large number of carriage images containing coal residues and carriage images without coal residues as training input samples, and using the carriage images containing coal residues as the output.

[0022] In this embodiment, the positions of the carriages where the coal residue areas are located include the carriage bottom, the carriage wall, and the carriage corners.

[0023] In this embodiment, the current severity of coal accumulation in the coal residue area is represented by the coefficient of the current severity of coal accumulation in the coal residue area.

[0024] In this embodiment, the key monitoring area is the coal residue area where the coefficient of the current severity of coal accumulation in the coal residue area is greater than the reference coefficient of the severity of coal accumulation.

[0025] In this embodiment, based on the marked result of the key monitoring area of the carriage three-dimensional point cloud model, controlling the operation of the monitoring camera includes increasing the number of video acquisitions of the key monitoring area by the monitoring camera within the detection period.

[0026] In this embodiment, the feedback results of multiple acquisitions by the monitoring camera are the degree of expansion of the area of the key monitoring area corresponding to two adjacent detection periods.

[0027] In this embodiment, the methods of manual intervention include operating the carriage multiple times for dumping, using auxiliary tools to clean the coal residues, repairing the concave and convex planes of the carriage, etc.

[0028] The working principle and beneficial effects of the above technical solutions are as follows: The present invention obtains the key frame images inside the carriage through the video monitoring module, and uses the coal residue identification model to analyze these images to identify whether there are coal residues. Combining with the carriage three-dimensional point cloud model, the position and severity of the coal residue area are further locked and evaluated through the coal residue area locking and evaluation module. The key monitoring area marking module adjusts the monitoring strategy according to the evaluation results, increases the number of acquisitions by the monitoring camera, and ensures more attention to the severe areas. When the feedback results of multiple acquisitions by the monitoring camera are that the degree of expansion of the area of the key monitoring area corresponding to two adjacent detection periods exceeds the preset degree, a manual intervention alarm prompt is given. In this way, the coal residue situation can be monitored in real time, measures can be taken in a timely manner, and potential safety hazards can be avoided; The present invention solves the technical problem that in the prior art, due to the lack of a coal residue monitoring component, the coal residue cannot be discovered in time.

[0029] Embodiment 2 On the basis of Embodiment 1, the carriage three-dimensional point cloud model construction module includes: An image preprocessing unit for preprocessing a number of key frame images; A feature extraction unit for extracting features from the preprocessed key-frame images; A feature matching unit for matching feature points in a number of key-frame images; A three-dimensional reconstruction unit for obtaining the coordinate positions of feature points in three-dimensional space based on the position information of feature point pairs under different key-frame images and constructing a preliminary three-dimensional point cloud model; A point cloud optimization unit for optimizing the preliminary three-dimensional point cloud model; A texture mapping unit for mapping the texture information of the key-frame images onto the optimized three-dimensional point cloud model to obtain a three-dimensional point cloud model of the corresponding carriage.

[0030] In this embodiment, image preprocessing includes denoising, contrast enhancement, and color correction.

[0031] In this embodiment, feature extraction includes carriage edges, carriage corner points, texture features, and color features.

[0032] In this embodiment, the optimization process includes denoising and filling holes.

[0033] The working principle and beneficial effects of the above technical solution are as follows: By steps such as image preprocessing, feature extraction, feature matching, three-dimensional reconstruction, and point cloud optimization, a detailed three-dimensional point cloud model of the carriage can be generated, which not only improves the recognition accuracy but also can more accurately locate and evaluate the coal residue area, contributing to improving the reliability and accuracy of the system.

[0034] Embodiment 3 Based on Embodiment 1, the coal residue area locking and evaluation module includes: A coal residue area locking sub-module for locking the coal residue area based on the key-frame images identified as having coal residues; A coal residue area position recognition module for inputting the key-frame images identified as having coal residues into a trained coal residue area position recognition model to obtain the position information of the current coal residue area in the carriage, where the position information includes the carriage bottom, carriage walls, and carriage corners; A coal residue area evaluation sub-module for evaluating the current severity of coal accumulation in the coal residue area based on the pixel information of the coal residue area, the thickness of the coal accumulated in the coal residue area, and the current environmental temperature and humidity of the carriage.

[0035] In this embodiment, the coal residue area position recognition model is a model obtained by training a neural network model with a large number of carriage images containing coal residues as training input samples and the carriage position information where the coal residues are located as the output quantity. The carriage position information includes the carriage bottom, carriage walls, and carriage corners.

[0036] The working principle and beneficial effects of the above technical solution are as follows: Through the coal combustion residue area locking and evaluation module, the locking and evaluation process of the coal combustion residue area is further refined. The specific location of the coal combustion residue area is determined through the recognition model, and its severity is evaluated by combining factors such as pixel information and environmental temperature and humidity, enabling the system to more accurately identify and evaluate the coal combustion residue, and thus taking targeted measures.

[0037] Embodiment 4 Based on Embodiment 3, the coal combustion residue area locking sub-module includes: The coal combustion background pixel acquisition unit is used to acquire the pixel value of each pixel point in the key frame image with the recognition result of coal combustion residue, calculate the pixel mean value of the adjacent pixel points corresponding to each pixel point in each key frame image, and use it as the background pixel of this pixel point in this key frame image. The coal combustion background acquisition unit calculates the sum of the background pixels of the corresponding pixel points on several key frame images with the recognition result of coal combustion residue, and takes the quotient of the sum of the background pixels of the corresponding pixel points on the key frame images with the recognition result of coal combustion residue and the number of key frame images with the recognition result of coal combustion residue as the generalized background pixel value of this pixel point, and takes the image composed of the generalized background pixel values of each pixel point as the coal combustion background image. The coal combustion residue area locking unit calculates the absolute value of the difference between the pixel value of each pixel point in each key frame image with the recognition result of coal combustion residue and its generalized background pixel value. When the absolute value of the difference between the pixel value of the pixel point and its generalized background pixel value is greater than the preset pixel difference, mark this pixel point as a pixel point in the coal combustion residue area, and the area composed of all pixel points in the coal combustion residue area is the coal combustion residue area.

[0038] The working principle and beneficial effects of the above technical solution: By acquiring the background pixel of this pixel point in the key frame image, acquiring the generalized background pixel value of each pixel point, and acquiring the pixel points in the coal combustion residue area, the coal combustion residue area can be effectively separated from the image, providing reliable data support for subsequent evaluation and monitoring.

[0039] Embodiment 5 Based on Embodiment 4, the coal combustion residue area evaluation sub-module includes: The pixel point number statistics unit is used to count the number of pixel points in the coal combustion residue area in the coal combustion residue area. The laser detection unit is used to detect the thickness of the coal pile in the coal combustion residue area. The environmental temperature and humidity detection unit is used to detect the current temperature value and humidity value of the carriage. The coal pile severity calculation unit is used to calculate the current coal pile severity coefficient in the coal combustion residue area. The current coal pile severity coefficient in the coal combustion residue area: (1); where is the current coal accumulation severity coefficient of the x-th coal residue area, is the weight value of the coal residue area, is the number of pixel points in the coal residue area of the x-th coal residue area, is the number of pixel points in the reference coal residue area corresponding to the maximum acceptable coal residue area, is the weight value of the coal thickness, is the coal accumulation thickness of the x-th coal residue area, is the acceptable reference coal thickness, e is a natural number, and its value is 2.71, is the current temperature and humidity weight value of the carriage, is the preset reference temperature value of the carriage, is the current humidity value of the carriage, is the current temperature value of the carriage, is the preset humidity value of the coal.

[0040] The working principle and beneficial effects of the above technical solution are as follows: By counting the number of pixel points, detecting the coal thickness, and detecting environmental temperature and humidity and other parameters, combined with the preset weight values, the severity of the coal residue area is comprehensively evaluated, and the danger level of the coal residue can be quantified, providing a scientific basis for subsequent monitoring and intervention.

[0041] Example 6 On the basis of Example 5, the key monitoring area marking module includes: A marking judgment unit, which is used to compare the current coal accumulation severity coefficient of the coal residue area with the reference coal accumulation severity coefficient. If the current coal accumulation severity coefficient of the coal residue area is greater than the reference coal accumulation severity coefficient, the coal residue area is marked; A marking execution unit, which is used to correspond the pixel points of the coal residue area to be marked one by one with the coordinates of the three-dimensional point cloud model of the carriage, and use the corresponding coordinate points as the marking coordinate points. A color marking is performed on the marking coordinate points. If there are overlapping marking coordinate points in the remaining coal residue areas and the current coal residue area, repeated color overlay marking is performed on the marking coordinate points, and the saturation of each marking overlay is the same; A marking elimination unit, which is used to count the number of markings of each marking coordinate point. If the number of markings of the current marking coordinate point is zero in y consecutive detection cycles, a marking elimination is performed on the marking coordinate point, reducing one saturation. The area composed of all marking coordinate points is the key monitoring area.

[0042] The working principle and beneficial effects of the above technical solution are as follows: through the joint action of the marking judgment unit, the marking execution unit and the marking elimination unit, the final three-dimensional point cloud model of the carriage is presented in two colors. For example, if the marking color is red, in the area marked in red in the three-dimensional point cloud model of the carriage, the higher the saturation, the more serious the coal accumulation in the area.

[0043] Example 7 Based on Example 6, the monitoring control module includes: The monitoring area partitioning unit is used to partition the key monitoring area, take the areas composed of the marked coordinate points of the same saturation as the same type of areas to be sorted, sort the color saturations corresponding to several types of areas to be sorted, and sort the areas to be sorted from large to small according to the color saturation; The to-be-sorted area segmentation unit is used to segment each to-be-sorted area to obtain a number of discrete key monitoring areas corresponding to each to-be-sorted area; The surveillance camera acquisition times determination unit is used for the ranking results corresponding to each discrete key surveillance area to determine the video acquisition times of the surveillance camera in each discrete key surveillance area within the detection cycle: (2); among which, is the number of video acquisitions by the surveillance camera in the discrete key surveillance area with the ranking result j within the detection cycle, is the logarithm to base e, is the number of ranking result types, is the ranking number of the current discrete key monitoring area, is the maximum ranking number among all discrete key monitoring areas, It is the minimum ranking number among all discrete key monitoring areas.

[0044] In this embodiment, the higher the ranking of the level with the greatest color saturation, the smaller the corresponding level number, and the smallest level number starts from 1.

[0045] The working principle and beneficial effects of the above technical solution are as follows: The monitored area zoning unit ranks the areas to be sorted according to the color saturation from large to small. Then, the area segmentation unit for the areas to be sorted segments each area to be sorted into several discrete key monitored areas. Finally, the video capture times determination unit for the monitoring cameras determines the number of video captures for each discrete key monitored area within the detection period. The larger the ranking number of the current discrete key monitored area, the lower the corresponding color saturation, indicating that the severity of coal accumulation and the number of consecutive coal accumulations in this discrete key monitored area are less. Thus, the video capture for this area can be reduced, while the video capture times for the areas with smaller ranking numbers of discrete key monitored areas can be increased for key monitoring, so as to better allocate monitoring resources, improve the monitoring efficiency and response speed.

[0046] Embodiment 8 Based on Embodiment 7, the area segmentation unit for the areas to be sorted includes: The segmentation contour determination subunit is used to count the number of same-type marked coordinate points within a preset radius spherical space centered on each marked coordinate point of the area to be sorted. If the number of marked coordinate points with the same saturation is lower than the preset number, then this marked coordinate point is used as a contour coordinate point, and the closed line formed by all adjacent contour coordinate points is used as the segmentation contour line for each discrete key monitored area; The discrete key monitored area determination unit is used to take the area formed by the marked coordinate points of the area to be sorted corresponding to each segmentation contour line as the discrete key monitored area.

[0047] The working principle and beneficial effects of the above technical solution are as follows: The area segmentation unit for the areas to be sorted can decompose each area to be sorted into multiple discrete key monitored areas, which is convenient for management and precise monitoring, and improves the monitoring efficiency.

[0048] Embodiment 9 Based on Embodiment 7, it further includes an artificial intervention alarm prompt unit, and the artificial intervention alarm prompt unit includes: The coordinate point quantity statistics unit is used to count the total quantity of marked coordinate points corresponding to the saturation of each discrete key monitored area in each detection period; The alarm judgment unit is used to count the increase in the total quantity of marked coordinate points corresponding to the saturation of the same discrete key monitored area in two adjacent detection periods. If the increase in the total quantity of marked coordinate points corresponding to the saturation of the same discrete key monitored area in two adjacent detection periods is greater than the preset increase, then an artificial intervention alarm prompt for the corresponding discrete key monitored area is triggered.

[0049] The working principle and beneficial effects of the above technical solution are as follows: By determining that the increase in the total number of marked coordinate points corresponding to the saturation degrees in the same discrete key monitoring areas in two adjacent detection cycles is greater than the preset increase, the degree of area expansion of the key monitoring areas corresponding to the two adjacent detection cycles can be judged, so as to know the continuous coal accumulation degree of the discrete key monitoring areas, and then determine whether timely manual intervention is required to minimize losses to the greatest extent.

[0050] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and its equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

1. The coal residue detection system inside the carriage based on intelligent image recognition is characterized by: include: The video monitoring module is used to obtain the collected videos of all monitoring cameras in the car and obtain several key frame images based on the collected videos; A carriage 3D point cloud model building module is used to build a carriage 3D point cloud model of the corresponding carriage based on a number of key frame images; A coal residue preliminary recognition module is used to input a number of key frame images into a trained coal residue recognition model to obtain coal residue recognition results of a number of key frame images; The coal residue area locking assessment module is used to obtain the coal residue area corresponding to the key frame image whose recognition result is coal residue, confirm the position of the carriage where the coal residue area is located, and assess the current coal accumulation severity in the coal residue area; The key monitoring area marking module is used to confirm whether to mark the key monitoring area in the corresponding area of ​​the three-dimensional point cloud model of the carriage based on the current coal accumulation severity and the preset coal accumulation severity in the coal residue area; The monitoring control module is used to control the operation of the monitoring camera based on the marking results of the key monitoring areas of the three-dimensional point cloud model of the carriage, and to provide manual intervention alarm prompts based on the feedback results collected multiple times by the monitoring camera.

2. The system for detecting coal residue inside a carriage based on intelligent image recognition according to claim 1 is characterized in that: The module for building the 3D point cloud model of the carriage includes: An image preprocessing unit, used for performing image preprocessing on a plurality of key frame images; A feature extraction unit, used for extracting features from the preprocessed key frame image; A feature matching unit, used for matching feature points in a plurality of key frame images; A three-dimensional reconstruction unit is used to obtain the coordinate positions of feature points in three-dimensional space based on the position information of feature point pairs in different key frame images, and to construct a preliminary three-dimensional point cloud model; Point cloud optimization unit, used to optimize the preliminary three-dimensional point cloud model; The texture mapping unit is used to map the texture information of the key frame image to the optimized three-dimensional point cloud model to obtain the three-dimensional point cloud model of the corresponding carriage.

3. The system for detecting coal residue in a carriage based on intelligent image recognition according to claim 1 is characterized in that: The Coal Residual Zone Lock Assessment Module includes: A coal residue area locking submodule is used to lock the coal residue area based on the key frame image whose recognition result is coal residue; The coal residue area position recognition module is used to input the key frame image with the recognition result of coal residue into the trained coal residue area position recognition model to obtain the position information of the current coal residue area in the carriage, and the position information includes the carriage bottom, carriage wall and carriage corner; The coal residue area assessment submodule is used to assess the severity of the current coal accumulation in the coal residue area based on the pixel information of the coal residue area, the thickness of the coal accumulated in the coal residue area and the current ambient temperature and humidity of the carriage.

4. The system for detecting coal residue inside a carriage based on intelligent image recognition according to claim 3 is characterized in that: The coal burning residue area locking submodule includes: The coal burning background pixel acquisition unit is used to acquire the pixel value of each pixel point in the key frame image whose recognition result is coal burning residue, and calculate the pixel mean value of the adjacent pixel points corresponding to each pixel point of each key frame image, and use it as the background pixel of the pixel point of the key frame image; The coal burning background acquisition unit calculates the sum of background pixels of corresponding pixels on a number of key frame images whose recognition results are coal burning residues, and uses the quotient of the sum of background pixels of corresponding pixels on the key frame images whose recognition results are coal burning residues and the number of key frame images whose recognition results are coal burning residues as the generalized background pixel value of the pixel, and uses the image composed of the generalized background pixel value of each pixel as the coal burning background image; The coal residue area locking unit calculates the absolute value of the difference between the pixel value of each pixel point in each key frame image whose identification result is coal residue and its generalized background pixel value. When the absolute value of the difference between the pixel value of the pixel point and its generalized background pixel value is greater than the preset pixel difference, the pixel point is marked as a coal residue area pixel point, and the area formed by all coal residue area pixels is the coal residue area.

5. The system for detecting coal residue in a carriage based on intelligent image recognition according to claim 4 is characterized in that: The coal combustion residual area assessment submodules include: A pixel point number counting unit is used to count the number of pixels in the coal burning residue area; A laser detection unit is used to detect the thickness of coal accumulated in the coal residue area; Ambient temperature and humidity detection unit, used to detect the current temperature and humidity values ​​of the compartment; The coal accumulation severity calculation unit is used to calculate the current coal accumulation severity coefficient in the coal residual area.

6. The system for detecting coal residue in a carriage based on intelligent image recognition according to claim 5 is characterized in that: Current coal accumulation severity coefficient in coal-burning residual area: (1); among which, is the current coal accumulation severity coefficient of the xth coal residue area, is the weight value of the coal burning residual area, is the number of pixels in the coal-burning residue area of ​​the x-th coal-burning residue area, is the number of pixels in the benchmark coal residue area corresponding to the maximum acceptable coal residue area, is the coal thickness weight value, is the coal accumulation thickness in the xth coal-burning residual area, is the acceptable benchmark coal thickness, e is a natural number, and its value is 2.

71. is the current temperature and humidity weight value of the compartment, is the preset cabin reference temperature value, is the current humidity value in the car, is the current temperature of the car, Preset humidity value for coal.

7. The system for detecting coal residue inside a carriage based on intelligent image recognition according to claim 1 is characterized in that: The key monitoring area marking module includes: A marking judgment unit is used to compare the current coal accumulation severity coefficient of the coal residue area with the benchmark coal accumulation severity coefficient, and if the current coal accumulation severity coefficient of the coal residue area is greater than the benchmark coal accumulation severity coefficient, the coal residue area is marked; The marking execution unit is used to make a one-to-one correspondence between the coal residue area pixel points of the coal residue area to be marked and the coordinates of the three-dimensional point cloud model of the carriage, take the corresponding coordinate points as marking coordinate points, perform color marking on the marking coordinate points once, and if there are overlapping marking coordinate points of the remaining coal residue areas and the current coal residue area, then the marking coordinate points are repeatedly color-superimposed and marked, and the saturation of each marking superposition is the same; The marking elimination unit is used to count the number of marking times of each marking coordinate point. If the number of marking times of the current marking coordinate point for y consecutive detection cycles is zero, the marking coordinate point is marked once to reduce one saturation, and the area formed by all marking coordinate points is the key monitoring area.

8. The system for detecting coal residue in a carriage based on intelligent image recognition according to claim 7 is characterized in that: The monitoring and control module includes: The monitoring area partitioning unit is used to partition the key monitoring area, take the areas composed of the marked coordinate points of the same saturation as the same type of areas to be sorted, sort the color saturations corresponding to several types of areas to be sorted, and sort the areas to be sorted from large to small according to the color saturation; The to-be-sorted area segmentation unit is used to segment each to-be-sorted area to obtain a number of discrete key monitoring areas corresponding to each to-be-sorted area; The surveillance camera acquisition times determination unit is used for the ranking results corresponding to each discrete key surveillance area to determine the video acquisition times of the surveillance camera in each discrete key surveillance area within the detection cycle: (2); among which, is the number of video acquisitions by the surveillance camera in the discrete key surveillance area with the ranking result j within the detection cycle, is the logarithm to base e, is the number of ranking result types, is the ranking number of the current discrete key monitoring area, is the maximum ranking number among all discrete key monitoring areas, It is the minimum ranking number among all discrete key monitoring areas.

9. The system for detecting coal residue inside a carriage based on intelligent image recognition according to claim 8 is characterized in that: The area segmentation unit to be sorted includes: The segmentation contour determination subunit is used to count the number of the same type of marked coordinate points in the space sphere with a preset radius and each marked coordinate point of the area to be sorted as the center. If the number of marked coordinate points with the same saturation is lower than the preset number, the marked coordinate point is used as the contour coordinate point, and the closed line surrounded by all adjacent contour coordinate points is used as the segmentation contour line of each discrete key monitoring area; The discrete key monitoring area determination unit is used to take the area formed by the marked coordinate points corresponding to the to-be-sorted area within each segmentation contour line as the discrete key monitoring area.

10. The system for detecting coal residue inside a carriage based on intelligent image recognition according to claim 8, characterized in that: It also includes a manual intervention alarm prompt unit, which includes: A coordinate point quantity statistics unit is used to count the total number of marked coordinate points corresponding to the saturation of each discrete key monitoring area in each detection cycle; The alarm judgment unit is used to count the increase in the total number of marked coordinate points of the same discrete key monitoring area corresponding to the saturation in two adjacent detection cycles. If the increase in the total number of marked coordinate points of the same discrete key monitoring area corresponding to the saturation in two adjacent detection cycles is greater than the preset increase, the manual intervention alarm prompt for the corresponding discrete key monitoring area is triggered.

Images