AI analysis and detection method and device for coal mine power distribution room

Abstract

The invention discloses an AI analysis and detection method and device for a coal mine power distribution room, and belongs to the field of AI intelligent analysis and detection. The method comprises the steps of judging whether a current video image of an AI camera of a coal mine power distribution room needs to be subjected to image enhancement or not; if yes, performing angle transformation on an infrared image which is shot by a thermal imaging camera and is located at the same position as the current video image, and inputting the infrared image into a pre-trained growth adversarial network model, so as to obtain the infrared image according to the mapping relation between the thermal radiation information of the infrared image and the texture information of the visible light image, a new visible light video image is generated for AI analysis detection; and on the basis of the AI analysis detection result and the temperature monitoring of the thermal imaging camera, performing in-well and out-well early warning on each corresponding detection event of the coal mine power distribution room. According to the scheme, the detail texture information of the infrared image can be injected into the current video image, so that the visible light image in the dark coal mine can be effectively enhanced, and the accuracy of AI analysis detection and early warning is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of AI intelligent analysis and detection, and particularly to an AI analysis and detection method and device for a coal mine shaft power distribution room. Background Art

[0002] With the rapid development of the coal mining industry in China, not only can the operation status of coal mining machinery and equipment be monitored by using AI intelligent analysis technology, but also the behavior safety of coal miners can be detected and supervised by AI.

[0003] For a coal mine shaft power distribution room, not only scene detection is required, but also temperature warning is needed. With the development and application of AI intelligent technology, a high-definition spherical camera can be installed in the coal mine shaft power distribution room to perform AI detection by using AI intelligent analysis technology, and at the same time, a mine-used thermal imaging camera is equipped to achieve temperature monitoring. However, due to the complex terrain and dim lighting in coal mines, the images captured by cameras are prone to problems such as low brightness, serious noise, and loss of details. Traditional image enhancement methods mainly increase the contrast of images through histogram equalization and Laplace transform, etc. But these methods do not consider the context information in the images and cannot obtain real and clear images, which greatly affects the accuracy of AI analysis and detection in the inclined shaft for hoisting in the auxiliary shaft of the coal mine, resulting in ineffective early warnings.

[0004] Therefore, there is an urgent need to provide a new AI analysis and detection method for a coal mine shaft power distribution room. Summary of the Invention

[0005] In order to solve the problem that the existing image enhancement methods cannot perform high-precision detail restoration on the blurred images captured by AI cameras underground in coal mines, thereby affecting the effectiveness of AI monitoring and early warning in the inclined shaft for hoisting in the auxiliary shaft of the coal mine, the embodiments of the present invention provide a comprehensive monitoring and early warning method and device for the inclined shaft for hoisting in the auxiliary shaft of the coal mine.

[0006] On the one hand, an AI analysis and detection method for a coal mine shaft power distribution room is provided. The method includes:

[0007] Determine whether the current video image of the AI camera in the coal mine shaft power distribution room needs to be image-enhanced;

[0008] If so, perform angle transformation on the infrared image taken by the mine-used thermal imaging camera at the same position as the current video image, and then input it into a pre-trained growth adversarial network model to generate a new visible light video image by using the mapping relationship between the pre-learned thermal radiation information of the infrared image and the texture information of the visible light image, and perform AI analysis and detection after fusing the new visible light video image and the current video image; if not, directly perform AI analysis and detection based on the current video image;

[0009] Based on the AI analysis and detection results of the AI camera and the temperature monitoring of the mine thermal imaging camera, early warnings inside and outside the mine are given for each corresponding detection event in the coal mine shaft power distribution room.

[0010] On the other hand, an AI analysis and detection device for a coal mine shaft power distribution room is provided, which is used to implement the steps described in any method embodiment of the specification. The device includes:

[0011] A judgment unit, which is used to judge whether the current video image of the AI camera in the coal mine shaft power distribution room needs to be image-enhanced;

[0012] An analysis unit, if so, is used to input the infrared image taken by the mine thermal imaging camera at the same position as the current video image after angle transformation into a pre-trained growing adversarial network model, so as to utilize the mapping relationship between the thermal radiation information of the pre-learned infrared image and the texture information of the visible light image to generate a new visible light video image, and to perform AI analysis and detection after fusing the new visible light video image and the current video image; if not, directly perform AI analysis and detection based on the current video image;

[0013] An early warning unit, which is used to give early warnings inside and outside the mine for each corresponding detection event in the coal mine shaft power distribution room based on the AI analysis and detection results of the AI camera and the temperature monitoring of the mine thermal imaging camera.

[0014] On the other hand, a computer device is provided. The computer device includes a memory and a processor. The memory is used to store a computer program, and the processor is used to execute the computer program stored on the memory to implement the steps of the above-mentioned method.

[0015] On the other hand, a computer-readable storage medium is provided. The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned method are implemented.

[0016] On the other hand, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned method are implemented.

[0017] The technical solution provided by the present invention can at least bring the following beneficial effects:

[0018] By using the growing adversarial network model, the infrared image taken by the thermal imaging camera is mapped and transformed into a visible light video image, and then fused with the current video image, the detailed texture information of the infrared image can be injected into the current video image, so as to effectively enhance the visible light image in a dim coal mine and improve the accuracy of AI analysis, detection and early warning in the coal mine shaft power distribution room. Description of the Drawings

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0020] Figure 1 It is a flowchart of an AI analysis and detection method for a coal mine shaft power distribution room provided by an embodiment of the present invention;

[0021] Figure 2 It is a structural diagram of an AI analysis and detection device for a coal mine shaft power distribution room provided by an embodiment of the present invention;

[0022] Figure 3 It is a hardware architecture diagram of a computer device provided by an embodiment of the present invention. Detailed implementation manners

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0024] The following describes the specific implementation manners of the above concepts.

[0025] Please refer to Figure 1 , an AI analysis and detection method for a coal mine shaft power distribution room provided by an embodiment of the present invention, the method includes:

[0026] Step 100: Determine whether the current video image of the AI camera in the coal mine shaft power distribution room needs to be image-enhanced;

[0027] Step 102: If so, perform an angle transformation on the infrared image taken by the mine thermal imaging camera at the same position as the current video image and input it into a pre-trained growth adversarial network model to generate a new visible light video image by using the mapping relationship between the pre-learned thermal radiation information of the infrared image and the texture information of the visible light image, and perform AI analysis and detection after fusing the new visible light video image and the current video image; if not, directly perform AI analysis and detection based on the current video image;

[0028] Step 104: Based on the AI analysis and detection results of the AI camera and the temperature monitoring of the mine thermal imaging camera, give early warnings inside and outside the well for each corresponding detection event in the coal mine shaft power distribution room.

[0029] In the embodiments of the present invention, by using a generative adversarial network model, the infrared images captured by a thermal imaging camera are mapped and transformed into visible light video images, and then fused with the current video images. The detailed texture information of the infrared images can be injected into the current video images, so as to effectively enhance the visible light images in a dim coal mine and improve the accuracy of AI analysis detection and early warning in the distribution room of the coal mine shaft.

[0030] The following describes Figure 1 the execution manners of the respective steps shown.

[0031] For step 100:

[0032] A high-definition spherical camera and a mine thermal imaging camera KBA12R can be installed in the distribution room. The device has two channels. The visible light channel is used for AI scene detection, and the thermal imaging channel is used for equipment temperature measurement and alarm. By installing a two-channel photography device in the distribution room, real-time monitoring of the details of the hoist and real-time monitoring of the temperature of the power distribution equipment are realized, and real-time monitoring and alarm of the state of the electromechanical equipment of the auxiliary shaft hoist in the coal mine are realized, ensuring the safety of the equipment performance.

[0033] It can be understood that AI video analysis includes comprehensive monitoring and full-process video recording of the behaviors of operating personnel and the states of mechanical equipment, forming a visual, traceable, and accountable video safety supervision and management system.

[0034] In some embodiments, step 100 may include:

[0035] Determining the operating state of the AI camera based on the confidence level and image frame rate of AI analysis detection within a historical set time;

[0036] Evaluating the quality of the current video image based on the contrast and blur of the current video image;

[0037] Judging whether to perform image enhancement on the current video image based on the confidence level and image frame rate of AI analysis detection within a historical set time and the contrast and blur of the current video image.

[0038] In this embodiment, for each AI camera, the operating state of the AI camera and the quality of the video image can be comprehensively considered to first judge whether the current video image needs to be enhanced. If it needs to be enhanced, it will be enhanced and fused and then AI analysis detection will be performed. Otherwise, AI analysis detection will be directly performed, so as to avoid wasting the computing power resources of the backend intelligent analysis platform.

[0039] The confidence and image frame rate of AI analysis detection within the historical set time are used to evaluate the operating status of the AI ​​camera. Abnormal confidence may cause confusion and unreliability in AI analysis detection. Low frame rate or unstable frame rate may make the target's motion trajectory between consecutive frames incoherent, which may not be directly reflected in the video image quality. However, due to the incoherence of the video frame, the noise and changes in the image may be mistakenly identified as the target, thereby increasing the probability of false alarms. Therefore, the confidence and image frame rate of AI analysis detection within the historical set time can be monitored. When the confidence of the AI ​​intelligent analysis for the recognition result is abnormal or the video frame rate is abnormal, on the one hand, the staff can be notified to perform fault maintenance, and on the other hand, the video image is enhanced and fused to improve the accuracy of AI analysis detection. Therefore, this embodiment not only determines whether image enhancement is needed based on the quality of the video image, but also takes into account the operating status of the AI ​​camera, fully integrates the adverse factors that affect the accuracy of AI analysis, ensures the accuracy and effectiveness of the judgment, and thus improves the accuracy of AI analysis warning on the basis of saving computing resources.

[0040] In some embodiments, “determining whether to perform image enhancement on the current video image based on the confidence level, image frame rate, and contrast and blur of the current video image detected by AI analysis within a historical set time” includes:

[0041] The composite index z is calculated based on the following formula:

[0042]

[0043] Among them, Q p Q is the average confidence value of AI analysis and detection within the historical set time. p′ is the confidence standard value, F ps is the image frame rate, F ps ′ is the standard value of image frame rate, D v is the standard value of the current video image, x i is the gray value of each pixel of the current video image, is the grayscale mean of the current video image, N is the number of pixels in the current video image, s i is the fuzzy block area, S is the effective area of ​​the current video image, n is the number of blocks, g 1 , g 2 , g 3 and g 4 is the weight factor;

[0044] When the comprehensive index of the current video image is less than a preset threshold, it is determined that the current video image needs to be enhanced, otherwise, image enhancement is not required.

[0045] In this embodiment, compared with the quality of video images, the operating state of the AI camera is a secondary factor. Therefore, using logarithmic operations can reduce the impact of the operating state of the AI camera on the comprehensive index, and can more accurately quantify and evaluate the comprehensive index to fully integrate the adverse factors affecting the accuracy of AI analysis, ensuring the accuracy and effectiveness of the judgment on whether image enhancement is required. Furthermore, on the basis of saving computing resources, the accuracy of AI analysis and early warning can be improved.

[0046] Regarding step 102:

[0047] In some embodiments, the growth adversarial network model is trained in the following manner:

[0048] The generator of the growth adversarial network model contains several learning modules with gradually increasing convolutional channels until the convolutional channels of the last learning module are the same as the size of the video image. The convolutional channels of the learning modules gradually increase to map the input infrared image from a low-resolution image to a high-resolution image layer by layer to generate a visible light image;

[0049] For each learning module of the generator, a discriminant module corresponding to the learning module is designed to form a discriminator; among them, the generator is used to generate a visible light image based on the infrared image, and the discriminator is used to distinguish the visible light image generated by the generator from the real visible light image sample;

[0050] Use the standard normal distribution to initialize the weights;

[0051] Use the training set to train the growth adversarial network model to generate a visible light image using the infrared image samples in the training set, and calculate the difference between the generated visible light image and the real visible light image samples in the training set. Use the stochastic gradient descent method to optimize the network parameters to update the network parameters of the generator and the discriminator.

[0052] In this embodiment, the growth adversarial network includes a generator and a discriminator. The generator contains several learning modules with gradually increasing convolutional channels until the convolutional channels of the last learning module are the same as the size of the video image; the discriminator contains several discriminant modules corresponding one by one to the learning modules. In addition, add a layer at the end of the discriminator to gradually downsample and learn large tensors, and project the input into a set of statistical information. With the goal of minimizing the generation error and the feature matching error, construct the loss function of the generative adversarial network.

[0053] In the growth adversarial network model of this embodiment, it includes 4 learning modules. The convolutional channels of the learning modules of the generator gradually increase until the output image of the last learning module has the same size as the video image output by the AI camera. This smooth growth adversarial network can smoothly advance the mapping relationship between the thermal radiation information of the infrared image and the texture information of the visible light image from a low-resolution image to a high-resolution image. Compared with the traditional generative adversarial network, which mostly directly trains and learns the original video image, the progressive learning of this solution can effectively learn the mapping relationship with rich details and improve the quality of the generated new visible light video image.

[0054] In some embodiments, each learning module includes a low-resolution convolutional block and a high-resolution convolutional block. The convolutional channels of the high-resolution convolutional block in this learning module are greater than those of the low-resolution convolutional block, and the convolutional channels of the low-resolution convolutional block of the next learning module are equal to those of the high-resolution convolutional block of this learning module, so that the convolutional channels of the learning modules in the generator gradually increase, and the input infrared image is gradually mapped layer by layer from a low-resolution image to a high-resolution image to generate a visible light image.

[0055] In this embodiment, the low-resolution convolutional block in the first learning module is 16×16, and the high-resolution convolutional block is 32×32; the low-resolution convolutional block in the second learning module is 32×32, and the high-resolution convolutional block is 64×64, and so on. Assuming the video image size is 256×256, then there is also a learning module and a discriminator module in the middle, the low-resolution convolutional block is 64×64, the high-resolution convolutional block is 128×128, the low-resolution convolutional block of the last learning module is 128×128, and the high-resolution convolutional block is 256×256.

[0056] In this embodiment, the learning module is trained using the standard deviation structure. The standard deviation structure includes a low-resolution convolutional block, an upsampling layer, two branches, and a weighted summation layer. Among them, one branch is used to generate a first feature image using the upsampled image, and the other branch contains a high-resolution convolutional block to generate a second feature image. The first feature image and the second feature image are weighted and summed in a ratio of 1 - t:t to output a feature image. The image enhancement method using the standard deviation structure can effectively learn the mapping relationship with rich details and improve the quality of the generated new visible light video image.

[0057] In this embodiment, the discrimination module is also trained using the standard deviation structure. The discrimination module includes, from top to bottom, two branches, a weighted summation layer, several convolutional blocks with gradually decreasing convolutional kernels, and a downsampling layer between every two convolutional blocks. Among them, the third branch downsamples the feature image to obtain a third feature image. The fourth branch contains a convolutional layer and a downsampling layer of the same size as the high-resolution convolutional block of the corresponding learning module to obtain a fourth feature image. The third feature image and the fourth feature image are weighted and summed at a ratio of 1 - t:t until they are downsampled to the same size as the low-resolution convolutional block of the first learning module, thereby improving the training accuracy of the corresponding learning module.

[0058] In some embodiments, the step of "training the growth adversarial network model using the training set to generate visible light images from the infrared image samples in the training set, calculating the difference between the generated visible light images and the real visible light image samples in the training set, and using the stochastic gradient descent method to optimize the network parameters to update the network parameters of the generator and the discriminator" may include:

[0059] Input the infrared image samples in the training set into the first learning module, and based on the standard deviation structure of the current scale factor, train the network parameters of the high-resolution convolutional block of the first learning module, and output the feature image to the corresponding first discrimination module;

[0060] The first discrimination module calculates the loss function based on the feature image and the corresponding real visible light image sample and feeds it back to the first learning module;

[0061] Increase the scale factor of the first learning module multiple times, and each time after the increase, jump to execute the training of the network parameters of the high-resolution convolutional block of the first learning module based on the standard deviation structure of the current scale factor until the scale factor is 1, complete the multiple training of the low-resolution convolutional block and the high-resolution convolutional block of the first learning module, and select the scale factor of this learning module by comparing the loss function values under each scale factor;

[0062] Inherit the network parameters of the high-resolution convolutional block of the first learning module to the low-resolution convolutional block of the second learning module;

[0063] Input the feature image output by the first learning module under the selected scale factor into the second learning module to train the network parameters of the high-resolution convolutional block of the second learning module based on the standard deviation structure of the current scale factor, and output the feature image to the corresponding second discrimination module;

[0064] The second discrimination module calculates the loss function based on the feature image and the corresponding real visible light image sample and feeds it back to the second learning module;

[0065] Increase the proportionality coefficient of the first learning module multiple times, and after each increase, jump to execute the standard deviation structure based on the current proportionality coefficient, and train the network parameters of the high-resolution convolutional block of the second learning module until the proportionality coefficient is 1, completing multiple trainings of the low-resolution convolutional block and the high-resolution convolutional block of the second learning module, and select the proportionality coefficient of this learning module by comparing the loss function values under each proportionality coefficient;

[0066] And so on, train the remaining learning modules in turn until the loss function of the last discriminant module is less than the set threshold, and obtain the growth adversarial network model.

[0067] In this embodiment, each learning module will be trained multiple times. The proportionality coefficient t for the first time is 0, and it increases in steps. For example, 0.25 is the step. The proportionality coefficient t for the second execution is 0.25, until the proportionality coefficient t for the fifth execution is 1. The first learning module and the first discriminant module are repeatedly executed 5 times to obtain the final network parameters of the high-resolution convolutional block of the first learning module, and the final network parameters are copied to the low-resolution convolutional block of the second learning module. It can be understood that each learning module is repeatedly executed five times to complete the training of this learning module and achieve smooth progressive learning from a low-resolution image to a high-resolution image.

[0068] Therefore, the embodiment of the present invention improves the traditional adversarial network, constructs a smooth growth adversarial network, and uses the trained growth adversarial network model to smoothly advance from a low-resolution image to a high-resolution image, so as to fully learn the mapping relationship between the thermal radiation information of the infrared image and the texture information of the visible light image at multiple levels, and generate a new visible light video image. Compared with the traditional image enhancement method, progressive enhancement can effectively learn the mapping relationship of rich details, improve the quality of the generated new visible light video image, and improve the AI analysis and detection accuracy of the coal mine shaft distribution room. In addition, it not only judges whether image enhancement is needed based on the quality of the video image, but also considers the operating state of the AI camera, fully synthesizes the adverse factors affecting the AI analysis accuracy, ensures the accuracy and effectiveness of the judgment, and then improves the AI analysis and early warning accuracy on the basis of saving computing power resources. Therefore, this solution can not only effectively improve the quality of the generated new visible light video image, improve the AI analysis, detection and early warning accuracy of the auxiliary shaft hoisting inclined lane of the coal mine, but also avoid waste of computing power resources.

[0069] In some embodiments, "fusing the new visible light video image and the current video image and then performing AI analysis and detection" includes:

[0070] Based on the pixel brightness of the new visible light video image, calculate the fusion weight of each pixel:

[0071]

[0072] Wherein, I(x) is the brightness of the x-th pixel of the new visible light video image, and T(x) is the pixel value of the x-th pixel of the new visible light video image. is the pixel value of the x-th pixel of the current video image. is the convolution operation, K is the convolution kernel, ρ is the control parameter, k is the brightness threshold, and α is the adjustment system, with a value less than e.

[0073] After multiplying the pixel value at each position of the new visible light video image by its corresponding fusion weight and adding it to the corresponding pixel of the current video image, a fused video image is obtained for AI analysis and detection based on the fused video image.

[0074] In this embodiment, by calculating the fusion weight of each pixel point based on the pixel brightness of the new visible light video image, it can be understood that the higher the brightness in the formula, the higher the fusion weight. Fusing the new visible light video image with the current video image based on the fusion weight can effectively enhance the spatial texture detail information of the current video image, further achieving effective enhancement of the visible light image under a dim coal mine and improving the accuracy of AI analysis, detection, and early warning in the coal mine substation.

[0075] Please refer to Figure 2 , this embodiment of the present invention provides an AI analysis and detection device for a coal mine substation, which is used to implement the steps of any method embodiment in the specification. The device includes:

[0076] A judgment unit 201, configured to judge whether the current video image of the AI camera in the coal mine substation needs to be image-enhanced;

[0077] An analysis unit 202, if so, is configured to input the infrared image taken by the mine thermal imaging camera at the same position as the current video image after angle transformation into a pre-trained growing adversarial network model, so as to utilize the mapping relationship between the thermal radiation information of the pre-learned infrared image and the texture information of the visible light image to generate a new visible light video image, and perform AI analysis and detection after fusing the new visible light video image and the current video image; if not, directly perform AI analysis and detection based on the current video image;

[0078] An early warning unit 203, configured to perform in-well and out-of-well early warning on each corresponding detection event in the coal mine substation based on the AI analysis and detection results of the AI camera and the temperature monitoring of the mine thermal imaging camera.

[0079] In an embodiment of the present invention, the judgment unit 201 is configured to execute:

[0080] Determine the operating state of the AI camera based on the confidence level and image frame rate of AI analysis and detection within a historical set time.

[0081] Evaluate the quality of the current video image based on the contrast and blurriness of the current video image;

[0082] Based on the confidence level of AI analysis and detection, the image frame rate, and the contrast and blurriness of the current video image within the historical set time, determine whether to perform image enhancement on the current video image.

[0083] In an embodiment of the present invention, when the determination unit 201 executes the determination of whether to perform image enhancement on the current video image based on the confidence level of AI analysis and detection, the image frame rate, and the contrast and blurriness of the current video image within the historical set time, it is used for:

[0084] Calculate the comprehensive index z based on the following formula:

[0085]

[0086] where Q p is the average confidence level of AI analysis and detection within the historical set time, Q p′ is the confidence level standard value, F ps is the image frame rate, F ps ′ is the image frame rate standard value, D v is the standard value of the current video image, x i is the gray value of each pixel point of the current video image, is the average gray value of the current video image, N is the number of pixel points of the current video image, s i is the area of the blurred block, S is the effective area of the current video image, n is the number of blocks, g 1 、g 2 、g 3 and g 4 are weight factors;

[0087] When the comprehensive index of the current video image is less than the preset threshold, it is determined that the current video image needs to be enhanced, otherwise it does not need to be enhanced.

[0088] In an embodiment of the present invention, the growth adversarial network model in the analysis unit 202 is trained in the following manner:

[0089] The generator of the growth adversarial network model contains several learning modules with gradually increasing convolutional channels until the convolutional channels of the last learning module are the same as the size of the video image. The convolutional channels of the learning modules gradually increase to map the input infrared image from a low-resolution image to a high-resolution image layer by layer to generate a visible light image;

[0090] For each learning module of the generator, a discriminant module corresponding to the learning module is designed to form a discriminator. Among them, the generator is used to generate visible light images based on infrared images, and the discriminator is used to distinguish the visible light images generated by the generator from the real visible light image samples.

[0091] Initialize the weights using the standard normal distribution.

[0092] Use the training set to train the growth adversarial network model to generate visible light images using the infrared image samples in the training set, calculate the difference between the generated visible light images and the real visible light image samples in the training set, and use the stochastic gradient descent method to optimize the network parameters to update the network parameters of the generator and the discriminator.

[0093] In an embodiment of the present invention, each learning module in the analysis unit 202 includes a low-resolution convolutional block and a high-resolution convolutional block. The number of convolutional channels of the high-resolution convolutional block in the learning module is greater than that of the low-resolution convolutional block, and the number of convolutional channels of the low-resolution convolutional block of the next learning module is equal to the number of convolutional channels of the high-resolution convolutional block of this learning module, so that the convolutional channels of the learning modules in the generator gradually increase to gradually map the input infrared image from a low-resolution image to a high-resolution image to generate a visible light image.

[0094] In an embodiment of the present invention, when the analysis unit 202 performs AI analysis and detection after fusing the new visible light video image and the current video image, it is used for:

[0095] Based on the pixel brightness of the new visible light video image, calculate the fusion weight of each pixel:

[0096]

[0097] where I(x) is the brightness of the x-th pixel of the new visible light video image, T(x) is the pixel value of the x-th pixel point of the new visible light video image, is the pixel value of the x-th pixel point of the current video image, is the convolution operation, K is the convolution kernel, ρ is the control parameter, k is the brightness threshold, and α is the adjustment system, with a value less than e;

[0098] Multiply the pixel value at each position of the new visible light video image by its corresponding fusion weight and then add it to the corresponding pixel of the current video image to obtain the fused video image for AI analysis and detection based on the fused video image.

[0099] It should be noted that: The AI analysis and detection device for the coal mine shaft power distribution room provided in the above embodiments is only illustrated by dividing the above functional units. In actual applications, the above functions can be allocated to different functional units according to needs, that is, the internal structure of the device is divided into different functional units to complete all or part of the functions described above. In addition, the above device embodiments and method embodiments belong to the same concept. For the specific implementation process, please refer to the method embodiments and will not be elaborated here.

[0100] An embodiment of the present application also provides a computer device. Please refer to Figure 3 , which includes a processor and a memory. At least one instruction, at least one program, a code set, or an instruction set is stored in the memory. The at least one instruction, at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the AI analysis and detection method for the coal mine shaft power distribution room provided in each of the above method embodiments.

[0101] An embodiment of the present application also provides a computer-readable storage medium. At least one instruction, at least one program, a code set, or an instruction set is stored on the computer-readable storage medium. The at least one instruction, at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the AI analysis and detection method for the coal mine shaft power distribution room provided in each of the above method embodiments.

[0102] An embodiment of the present application also provides a computer program product. The computer program product includes a computer program. The processor of the computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program to enable the computer device to execute the AI analysis and detection method for the coal mine shaft power distribution room in any one of the above embodiments.

[0103] For the convenience of description, when describing the above system or device, various modules or units are described separately according to functions. Of course, when implementing the present application, the functions of each unit can be implemented in one or more software and / or hardware.

[0104] From the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods in each embodiment or some parts of the embodiments of the present application.

[0105] Finally, it should also be noted that in this text, relational terms such as first, second, third, and fourth are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article, or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0106] The above are only the preferred embodiments of the present application. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. An AI analysis and detection method for a coal mine power distribution room, characterized in that: The method comprises: Determine whether the current video image of the AI ​​camera in the power distribution room of a coal mine needs to be enhanced; If yes, the infrared image taken by the mine thermal imaging camera at the same position as the current video image is input into the pre-trained growing adversarial network model after the angle is transformed, so as to generate a new visible light video image by using the mapping relationship between the thermal radiation information of the infrared image and the texture information of the visible light image learned in advance, so as to fuse the new visible light video image with the current video image for AI analysis and detection; if no, AI analysis and detection is directly performed based on the current video image; Based on the AI ​​analysis detection results of the AI ​​camera and the temperature monitoring of the mine thermal imaging camera, early warnings are issued inside and outside the mine for each corresponding detection event in the power distribution room of the coal mine.

2. The method according to claim 1, characterized in that The determining whether the current video image of the AI ​​camera in the power distribution room of the coal mine needs to be enhanced includes: Determine the operating status of the AI ​​camera based on the confidence and image frame rate of AI analysis detection within the historical set time; Based on the contrast and blur of the current video image, the quality of the current video image is evaluated; Based on the confidence level of AI analysis and detection within the historical set time, the image frame rate, and the contrast and blur of the current video image, determine whether to perform image enhancement on the current video image.

3. The method according to claim 2, characterized in that The step of determining whether to perform image enhancement on the current video image based on the confidence level, image frame rate, and contrast and blur of the current video image detected by the AI ​​analysis within the historical set time includes: The composite index z is calculated based on the following formula: Among them, Q p Q is the average confidence value of AI analysis and detection within the historical set time. p′ is the confidence standard value, F ps is the image frame rate, F ps ′ is the standard value of image frame rate, D v is the standard value of the current video image, x i is the grayscale value of each pixel of the current video image, x is the grayscale mean of the current video image, N is the number of pixels of the current video image, s i is the fuzzy block area, S is the effective area of ​​the current video image, n is the number of blocks, g1, g2, g3 and g4 are weight factors; When the comprehensive index of the current video image is less than a preset threshold, it is determined that the current video image needs to be enhanced, otherwise, image enhancement is not required.

4. The method according to claim 1, characterized in that The growth adversarial network model is trained in the following way: The generator of the growing adversarial network model contains a plurality of learning modules with gradually increasing convolution channels until the convolution channel of the last learning module is the same size as the video image, and the convolution channels of the learning modules are gradually increased to map the input infrared image from a low-resolution image to a high-resolution image layer by layer to generate a visible light image; For each learning module of the generator, a discriminant module corresponding to the learning module is designed to form a discriminator; wherein the generator is used to generate a visible light image based on an infrared image, and the discriminator is used to distinguish between the visible light image generated by the generator and a real visible light image sample; Initialize weights using a standard normal distribution; The growing adversarial network model is trained using the training set to generate visible light images using the infrared image samples in the training set, and the difference between the generated visible light images and the real visible light image samples in the training set is calculated. The network parameters are optimized using the stochastic gradient descent method to update the network parameters of the generator and discriminator.

5. The method according to claim 4, characterized in that Each of the learning modules includes a low-resolution convolution block and a high-resolution convolution block. The convolution channel of the high-resolution convolution block in the learning module is larger than the convolution channel of the low-resolution convolution block, and the convolution channel of the low-resolution convolution block of the next learning module is equal to the convolution channel of the high-resolution convolution block of the learning module, so that the convolution channels of the learning modules in the generator are gradually increased to progressively map the input infrared image from the low-resolution image to the high-resolution image layer by layer to generate a visible light image.

6. The method according to any one of claims 1 to 5, characterized in that The AI ​​analysis and detection after fusing the new visible light video image with the current video image includes: Based on the pixel brightness of the new visible light video image, calculate the fusion weight of each pixel: Where I(x) is the brightness of the x-th pixel of the new visible light video image, T(x) is the pixel value of the x-th pixel of the new visible light video image, is the pixel value of the xth pixel of the current video image, is the convolution operation, K is the convolution kernel, ρ is the control parameter, k is the brightness threshold, α is the adjustment system, and its value is less than e; The pixel value at each position of the new visible light video image is multiplied by the corresponding fusion weight, and then added to the corresponding pixel of the current video image to obtain a fused video image, so as to perform AI analysis and detection based on the fused video image.

7. An AI analysis and detection device for a power distribution room in a coal mine, used to implement the steps of any of the methods described in claims 1 to 6, characterized in that: The device comprises: A judgment unit, used to judge whether the current video image of the AI ​​camera in the power distribution room of the coal mine needs to be enhanced; The analysis unit, if yes, is used to transform the angle of the infrared image taken by the mine thermal imaging camera at the same position as the current video image and input it into the pre-trained growing adversarial network model, so as to generate a new visible light video image by using the mapping relationship between the thermal radiation information of the infrared image and the texture information of the visible light image learned in advance, so as to fuse the new visible light video image with the current video image and perform AI analysis and detection; if no, perform AI analysis and detection directly based on the current video image; The early warning unit is used to provide early warning inside and outside the mine for each corresponding detection event in the power distribution room of the coal mine based on the AI ​​analysis detection results of the AI ​​camera and the temperature monitoring of the mine thermal imaging camera.

8. A computer device, characterized in that: The computer device includes a memory and a processor, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the steps of any one of the methods described in claims 1-6.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The method comprises a computer program, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images