Coal mine robot inspection image acquisition and analysis system based on machine vision

Abstract

The present application relates to the technical field of image processing, in particular to a coal mine robot inspection image acquisition and analysis system based on machine vision; different image blocks of the inspection image are obtained according to the brightness difference characteristics of the pixel points in the inspection image; the weight of the neighborhood pixel points is obtained according to the position distribution characteristics of the target pixel points and the neighborhood pixel points in the image block; the comprehensive feature descriptor of the target pixel points is obtained according to the gradient difference characteristics of the target pixel points and the neighborhood pixel points, the weight, and the discrete characteristics of the neighborhood pixel points; the image enhancement method and the suspected edge pixel points are judged according to the comprehensive feature descriptor and the comprehensive feature descriptor of the preset sample image; and the gradient feature evaluation value is obtained according to the gradient characteristics and the comprehensive feature descriptor of the suspected edge pixel points. According to the brightness characteristics of the image block and the gradient feature evaluation value, the present application carries out image enhancement and iteration to obtain the enhanced inspection image, thereby improving the image enhancement effect and the image analysis accuracy.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a machine vision-based image acquisition and analysis system for coal mine robot inspections. Background Technology

[0002] Coal mine production environments are characterized by complex spatial structures and high safety risks. To reduce the intensity of manual inspections and improve inspection efficiency and safety, coal mine inspection robots are gradually being applied to underground operations. These robots, equipped with visual sensors, automatically inspect and acquire images of the roadway environment, equipment operating status, and potential safety hazards. Existing inspection systems mostly use visible light cameras to acquire images and combine them with deep learning algorithms to achieve target recognition and status analysis. However, due to factors such as uneven distribution of light sources and limited lighting conditions underground, the acquired images are prone to problems such as insufficient brightness, local overexposure, or obvious shadows. This can easily lead to missed or false detections during the anomaly identification and detection process. Therefore, it is necessary to process the images acquired in inspection scenarios to improve image quality.

[0003] In the processing of inspection images, existing methods often employ fixed image processing workflows, uniformly performing enhancement and denoising operations on the acquired images. However, in actual underground inspection scenarios, image quality can vary significantly at different times and locations, and not all inspection images require enhancement. Furthermore, the brightness and sharpness distribution of the same image can be uneven across different areas. Simultaneously, the inspection robot is affected by changes in lighting, posture, and environmental interference during movement, causing image features to dynamically change. Using a uniform, fixed processing workflow and intensity can easily lead to over-enhancement of high-quality images, introducing additional noise or overexposure, while under-enhancement of low-quality images may result in inaccurate feature recognition. Therefore, fixed processing methods are insufficient to improve the enhancement effect on different inspection images, thus affecting the accuracy of image analysis and recognition. Summary of the Invention

[0004] To address the aforementioned technical problems, the present invention aims to provide a machine vision-based coal mine robot inspection image acquisition and analysis system, the specific technical solution of which is as follows:

[0005] The image acquisition module is used to acquire inspection images of the coal mine robot.

[0006] The image analysis module is used to divide the inspection image into blocks based on the brightness difference characteristics of pixels in the inspection image to obtain different image blocks; to obtain the weights of neighboring pixels based on the positional distribution characteristics of the target pixel and its neighboring pixels in the image blocks; to obtain a first feature descriptor based on the gradient difference characteristics of the target pixel and its neighboring pixels and the weights; to obtain a second feature descriptor based on the discrete characteristics of the neighboring pixels; and to obtain a comprehensive feature descriptor for the target pixel based on the first feature descriptor and the second feature descriptor.

[0007] The enhancement judgment module is used to initially determine the image enhancement method and obtain suspected edge pixels based on the comprehensive feature descriptor of the target pixels in the image block and the comprehensive feature descriptor of the preset sample image; obtain the gradient feature evaluation value based on the gradient features and comprehensive feature descriptor of the suspected edge pixels; and perform image enhancement based on the brightness features of the image block, the gradient feature evaluation value, and the gradient features and brightness features of the preset sample image.

[0008] The enhancement iteration module is used to judge the enhancement effect based on the gradient features and comprehensive feature descriptors of the enhanced image patch and perform image enhancement iteration to obtain the final enhanced image patch; and to obtain the enhanced inspection image based on all the final enhanced image patches.

[0009] Furthermore, the step of dividing the inspection image into blocks based on the brightness difference features of pixels in the inspection image to obtain different image blocks includes:

[0010] In the formula, K represents the adjustment coefficient. Let Y represent an exponential function with the natural constant as the base, and let Y represent the coefficient of variation of the brightness values ​​of all pixels in an image in the Lab color model. This represents the average brightness of all pixels. This represents the maximum brightness of a pixel. This represents the minimum brightness value of a pixel;

[0011] In the formula, C represents the adaptive block parameter. This indicates rounding down. This indicates the preset minimum block parameter. This represents the preset maximum block size parameter, and K represents the adjustment coefficient. Indicates the adjustment baseline; the inspection image is divided into an average of [number] parts. Image blocks.

[0012] Further, the step of obtaining the weights of neighboring pixels based on the positional distribution features of the target pixel and its neighboring pixels in the image block includes:

[0013] Pixels in the image block whose gradient magnitude is not a constant 0 are taken as target pixels, where... This represents the weight of the nth neighboring pixel of the target pixel. Indicates linear normalization, This represents the Euclidean distance between the target pixel and its nth neighboring pixel. A line is then drawn connecting the nth neighboring pixel and the target pixel. The sine of the angle between the line connecting the nth neighboring pixel and the gradient direction of the target pixel is represented by N, where N represents the number of neighboring pixels.

[0014] Further, the step of obtaining the first feature descriptor based on the gradient difference features between the target pixel and its neighboring pixels and the weights includes:

[0015] In the formula The first feature descriptor of the target pixel is represented by N, where N represents the number of neighboring pixels. Let F represent an exponential function with the natural constant as the base, and let F represent the gradient magnitude of the target pixel. This represents the gradient magnitude of the nth neighboring pixel. This represents the weight of the nth neighboring pixel.

[0016] Further, the step of obtaining the second feature descriptor based on the discrete features of the neighboring pixels includes:

[0017] In the formula The second feature descriptor represents the target pixel. Based on the first feature descriptors of neighboring pixels, a predetermined number of neighboring pixels are selected as neighborhood keypoints in descending order. The Euclidean distance between each neighborhood keypoint and its nearest neighbor keypoint is calculated to obtain the nearest neighbor distance. This represents the average nearest neighbor distance of all neighboring keypoints of the target pixel. This represents the maximum value of the nearest neighbor distance. This represents the minimum nearest neighbor distance.

[0018] Further, the step of obtaining the comprehensive feature descriptor of the target pixel based on the first feature descriptor and the second feature descriptor includes:

[0019] Calculate the product of the first feature descriptor and the second feature descriptor to obtain the comprehensive feature descriptor of the target pixel.

[0020] Furthermore, the steps of initially determining the image enhancement method and obtaining suspected edge pixels based on the comprehensive feature descriptor of the target pixels in the image block and the comprehensive feature descriptor of the preset sample image include:

[0021] When the gradient magnitude of all pixels in the image block is a constant of 0, the image block is not enhanced.

[0022] When the gradient magnitude of the pixels in the image block is not all constant 0, the average value of the comprehensive feature descriptors of the edge pixels in the preset sample image is calculated to obtain the judgment threshold; if the comprehensive feature descriptors of the target pixels in the image block are all less than the judgment threshold, the image block is subjected to Gaussian filtering; otherwise, the image block is subjected to adaptive image enhancement, and the target pixels whose comprehensive feature descriptors are not less than the judgment threshold are regarded as the suspected edge pixels.

[0023] Further, the step of obtaining the gradient feature evaluation value based on the gradient features and comprehensive feature descriptor of the suspected edge pixels includes:

[0024] In the formula, R represents the gradient feature evaluation value of the image patch, and M represents the number of suspected edge pixels. This represents the comprehensive feature descriptor of the m-th suspected edge pixel. This represents the gradient magnitude of the m-th suspected edge pixel.

[0025] Further, the step of image enhancement based on the brightness features of the image patch and the gradient feature evaluation value, and the gradient features and brightness features of the preset sample image includes:

[0026] The average gradient magnitude of edge pixels in a preset sample image is calculated to obtain a gradient threshold. When the gradient feature evaluation value is not lower than the gradient threshold, the image block is not enhanced. The average brightness value of all pixels in the preset sample image is calculated to obtain a brightness threshold. When the gradient feature value is lower than the gradient threshold, if the average brightness value of the pixels in the image block is less than or equal to the brightness threshold, the brightness of the image block is increased; otherwise, the brightness of the image block is decreased. The contrast parameter is increased by default. Based on the brightness adjustment direction, the brightness parameter and contrast parameter in the linear brightness and contrast enhancement model are selected using a particle swarm optimization algorithm. The image block is then enhanced based on the results of the brightness parameter and contrast parameter.

[0027] Furthermore, the step of judging the enhancement effect based on the gradient features and comprehensive feature descriptors of the enhanced image patch and performing image enhancement iterations to obtain the final enhanced image patch includes:

[0028] In the formula, W represents the enhanced quality assessment value. G represents the gradient feature evaluation value after image patch enhancement, where G represents the number of pixels whose gradient magnitude is not constant zero and do not belong to suspected edge pixels after image patch enhancement. This represents the gradient magnitude at the g-th pixel. This represents the comprehensive feature descriptor of the g-th pixel.

[0029] After the image block completes one image enhancement, if the enhanced gradient feature evaluation value is not lower than the gradient threshold, the image block stops image enhancement and obtains the final enhanced image block; if the enhancement quality evaluation value is less than the enhancement quality evaluation value after the previous round of enhancement, the image block stops enhancement and uses the enhancement result of the previous round as the final enhanced image block.

[0030] The present invention has the following beneficial effects:

[0031] In this invention, acquiring different image blocks enables the refinement of the inspected image, allowing for the implementation of different enhancement strategies for regions with varying display quality, thus initially improving image enhancement quality. Acquiring the weights of neighboring pixels improves the accuracy of analyzing edge texture features in image blocks; acquiring the first and second feature descriptors allows for the identification of potential edge texture structures within image blocks based on the inherent features of texture edge structures in the image, further improving enhancement accuracy; acquiring the comprehensive feature descriptor characterizes the edge texture structure features in image blocks. Preliminary judgment of image enhancement methods and acquisition of suspected edge pixels based on the comprehensive feature descriptors of target pixels in image blocks and preset sample images allows for the initial implementation of different enhancement methods for regions with different display effects, thereby improving the final display quality. Acquiring gradient feature evaluation values ​​allows for the assessment of the display effect of texture edge structures in image blocks, further determining the enhancement method; image enhancement based on the brightness and gradient feature evaluation values ​​of image blocks, and the gradient and brightness features of preset sample images, accurately implements different enhancement strategies for different image blocks, improving image enhancement accuracy. The enhancement effect is judged based on the gradient features and comprehensive feature descriptors of the enhanced image patches, and image enhancement iteration is performed to obtain the final enhanced image patches. This allows for the assessment of enhancement effects and avoidance of over-enhancement, further improving enhancement accuracy. Enhanced inspection images are obtained from all the final enhanced image patches, improving both image enhancement accuracy and image analysis accuracy. Attached Figure Description

[0032] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a block diagram of a machine vision-based coal mine robot inspection image acquisition and analysis system provided in one embodiment of the present invention. Detailed Implementation

[0034] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a machine vision-based coal mine robot inspection image acquisition and analysis system proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0036] The following description, in conjunction with the accompanying drawings, details a specific scheme for a machine vision-based coal mine robot inspection image acquisition and analysis system provided by the present invention.

[0037] Please see Figure 1 The diagram illustrates a block diagram of a machine vision-based coal mine robot inspection image acquisition and analysis system according to an embodiment of the present invention. The system includes the following modules:

[0038] The image acquisition module S1 is used to acquire inspection images of the coal mine robot.

[0039] In this embodiment of the invention, the implementation scenario is to adaptively enhance the inspection images captured by the coal mine robot, thereby improving the image enhancement quality and the accuracy of image analysis; firstly, the inspection images captured by the coal mine robot during underground inspection are acquired.

[0040] The image analysis module S2 is used to divide the inspection image into blocks based on the brightness difference features of the pixels in the inspection image to obtain different image blocks; to obtain the weights of the neighboring pixels based on the positional distribution features of the target pixel and the neighboring pixels in the image block; to obtain a first feature descriptor based on the gradient difference features and weights of the target pixel and the neighboring pixels; to obtain a second feature descriptor based on the discrete features of the neighboring pixels; and to obtain a comprehensive feature descriptor of the target pixel based on the first feature descriptor and the second feature descriptor.

[0041] During underground inspections, coal mine inspection robots are susceptible to environmental factors such as insufficient and uneven lighting, high dust concentration, and complex terrain. This leads to image quality issues like insufficient brightness and localized overexposure, making it difficult to guarantee image stability. Furthermore, the movement of the inspection robot and the dynamic changes of personnel and equipment underground cause continuous variations in lighting intensity and uniformity, resulting in significant fluctuations in image quality at different times and locations. Therefore, using a uniform, fixed-intensity image processing strategy for all inspection images can easily lead to over-enhancement or noise amplification of high-quality images, while under-enhancement may affect the effective representation of target contours and key features, thus reducing the accuracy of anomaly detection and analysis. Moreover, different regions within the same image frame are affected by lighting to varying degrees, resulting in different enhancement intensity requirements. Therefore, it is necessary to introduce an image quality assessment mechanism during the inspection image enhancement process to adaptively adjust the enhancement strategy based on the image state, thereby improving the enhancement effect and the accuracy of subsequent image analysis.

[0042] First, the inspection image is converted to the Lab color model space for analysis. The L component represents pixel brightness information, and the a and b components are used to represent color distribution characteristics. By analyzing pixel brightness within this color model, the interference of color information on illumination evaluation can be effectively reduced, thus facilitating accurate evaluation of inspection images acquired in complex environments. Because uneven illumination may occur in different areas of the same inspection image, leading to image display quality degradation, the image can be divided into blocks for more accurate enhancement. Different enhancement strategies are applied to different image blocks. Therefore, the inspection image is divided into blocks based on the brightness difference characteristics of pixels in the inspection image to obtain different image blocks. Preferably, in this embodiment of the invention, the step of obtaining different image blocks includes:

[0043]

[0044] In the formula, K represents the adjustment coefficient. Y represents the coefficient of variation of the brightness values ​​of all pixels in the image in the Lab color model. The smaller the coefficient of variation, the better the illumination uniformity of the image, which can reduce the number of image blocks. This represents the average brightness of all pixels. This represents the maximum brightness of a pixel. This represents the minimum brightness value of a pixel; the smaller the brightness difference in the image, the lower the value. A smaller adjustment factor means a smaller range of differences in illumination distribution, thus reducing the number of image blocks. Therefore, a larger adjustment factor means a more uneven illumination distribution and a larger range of illumination differences in the image. To improve the enhancement effect, different regions in different parts of the image should be enhanced to varying degrees, resulting in a larger number of image blocks. Conversely, a smaller adjustment factor means a more uniform illumination and smaller illumination differences throughout the image, allowing for a similarity enhancement strategy to be applied to the entire image, resulting in fewer image blocks. This can then be used to calculate... In the formula, C represents the adaptive block parameter. This indicates rounding down. This indicates the preset minimum block parameter. This represents the preset maximum block size parameter, and K represents the adjustment coefficient. This indicates the adjustment baseline; the inspection image is divided into equal parts. The image blocks are divided into several parts. In this embodiment of the invention, the preset minimum block parameter is 2 and the preset maximum block parameter is 15, which can be determined by the implementer according to the implementation scenario. The larger the adjustment coefficient, the more image blocks there are, and vice versa. The division method is to divide the inspection image into C equal parts horizontally and vertically, thereby obtaining the following: Each image patch can be analyzed independently to improve accuracy.

[0045] Furthermore, the reasons for weak texture display in image blocks may vary. For example, insufficient lighting or overexposure may cause abnormal brightness, or the target surface may have a single color and weak texture, resulting in unclear details. The purpose of image enhancement processing is to ensure the clear presentation of texture and edge structure in the image and improve the accuracy of subsequent analysis. Therefore, the enhancement strategies for unclear textures caused by different reasons will also differ, requiring further analysis of different image blocks to facilitate the implementation of adaptive processing strategies. Since one of the important evaluation indicators of image display quality is the clarity of texture and edge structure, this embodiment of the invention further performs grayscale processing on the image blocks and calculates the gradient magnitude and gradient direction of pixels in the grayscale image using the existing Sobel operator, thereby obtaining pixels in the image block that may contain texture or edge structure. Since the gradient magnitude of edge pixels is usually not a constant zero, and edge pixels have more prominent gradient changes compared to pixels on both sides, and the edge structure has a certain degree of spatial continuity, the weights of neighboring pixels can be obtained based on the positional distribution characteristics of the target pixel and neighboring pixels in the image block. Preferably, in this embodiment of the invention, the step of obtaining the weights includes:

[0046]

[0047] Pixels in the image patch whose gradient magnitude is not a constant 0 are taken as target pixels; where, This represents the weight of the nth neighboring pixel of the target pixel. Indicates linear normalization, This represents the Euclidean distance between the target pixel and its nth neighboring pixel. The closer the distance, the higher the information contribution of that neighboring pixel, and the greater its weight. A line is drawn between the nth neighboring pixel and the target pixel. The sine value represents the angle between the line connecting the nth neighboring pixel and the gradient direction of the target pixel. Since the gradient direction of the pixel is perpendicular to the edge direction, the smaller the sine value, the farther the line is from the edge direction of the target pixel, and the less likely the neighboring pixel is to be affected by its edge during analysis, thus giving it a larger weight. N represents the number of neighboring pixels. In this embodiment, the neighboring pixels are within a range of 7 pixels on each side centered on the target pixel. The implementer can determine this range according to the implementation scenario. The larger the weight, the greater the information contribution of the neighboring pixel. Therefore, the first feature descriptor can be obtained based on the gradient difference characteristics and weights between the target pixel and its neighboring pixels. Preferably, in this embodiment, the step of obtaining the first feature descriptor includes:

[0048]

[0049] In the formula, The first feature descriptor of the target pixel is represented by N, where N represents the number of neighboring pixels. Let F represent an exponential function with the natural constant as the base, and let F represent the gradient magnitude of the target pixel. This represents the gradient magnitude of the nth neighboring pixel. This represents the weight of the nth neighboring pixel. The greater the gradient difference between the target pixel and its neighboring pixels, and the greater the weight of the neighboring pixel, the larger the first feature descriptor. This indicates a significant gradient change between the target pixel and its neighboring pixels, making the target pixel more likely to be an edge pixel. Since edge pixels have spatial continuity, a second feature descriptor can be obtained based on the discrete features of the neighboring pixels. Preferably, in this embodiment, the step of obtaining the second feature descriptor includes:

[0050]

[0051] In the formula, The second feature descriptor represents the target pixel. Based on the first feature descriptors of neighboring pixels, a predetermined number of neighboring pixels are selected as key points. In this embodiment, the predetermined number is 8. Due to the continuity of edges, this solution assumes that there are at least 8 pixels on the edge line. The implementer can determine this number according to the implementation scenario. Key points in the neighborhood mean that there is a high probability that they are other edge pixels within the neighborhood of the target pixel, thus allowing analysis based on the continuous features of edge pixels. The Euclidean distance between the key point and its nearest neighbor is calculated to obtain the nearest neighbor distance. If the key point is a true edge point, the nearest neighbor distance is small and they are relatively similar due to their connected positions. If it is not a true edge point, the key points are more likely to be more dispersed. This represents the average nearest neighbor distance of all neighboring keypoints of the target pixel. This represents the maximum value of the nearest neighbor distance. This represents the minimum nearest neighbor distance. Therefore, the smaller the average nearest neighbor distance and the smaller the difference, the larger the second feature descriptor of the target pixel.

[0052] Further, after obtaining the first feature descriptor and the second feature descriptor, a comprehensive feature descriptor for the target pixel can be obtained based on the first feature descriptor and the second feature descriptor. Preferably, in this embodiment of the invention, the step of obtaining the comprehensive feature descriptor includes: calculating the product of the first feature descriptor and the second feature descriptor to obtain the comprehensive feature descriptor for the target pixel. When both the first feature descriptor and the second feature descriptor of the target pixel are larger, the target pixel is more likely to be an edge pixel representing a texture edge structure, and the comprehensive feature descriptor is larger. The comprehensive feature descriptor can accurately represent the texture edge structure that may exist in the image block.

[0053] The enhancement judgment module S3 is used to initially judge the image enhancement method and obtain suspected edge pixels based on the comprehensive feature descriptor of the target pixels in the image block and the comprehensive feature descriptor of the preset sample image; obtain the gradient feature evaluation value based on the gradient features and comprehensive feature descriptor of the suspected edge pixels; and perform image enhancement based on the brightness features and gradient feature evaluation value of the image block, and the gradient features and brightness features of the preset sample image.

[0054] After obtaining the comprehensive feature descriptors of different target pixels in an image block, since edge pixels have relatively stable comprehensive descriptive characteristics in different images, several structurally clear sample images can be selected as prior samples to assist in analyzing the enhancement strategy of the current image block. Therefore, the image enhancement method and suspected edge pixels are initially judged based on the comprehensive feature descriptors of the target pixels in the image block and the comprehensive feature descriptors of the preset sample images. Preferably, in this embodiment of the invention, the steps of initially judging the image enhancement method and obtaining suspected edge pixels include: when the gradient magnitude of the pixels in the image block is all constant 0, that is, there are no target pixels, it means that there are no obvious edges or texture structures in the image block; such areas usually correspond to the transition areas of the object surface or flat background areas. If enhancement processing is performed, it will not only be difficult to improve the effective information, but may also introduce noise interference; therefore, image enhancement is not performed on such image blocks. When the gradient magnitude of the pixels in the image block is not all constant 0, the average value of the comprehensive feature descriptors of the edge pixels in the preset sample images is calculated to obtain the judgment threshold; the preset sample images refer to several structurally clear inspection images, which can be determined by the implementer; the judgment threshold can be used as a reference value for edge feature description. If the comprehensive feature descriptors of all target pixels in an image patch are less than the judgment threshold, it means that the image patch does not contain effective edge or texture information. Therefore, Gaussian filtering is applied to the image patch to improve the continuity and stability of the region display. Conversely, it means that there are potential textures or edge structures, and adaptive image enhancement is required for the image patch. Target pixels with comprehensive feature descriptors not less than the judgment threshold are then identified as suspected edge pixels. The edge display quality of the image patch is further analyzed using suspected edge pixels to determine a more suitable enhancement strategy that suppresses noise while preserving texture edge information, thereby improving enhancement quality.

[0055] Furthermore, for image blocks containing suspected edge pixels, gradient feature evaluation values ​​can be obtained based on the gradient features of the suspected edge pixels and the comprehensive feature descriptor; preferably, in this embodiment of the invention, the step of obtaining gradient feature evaluation values ​​includes:

[0056]

[0057] In the formula, R represents the gradient feature evaluation value of the image patch, and M represents the number of suspected edge pixels. This represents the comprehensive feature descriptor of the m-th suspected edge pixel. This represents the gradient magnitude of the m-th suspected edge pixel. The comprehensive feature descriptor weights the suspected edge pixels, making the gradient features of pixels more likely to be edge pixels contribute more significantly. The gradient feature evaluation value is the weighted average of all suspected edge pixels in the image patch. A larger gradient feature evaluation value indicates more pronounced texture edge features in that image patch.

[0058] Image enhancement can then be performed based on the brightness and gradient feature evaluation values ​​of the image block, and the gradient and brightness features of the preset sample image. Preferably, in this embodiment, the average gradient magnitude of the edge pixels of the preset sample image is calculated to obtain a gradient threshold. When the gradient feature evaluation value is not lower than the gradient threshold, it means that the edge texture display effect of the image block is good; therefore, the image block is not enhanced. The average brightness value of all pixels in the preset sample image is calculated to obtain a brightness threshold. When the gradient feature value is lower than the gradient threshold, it is necessary to further determine the cause of the influence. If the average brightness value of the pixels of the image block is less than or equal to the brightness threshold, it means that the texture feature of the area may be weak due to insufficient brightness; therefore, the brightness of the image block is increased. Otherwise, it may be due to excessive brightness causing overexposure of the texture, and the brightness of the image block needs to be reduced. The contrast parameter is increased by default. According to the brightness adjustment direction, the brightness parameter and contrast parameter in the linear brightness-contrast enhancement model are selected by the particle swarm optimization algorithm, and the image block is enhanced according to the results of the brightness parameter and contrast parameter. It should be noted that the particle swarm optimization algorithm is an existing technology. This algorithm improves search efficiency by jointly searching the brightness and contrast parameters in the linear brightness-contrast enhancement model. The particle initialization method is set by the predetermined brightness adjustment direction and contrast change rules. It should also be noted that during parameter iteration, the particle swarm optimization objective function is not directly used as the final criterion. Instead, an enhancement quality assessment value is calculated in real time after each parameter update to characterize the relative relationship between edge gradient enhancement and non-edge gradient suppression effects. Furthermore, the linear brightness-contrast enhancement model is an existing technology, and the specific enhancement steps will not be elaborated further.

[0059] The enhancement iteration module S4 is used to judge the enhancement effect based on the gradient features and comprehensive feature descriptors of the enhanced image patch and to perform image enhancement iteration to obtain the final enhanced image patch; and to obtain the enhanced inspection image based on all the final enhanced image patches.

[0060] After enhancing the image patch, it is necessary to determine the enhancement effect to avoid over- or under-enhancement. Therefore, the enhancement effect is determined based on the gradient features and comprehensive feature descriptors of the enhanced image patch, and image enhancement iteration is performed to obtain the final enhanced image patch. Preferably, in this embodiment of the invention, the step of obtaining the final enhanced image patch includes:

[0061]

[0062] In the formula, W represents the enhanced quality assessment value. G represents the gradient feature evaluation value after image patch enhancement, where G represents the number of pixels whose gradient magnitude is not constant zero and do not belong to suspected edge pixels. These pixels refer to pixels that do not represent edge texture features and whose gradient magnitude is not zero. This represents the gradient magnitude at the g-th pixel. This represents the comprehensive feature descriptor of the g-th pixel. This represents the weighted average of the gradient magnitudes of pixels that do not represent texture edge features in an image patch. The larger the value, the less likely the pixel is to represent an edge texture structure, and the greater the likelihood that it is a normal pixel, thus giving it a larger weight in calculating the gradient magnitude. This value represents the ratio of the gradient magnitude of enhanced edge texture features to that of non-edge texture features. A larger value indicates better display of edge gradient features within the image patch, and thus a better enhancement effect. A larger value indicates a higher gradient feature evaluation value, resulting in a better display effect. Therefore, a higher enhancement quality evaluation value means a better enhancement effect for that image patch in this image enhancement.

[0063] Furthermore, after an image patch completes one image enhancement, if the enhanced gradient feature evaluation value is not lower than the gradient threshold, the image patch stops image enhancement and obtains the final enhanced image patch; this means that the reference standard has been met and no further enhancement is needed. If the enhancement quality evaluation value is lower than the enhancement quality evaluation value after the previous round of enhancement, it means that there is a risk of over-enhancement, and continuing enhancement will affect the display quality. Therefore, the image patch stops enhancement and the result of the previous round of enhancement is used as the final enhanced image patch. Either of the two enhancement iteration stopping rules can be satisfied. Through enhancement iteration, noise amplification and over-enhancement can be effectively suppressed while ensuring the edge structure enhancement effect. Finally, an enhanced inspection image can be obtained by stitching together all the final enhanced image patches. Anomaly detection and image analysis can be performed using the enhanced inspection image, which can improve the accuracy of analysis. The implementer can determine the analysis method of the enhanced inspection image at their own discretion, which is not limited here.

[0064] In summary, this invention provides a machine vision-based coal mine robot inspection image acquisition and analysis system. It obtains different image blocks in the inspection image based on the brightness difference characteristics of pixels. It obtains the weights of neighboring pixels based on the positional distribution characteristics of the target pixel and its neighbors within the image block. It obtains a comprehensive feature descriptor for the target pixel based on the gradient difference characteristics, weights, and discrete characteristics of the neighboring pixels. It determines the image enhancement method and identifies suspected edge pixels based on the comprehensive feature descriptor and the comprehensive feature descriptor of a preset sample image. Finally, it obtains a gradient feature evaluation value based on the gradient characteristics of the suspected edge pixels and the comprehensive feature descriptor. This invention performs image enhancement and iteration based on the brightness characteristics and gradient feature evaluation values ​​of image blocks to obtain enhanced inspection images, improving both the image enhancement effect and the accuracy of image analysis.

[0065] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0066] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A coal mine robot inspection image acquisition and analysis system based on machine vision, characterized in that, The system includes the following modules: The image acquisition module is used to acquire inspection images of the coal mine robot. The image analysis module is used to divide the inspection image into blocks based on the brightness difference characteristics of pixels in the inspection image to obtain different image blocks; to obtain the weights of neighboring pixels based on the positional distribution characteristics of the target pixel and its neighboring pixels in the image blocks; to obtain a first feature descriptor based on the gradient difference characteristics of the target pixel and its neighboring pixels and the weights; to obtain a second feature descriptor based on the discrete characteristics of the neighboring pixels; and to obtain a comprehensive feature descriptor for the target pixel based on the first feature descriptor and the second feature descriptor. The enhancement judgment module is used to initially determine the image enhancement method and obtain suspected edge pixels based on the comprehensive feature descriptor of the target pixels in the image block and the comprehensive feature descriptor of the preset sample image; obtain the gradient feature evaluation value based on the gradient features and comprehensive feature descriptor of the suspected edge pixels; and perform image enhancement based on the brightness features of the image block, the gradient feature evaluation value, and the gradient features and brightness features of the preset sample image. The enhancement iteration module is used to determine the enhancement effect based on the gradient features and comprehensive feature descriptors of the enhanced image patch and to perform image enhancement iteration to obtain the final enhanced image patch. Obtain enhanced inspection images based on all final enhanced image blocks; The step of obtaining the weights of neighboring pixels based on the positional distribution features of the target pixel and its neighboring pixels in the image block includes: a pixel point with a gradient amplitude that is not a constant 0 in the image block as a target pixel point, wherein denotes a weight of an nth neighborhood pixel point of the target pixel point, denotes linear normalization, denotes an Euclidean distance between the target pixel point and the nth neighborhood pixel point, and a line is connected between the nth neighborhood pixel point and the target pixel point, denotes a sine value of an angle between the line corresponding to the nth neighborhood pixel point and a gradient direction of the target pixel point, and N denotes a number of neighborhood pixel points. The step of obtaining the first feature descriptor based on the gradient difference features between the target pixel and its neighboring pixels and the weights includes: , wherein denotes the first feature descriptor of the target pixel point, N denotes the number of the neighborhood pixel points, denotes an exponential function with a natural constant as the base, F denotes the gradient amplitude of the target pixel point, denotes the gradient amplitude of the nth neighborhood pixel point, denotes the weight of the nth neighborhood pixel point; The step of obtaining the second feature descriptor based on the discrete features of the neighboring pixels includes: , wherein denotes a second feature descriptor of the target pixel point, and the first feature descriptors of the neighborhood pixel points are selected in descending order to obtain a preset number of neighborhood key points; a Euclidean distance between the neighborhood key points and other neighborhood key points closest to the neighborhood key points is calculated to obtain a nearest neighbor distance, denotes an average value of the nearest neighbor distances of all the neighborhood key points of the target pixel point, denotes a maximum value of the nearest neighbor distances, denotes a minimum value of the nearest neighbor distances.

2. The coal mine robot inspection image acquisition and analysis system based on machine vision according to claim 1, characterized in that, The step of dividing the inspection image into blocks based on the brightness difference characteristics of pixels in the inspection image to obtain different image blocks includes: wherein K represents an adjustment coefficient, represents an exponential function with a natural constant as base, Y represents a coefficient of variation of the luminance values of all the pixels of the inspection image in the Lab color model, represents the average value of the luminance of all the pixels, represents the maximum value of the luminance of the pixels, represents the minimum value of the luminance of the pixels; , wherein C represents an adaptive block parameter, represents a floor function, represents a preset minimum block parameter, represents a preset maximum block parameter, and K represents an adjustment coefficient, represents an adjustment reference; and dividing the inspection image into image blocks on average.

3. The image acquisition and analysis system for coal mine robot inspection based on machine vision according to claim 1, characterized in that, The step of obtaining the comprehensive feature descriptor of the target pixel based on the first feature descriptor and the second feature descriptor includes: Calculate the product of the first feature descriptor and the second feature descriptor to obtain the comprehensive feature descriptor of the target pixel.

4. The image acquisition and analysis system for coal mine robot inspection based on machine vision according to claim 1, characterized in that, The steps of initially determining the image enhancement method and obtaining suspected edge pixels based on the comprehensive feature descriptors of target pixels in the image block and the comprehensive feature descriptors of preset sample images include: When the gradient magnitude of all pixels in the image block is a constant of 0, the image block is not enhanced. When the gradient magnitude of the pixels in the image block is not all constant 0, the average value of the comprehensive feature descriptors of the edge pixels in the preset sample image is calculated to obtain the judgment threshold; if the comprehensive feature descriptors of the target pixels in the image block are all less than the judgment threshold, the image block is subjected to Gaussian filtering; otherwise, the image block is subjected to adaptive image enhancement, and the target pixels whose comprehensive feature descriptors are not less than the judgment threshold are regarded as the suspected edge pixels.

5. The image acquisition and analysis system for coal mine robot inspection based on machine vision according to claim 1, characterized in that, The step of obtaining the gradient feature evaluation value based on the gradient features and comprehensive feature descriptor of the suspected edge pixels includes: In the formula, R represents the gradient feature evaluation value of the image patch, and M represents the number of suspected edge pixels. This represents the comprehensive feature descriptor of the m-th suspected edge pixel. This represents the gradient magnitude of the m-th suspected edge pixel.

6. The image acquisition and analysis system for coal mine robot inspection based on machine vision according to claim 1, characterized in that, The step of image enhancement based on the brightness features of the image patch and the gradient feature evaluation value, and the gradient features and brightness features of the preset sample image includes: The average gradient magnitude of edge pixels in a preset sample image is calculated to obtain a gradient threshold. When the gradient feature evaluation value is not lower than the gradient threshold, the image block is not enhanced. The average brightness value of all pixels in the preset sample image is calculated to obtain a brightness threshold. When the gradient feature value is lower than the gradient threshold, if the average brightness value of the pixels in the image block is less than or equal to the brightness threshold, the brightness of the image block is increased; otherwise, the brightness of the image block is decreased. The contrast parameter is increased by default. Based on the brightness adjustment direction, the brightness parameter and contrast parameter in the linear brightness and contrast enhancement model are selected using a particle swarm optimization algorithm. The image block is then enhanced based on the results of the brightness parameter and contrast parameter.

7. The image acquisition and analysis system for coal mine robot inspection based on machine vision according to claim 6, characterized in that, The steps of judging the enhancement effect based on the gradient features and comprehensive feature descriptors of the enhanced image patch and performing image enhancement iterations to obtain the final enhanced image patch include: In the formula, W represents the enhanced quality assessment value. G represents the gradient feature evaluation value after image patch enhancement, where G represents the number of pixels whose gradient magnitude is not constant zero and do not belong to suspected edge pixels after image patch enhancement. This represents the gradient magnitude at the g-th pixel. This represents the comprehensive feature descriptor of the g-th pixel. After the image block completes one image enhancement, if the enhanced gradient feature evaluation value is not lower than the gradient threshold, the image block stops image enhancement and obtains the final enhanced image block; if the enhancement quality evaluation value is less than the enhancement quality evaluation value after the previous round of enhancement, the image block stops enhancement and uses the enhancement result of the previous round as the final enhanced image block.

Images