Method for identifying maintenance tools in shelter based on machine vision

Abstract

The invention relates to the technical field of image processing, in particular to a method for identifying a maintenance tool in a shelter based on machine vision. The method comprises the following steps: obtaining an original brightness value of each pixel point, and carrying out normalization processing on the original brightness values to obtain standard brightness of the pixel points; constructing an initial bright region mask and an initial dark region mask based on a preset initial bright region threshold value and an initial dark region threshold value; obtaining an optimal bright region mask and an optimal dark region mask based on an optimal bright region threshold and an optimal dark region threshold obtained after iteration is completed, and carrying out iteration on the initial bright region mask and the initial dark region mask; performing different gamma correction on the standard brightness of the pixel points under the optimal bright area mask and the optimal dark area mask to obtain the optimal brightness of the pixel points; and reconstructing the to-be-processed image based on the optimal brightness of each pixel point to obtain an enhanced image, and performing tool identification based on the enhanced image. The method has the effect of improving the accuracy of tool recognition.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method for identifying maintenance tools in a cabin based on machine vision. Background Art

[0002] A shelter system is a highly modular, rapidly deployable mobile support platform used in high-reliability scenarios such as aerospace, military operations, and field emergency maintenance. Maintenance shelter systems integrate a variety of operating tools and auxiliary equipment (such as wrenches, screwdrivers, and pliers) to perform complex equipment inspection, maintenance, and troubleshooting. Inventory of tools in shelter systems typically relies on manual visual inspection. This method is inefficient, places a heavy workload on staff, and is prone to omissions. With the continuous improvement of efficiency in actual production processes, traditional manual visual inspection methods are no longer able to meet current efficiency requirements.

[0003] To improve maintenance efficiency and accuracy, and ensure standardized and traceable tool use, modern maintenance shelter systems are gradually introducing tool identification and management capabilities, enabling automatic identification, status monitoring, and intelligent scheduling of tools within the shelter. Template matching is a common tool identification method. This method requires the creation of a library of tool image templates. The template image is then compared pixel by pixel within the target image to determine the grayscale distribution similarity between the template and the actual image area to determine whether the tool is a specific one. This method is simple to implement and deploy, making it suitable for use in shelter systems.

[0004] However, tool recognition in the confined and cramped space of a quarantine facility presents numerous challenges. First, maintenance tools are mostly made of metal, which can exhibit significant specular reflections. This can easily lead to overexposure and texture loss in images, especially under bright lighting or direct natural light. This can lead to misjudgments when using traditional template matching methods. Summary of the Invention

[0005] In order to reduce the occurrence of misjudgments in the tool identification process by the template matching algorithm, the present application provides a method for identifying maintenance tools in a cabin based on machine vision.

[0006] This application provides a method for identifying maintenance tools in a shelter based on machine vision, using the following technical solutions: The machine vision-based method for identifying maintenance tools in a shelter includes: obtaining an image to be processed, converting the image to be processed from a first color space to a second color space including a brightness channel, obtaining the original brightness value of each pixel, and normalizing the original brightness value to obtain a standard brightness of the pixel; Based on the preset initial bright area threshold and initial dark area threshold, the initial bright area mask and dark area mask are constructed; the texture richness of the bright area mask is calculated, and the bright area threshold is updated and iterated; the texture richness of the dark area mask is calculated, and the dark area threshold is updated and iterated; based on the optimal bright area threshold and the optimal dark area threshold obtained after the iteration, the optimal bright area mask and the optimal dark area mask are obtained and the initial bright area mask and dark area mask are iterated; different gamma corrections are performed on the standard brightness of the pixel points under the optimal bright area mask and the optimal dark area mask to obtain the optimal brightness of the pixel points; the image to be processed is reconstructed based on the optimal brightness of each pixel point to obtain an enhanced image, and tool recognition is performed based on the enhanced image.

[0007] In this application, a bright area mask and a dark area mask are first constructed based on the initial bright area threshold and dark area threshold, respectively. Texture richness calculation and an iterative optimization mechanism are then introduced to effectively distinguish between true high-exposure / dark corner areas and the normal texture areas of the tool itself. By dynamically updating the bright and dark area thresholds based on texture information, the problem of regional missegmentation caused by fixed thresholds is reduced. Subsequently, different gamma corrections are performed on the pixels under the masks based on the optimal bright area mask and the optimal dark area mask to suppress highlight areas and enhance dark corner areas. The corrected brightness component is fused with the original hue and saturation information to reconstruct the image. The resulting enhanced image maintains color reproduction while having a more balanced brightness distribution and clearer edge and texture features. This provides high-quality input feature maps for subsequent deep learning target detection algorithms (such as YOLO), significantly improving the accuracy and robustness of tool recognition and reducing false detections and missed detections caused by lighting interference such as reflections and dark corners.

[0008] Optionally, an initial bright area threshold and an initial dark area threshold are set based on a global brightness mean and a global brightness standard deviation of the brightness channel.

[0009] By calculating the global mean of the luminance channel, we can determine the overall brightness level of the image. The global standard deviation reflects the dispersion of the luminance distribution, that is, the magnitude of luminance variation. These two statistics are used to set the initial bright and dark region thresholds, allowing the threshold selection to be dynamically adjusted based on the overall brightness of each image.

[0010] Optionally, the step of calculating the texture richness of the bright area mask includes: for any pixel point in the area corresponding to the bright area mask, using the second-order gradient calculation value of the pixel point as the local contrast; constructing a local observation window based on the pixel point, calculating the fusion complexity index of the pixel point based on the information entropy of the grayscale value distribution in the observation window and the local contrast, and using the average of the texture complexity indexes of each pixel point in the bright area as the texture richness.

[0011] By calculating local contrast and grayscale information entropy, the texture richness of the area corresponding to the bright area mask is quantitatively evaluated, which can accurately distinguish the highly reflective area from the normal bright texture area on the tool surface, thereby providing a reliable basis for the iterative optimization of the bright area threshold.

[0012] Optionally, a weighted sum of the information entropy of the grayscale distribution in the observation window and the local contrast is used as the fusion complexity index.

[0013] The weighting coefficient can be adjusted according to different scenarios and lighting conditions. It has strong adaptability and is suitable for application scenarios with changeable environments such as shelters.

[0014] Optionally, the step of constructing the local observation window includes: for any pixel point, constructing a square area with a preset side length with the pixel point as the center, and using the square area as the local observation window.

[0015] Optionally, the step of iteratively updating the bright area threshold includes: in response to the texture richness of the area corresponding to the bright area mask being higher than a preset texture threshold, increasing the bright area threshold according to a preset step size until an iteration stop condition is reached or a preset number of iterations is reached.

[0016] Optionally, the iteration stopping condition is: the difference in texture richness of the area corresponding to the bright area mask before and after the iteration is less than a preset difference threshold.

[0017] The introduced texture richness difference determination mechanism objectively reflects the degree of change in structural information in the bright area mask before and after iteration. When the difference value is less than a preset threshold, it indicates that the regional texture features have basically stabilized and further iterations will not significantly improve the results. At this time, early termination can achieve the optimal balance between computational effort and accuracy.

[0018] Optionally, the step of performing gamma correction on the standard brightness of the pixel points under the optimal bright area mask includes: constructing a suppressed gamma index based on the mean and standard deviation of the original brightness of the pixel points in the area corresponding to the bright area mask; for any pixel point, the result of a power function with the original brightness of the pixel point as the base and the suppressed gamma index as the exponent is used as the corrected brightness, and the corrected brightness is negatively correlated with the suppressed gamma index.

[0019] The brightness of the pixels under the optimal bright area mask is suppressed by the suppressed gamma index to reduce pixel overexposure, highlight the texture of the image, and optimize the pixel brightness.

[0020] Optionally, the first color space is an RGB color space; the second color space is an HSV color space.

[0021] Optionally, the step of reconstructing the image to be processed based on the optimal brightness of each pixel to obtain an enhanced image includes: mapping the corrected brightness of each pixel to the range of [0, 255], merging it with other channel values ​​in the image to be processed and converting it to the RGB color space to obtain an enhanced image.

[0022] This application has the following technical effects: By adjusting the brightness of overexposed areas and dark corners in the image to varying degrees, the texture and details in the image to be processed are effectively preserved, thereby improving the accuracy of subsequent tool recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of a method for identifying maintenance tools in a shelter based on machine vision according to an embodiment of the present application.

[0024] Figure 2 This is a method flow chart of step S2 in the method for identifying maintenance tools in a shelter based on machine vision in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The embodiment of the present application discloses a method for identifying maintenance tools in a cabin based on machine vision. The method converts an image in an RGB color space into an HSV color space containing a brightness channel, constructs an initial bright area mask and a dark area mask based on a preset initial bright area threshold and an initial dark area threshold, and then iteratively updates the initial bright area threshold according to the texture richness of the area corresponding to the bright area mask; iteratively updates the initial dark area threshold according to the texture richness of the area corresponding to the dark area mask; obtains an optimal bright area mask and an optimal dark area mask; performs gamma correction on the pixels of the optimal bright area mask and the optimal dark area mask to obtain an enhanced image, and identifies tools based on the enhanced image. By performing different gamma corrections on different areas, the highly exposed areas and gray and dark areas in the image to be processed are reduced, thereby improving the accuracy of subsequent tool recognition.

[0026] Reference Figure 1 The method for identifying maintenance tools in a shelter based on machine vision includes: steps S1 to S5.

[0027] S1: Acquire an image to be processed, and convert the image to be processed from a first color space to a second color space including a brightness channel, obtain the original brightness value of each pixel, and normalize the original brightness value to obtain the standard brightness of the pixel.

[0028] In this example, an industrial camera deployed within the shelter first captures an image of the repair tool scene to be processed. To ensure image quality, a high-resolution industrial camera is used, fixed to ensure a stable viewing angle. The captured original image is in RGB format.

[0029] Because luminance and chrominance information are coupled in the RGB color space, direct brightness adjustment is not feasible. Therefore, it is necessary to convert the original RGB image to the HSV color space. The HSV space decomposes the image into three independent channel components: hue (H), saturation (S), and value (V). This application primarily processes the luminance component.

[0030] To avoid numerical overflow or uneven enhancement during subsequent calculations, the original brightness of the pixels needs to be normalized. The brightness values ​​are linearly scaled to the range [0, 1] to obtain the standard brightness of the pixels. To facilitate differentiation, the brightness of the pixels before normalization is used as the original brightness in this embodiment.

[0031] S2: constructing an initial bright area mask and a dark area mask based on a preset initial bright area threshold and an initial dark area threshold.

[0032] This step is mainly used to accurately identify and segment the highly reflective areas (bright areas) and dark corner occlusion areas (dark areas) in the image, providing a basis for subsequent targeted brightness adjustments.

[0033] Reference Figure 2 , step S2 includes step S21-step S22.

[0034] S21: setting an initial bright area threshold and an initial dark area threshold based on the global brightness mean and the global brightness standard deviation of the brightness channel.

[0035] Perform statistical analysis on the entire brightness channel and calculate its global mean and standard deviation. Based on these two statistics, set the initial bright and dark thresholds. This is based on the fact that, typically, the brightness values ​​of highly reflective areas are much higher than the mean, while those of dark corners are much higher than the mean.

[0036] Specifically, the calculation formula of the initial bright area threshold can be expressed as: Where, represents the initial bright area threshold; Represents the mean of the original brightness of all pixels in the image to be processed; Indicates the standard deviation of the mean original brightness of all pixels in the image to be processed.

[0037] The calculation formula of the initial dark area threshold can be expressed as: Where, represents the initial bright area threshold; Represents the mean of the original brightness of all pixels in the image to be processed; Indicates the standard deviation of the mean original brightness of all pixels in the image to be processed.

[0038] S22: Constructing an initial bright area mask and an initial dark area mask based on the initial bright area threshold and the initial dark area threshold. Specifically, the mask values ​​of pixels whose original brightness is greater than the initial bright area threshold are set to 1, and the mask values ​​of the remaining pixels are set to 0, thereby obtaining an initial bright area mask. Similarly, the mask values ​​of pixels whose original brightness is less than the initial dark area threshold are set to 1, and the mask values ​​of the remaining pixels are set to 0, thereby obtaining an initial dark area mask.

[0039] S3: Calculate the texture richness of the bright area mask and iterate the bright area threshold; calculate the texture richness of the dark area mask and iterate the dark area threshold; obtain the optimal bright area mask and the optimal dark area mask based on the optimal bright area threshold and the optimal dark area threshold obtained after the iteration, and iterate the initial bright area mask and dark area mask.

[0040] In order to distinguish the real overexposed / underexposed areas from the light and dark textures of the tool itself, this application introduces local texture analysis, that is, calculating the texture richness of the area under the bright area mask.

[0041] The steps of calculating the texture richness of the bright area mask include: for any pixel point in the area corresponding to the bright area mask, the second-order gradient calculation value of the pixel point is used as the local contrast; a local observation window is constructed based on the pixel point, the fusion complexity index of the pixel point is calculated based on the information entropy of the grayscale value distribution in the observation window and the local contrast, and the average of the texture complexity indexes of each pixel point in the bright area is used as the texture richness.

[0042] In this embodiment, a Laplacian operator is used to perform second-order gradient calculation on the original brightness of the image to be processed to obtain the local contrast corresponding to each pixel.

[0043] Then, a square area is constructed with the pixel as the center, and the square area is used as the observation window of the pixel. In this embodiment, the side length of the square area is 5. The distribution probability of the grayscale value in the observation window is statistically analyzed, and the information entropy is calculated to measure the texture complexity of the local area.

[0044] Specifically, the calculation formula of the information entropy of the grayscale distribution in the observation window corresponding to the pixel can be expressed as: Where, Represents the information entropy of the grayscale distribution in the observation window corresponding to the pixel; Indicates that the gray value in the observation window is This formula is a conventional technical means in this field and will not be described in detail here.

[0045] The fusion complexity index of the pixel is calculated based on the information entropy of the gray value distribution in the observation window and the local contrast.

[0046] In this embodiment, the result of weighted summation of information entropy and local contrast is used as the fusion complexity index, where the weights of information entropy and local contrast are both 0.5.

[0047] The mean of the fusion complexity index of each pixel is used as the texture richness corresponding to the bright area mask.

[0048] The texture richness corresponding to the dark area mask is similar to that of the bright area mask, and will not be repeated here.

[0049] After calculating the texture richness, the bright area threshold can be adjusted based on the texture richness, and the adjusted bright area threshold can be used to update and iterate.

[0050] Specifically, the step of iteratively updating the bright area threshold includes: in response to the texture richness of the area corresponding to the bright area mask being higher than the preset texture threshold, increasing the bright area threshold according to a preset step size until an iteration stop condition is reached or a preset number of iterations is reached.

[0051] In this embodiment, the preset texture threshold is 0.3. For a bright area mask, if the texture richness corresponding to the bright area mask is greater than the texture threshold, the area corresponding to the bright area mask is said to contain a high amount of texture. In other words, the area under the bright area mask contains a high number of pixels that are not considered high exposure. To improve the accuracy of subsequent processing of pixel standard brightness, it is necessary to increase the bright area threshold to reduce the area corresponding to the bright area mask.

[0052] In this embodiment, the method for adjusting the bright area threshold is: increasing the bright area threshold according to a preset step size until an iteration stop condition is reached or a preset number of iterations is reached.

[0053] Assuming the step size is set to 5, when the texture richness of the calculated bright area mask exceeds the texture threshold, the sum of the initial bright area threshold and 5 is calculated, and this result is used as the adjusted bright area threshold to iteratively update the bright area threshold. To prevent the bright area threshold from being too large, this embodiment sets a threshold protection boundary. The threshold protection boundary is obtained by obtaining the maximum brightness value in the image to be processed, subtracting a preset parameter from this maximum value, and obtaining the protection boundary. If the calculated bright area threshold is greater than the protection boundary, the value corresponding to the protection boundary is used as the bright area threshold.

[0054] Specifically, the formula for adjusting the bright area threshold can be expressed as: Where, Indicates the bright area threshold after one adjustment. Here it is mainly used to describe the changes before and after the bright area threshold adjustment. Indicates the bright area threshold before adjustment; Indicates preset parameters; Indicates protection boundary; represents the minimum value function; is the preset step size.

[0055] In this embodiment, the iteration stopping condition is: the difference in texture richness of the area corresponding to the bright area mask before and after the iteration is less than a preset difference threshold. In this embodiment, the preset difference threshold is set to 0.01.

[0056] The step of iteratively updating the dark area threshold includes: in response to the texture richness of the area corresponding to the dark area mask being higher than the preset texture threshold, reducing the bright area threshold according to a preset step size until an iteration stop condition is reached or a preset number of iterations is reached.

[0057] This step is similar to the bright area threshold adjustment step above. In an image, both dark corners and overexposed areas will result in texture loss. If a region has excessive texture richness, it indicates that it contains normal areas and should be adjusted. The difference is that for the dark area threshold, its value needs to be reduced.

[0058] When the iterations of the dark area threshold and the bright area threshold are stopped, the optimal bright area mask and the optimal dark area mask are constructed based on the bright area threshold and the dark area threshold.

[0059] S4: performing different gamma corrections on the standard brightness of the pixel points under the optimal bright area mask and the optimal dark area mask to obtain the optimal brightness of the pixel points.

[0060] For any pixel under the optimal bright area mask, suppressive gamma correction is performed on it.

[0061] The correction process can be expressed as the following formula: Where, Represents pixel points Optimal brightness after calibration; Represents pixel points Standard brightness; represents the inhibitory correction index.

[0062] The calculation formula of the inhibition correction index can be expressed as: Where, represents the inhibitory correction index; Represents the average of the original brightness of each pixel under the optimal bright area mask; Indicates the standard deviation of the brightness under the optimal bright area mask; represents the hyperbolic tangent function; represents the first adjustment factor, represents the second regulatory factor; Represents the median of the data set consisting of the original brightness of each pixel in the image to be processed. In the application environment of the method in this application, there will be no light-free situation, so there is no need to consider the situation where the denominator is 0 during the calculation process; of course, a hyperparameter can also be set in the distribution to place the denominator part at 0.

[0063] when The larger the value, the brighter the area is, closer to being overexposed, and requires stronger gamma compression. The bigger, The bigger, The bigger it is, the The larger the value, the stronger the suppression of brightness.

[0064] For any pixel under the optimal dark area mask, enhanced gamma correction is performed on it.

[0065] The correction process can be expressed as the following formula: Where, Represents pixel points Corrected brightness; Represents pixel points Standard brightness; Indicates the enhanced correction index.

[0066] The calculation formula of the enhanced correction index can be expressed as: Where, represents the inhibitory correction index; Represents the median of the data set consisting of the original brightness of each pixel in the image to be processed; Represents the average of the original brightness of each pixel under the optimal dark area mask; represents the standard deviation of brightness under the optimal dark area mask; is the third regulatory factor; is the fourth regulatory factor; is the hyperbolic tangent function.

[0067] When the overall brightness of the dark area The lower it is, the darker the area is and the stronger the enhancement is needed. The lower, The smaller, The bigger, the It decreases accordingly. According to the formula of the calibration process, the standard brightness range is 0-1. When it decreases, the standard brightness of the pixel will increase. near When the dark area is not actually dark, there is no need to enhance it too much, so Approaching 0, Approaches 1 (no enhancement).

[0068] When the standard deviation of the dark area brightness distribution When it is large, it indicates that the brightness of the area varies greatly and may contain some brighter pixels. Therefore, the enhancement amplitude should be reduced to avoid noise amplification or local overexposure. Therefore, add a positively correlated terms, so when When it is larger, will increase relatively, thus weakening the enhancement degree. Normalize the standard deviation to the range [0,1]. When is large, the contribution is 1 (maximum), which appropriately suppresses the enhancement strength.

[0069] S5: reconstructing the image to be processed based on the optimal brightness of each pixel to obtain an enhanced image, and performing tool recognition based on the enhanced image.

[0070] The optimal brightness component after processing Mapped to the range of [0,255], merged with the original hue H and saturation S, converted back to RGB space, and generated an enhanced image for subsequent recognition tasks.

[0071] Subsequent use of deep learning object detection algorithms such as the artificial intelligence (AI) YOLO (You Only Look Once) algorithm will be possible. Due to the significant improvement in input image quality, tool features such as outlines and textures are more prominent, providing higher-quality feature maps for the YOLO algorithm. This significantly improves the accuracy and robustness of tool recognition, laying a solid foundation for subsequent tasks such as automated inventory and intelligent crawling.

[0072] The above are all preferred embodiments of the present application, and are not intended to limit the scope of protection of the present application. Therefore, any equivalent changes made based on the structure, shape, and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A method for identifying maintenance tools in a shelter based on machine vision, characterized in that: include: Acquire an image to be processed, and convert the image to be processed from a first color space to a second color space including a brightness channel, obtain an original brightness value of each pixel, and normalize the original brightness value to obtain a standard brightness of the pixel; Constructing an initial bright area mask and a dark area mask based on a preset initial bright area threshold and an initial dark area threshold; Calculate the texture richness of the bright area mask and iterate the bright area threshold; Calculate the texture richness of the dark area mask and iterate the dark area threshold; Based on the optimal bright area threshold and the optimal dark area threshold obtained after the iteration, the optimal bright area mask and the optimal dark area mask are obtained, and the initial bright area mask and the optimal dark area mask are iterated; different gamma corrections are performed on the standard brightness of the pixels under the optimal bright area mask and the optimal dark area mask to obtain the optimal brightness of the pixels; based on the optimal brightness of each pixel, the image to be processed is reconstructed to obtain an enhanced image, and tool recognition is performed based on the enhanced image.

2. The method for identifying maintenance tools in a shelter based on machine vision according to claim 1, characterized in that: An initial bright area threshold and an initial dark area threshold are set based on a global brightness mean and a global brightness standard deviation of the brightness channel.

3. The method for identifying maintenance tools in a shelter based on machine vision according to claim 1, characterized in that: The steps of calculating the texture richness of the bright area mask include: for any pixel point in the area corresponding to the bright area mask, the second-order gradient calculation value of the pixel point is used as the local contrast; a local observation window is constructed based on the pixel point, the fusion complexity index of the pixel point is calculated based on the information entropy of the grayscale value distribution in the observation window and the local contrast, and the average of the texture complexity indexes of each pixel point in the bright area is used as the texture richness.

4. The method for identifying maintenance tools in a shelter based on machine vision according to claim 3, characterized in that: The result of the weighted summation of the information entropy of the grayscale distribution in the observation window and the local contrast is used as the fusion complexity index.

5. The method for identifying maintenance tools in a shelter based on machine vision according to claim 3, characterized in that: The step of constructing a local observation window includes: for any pixel point, constructing a square area with a preset side length with the pixel point as the center, and using the square area as the local observation window.

6. The method for identifying maintenance tools in a shelter based on machine vision according to claim 1, characterized in that: The step of iteratively updating the bright area threshold includes: in response to the texture richness of the area corresponding to the bright area mask being higher than the preset texture threshold, increasing the bright area threshold according to a preset step size until an iteration stop condition is reached or a preset number of iterations is reached.

7. The machine vision-based method for identifying maintenance tools in a shelter according to claim 6, characterized in that: The iteration stopping condition is: the difference in texture richness of the area corresponding to the bright area mask before and after iteration is less than the preset difference threshold.

8. The method for identifying maintenance tools in a shelter based on machine vision according to claim 1, characterized in that: The step of performing gamma correction on the standard brightness of the pixel points under the optimal bright area mask includes: constructing a suppressed gamma index based on the mean and standard deviation of the original brightness of the pixel points in the area corresponding to the bright area mask; for any pixel point, the result of a power function with the original brightness of the pixel point as the base and the suppressed gamma index as the exponent is used as the corrected brightness, and the corrected brightness is negatively correlated with the suppressed gamma index.

9. The method for identifying maintenance tools in a shelter based on machine vision according to claim 1, characterized in that: The first color space is the RGB color space; the second color space is the HSV color space.

10. The method for identifying maintenance tools in a shelter based on machine vision according to claim 1, characterized in that: The step of reconstructing the image to be processed based on the optimal brightness of each pixel to obtain an enhanced image includes: mapping the corrected brightness of each pixel to the range of [0, 255], merging it with other channel values ​​in the image to be processed and converting it to the RGB color space to obtain an enhanced image.

Images