Shelter maintenance tool recognition method based on machine vision

Abstract

The application relates to the field of image processing technology, in particular to a shelter internal maintenance tool identification method based on machine vision; the method comprises the following steps: acquiring original brightness values of each pixel point and performing normalization processing on the original brightness values to obtain standard brightness of the pixel points; an initial bright area mask and an initial dark area mask are constructed based on preset initial bright area threshold values and initial dark area threshold values; optimal bright area masks and optimal dark area masks are obtained based on optimal bright area threshold values and optimal dark area threshold values obtained after iteration, and the initial bright area mask and the initial dark area mask are iterated; the standard brightness of the pixel points under the optimal bright area mask and the optimal dark area mask is subjected to different gamma correction to obtain optimal brightness of the pixel points; an enhanced image is obtained by reconstructing a to-be-processed image based on the optimal brightness of each pixel point, and tool identification is performed based on the enhanced image. The application has the effect of improving the accuracy of tool identification.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a machine vision-based method for identifying maintenance tools inside modular shelters. Background Technology

[0002] A mobile support system is a highly modular and rapidly deployable platform used in high-reliability scenarios such as aerospace, military operations and maintenance, and field emergency repair. Maintenance-type mobile support systems integrate various types of operating tools and auxiliary equipment (e.g., wrenches, screwdrivers, pliers) for performing inspection, maintenance, and troubleshooting tasks on complex equipment. Tool inventory within these systems typically relies on manual visual inspection. This method is inefficient, labor-intensive, and prone to omissions. With the increasing demands for efficiency in actual production processes, traditional manual visual inspection methods are no longer adequate.

[0003] To improve maintenance efficiency and operational accuracy, and to ensure the standardization and traceability of tool use, modern maintenance modular systems are gradually introducing tool identification and management functions to achieve automatic identification, status monitoring, and intelligent scheduling of tools within the modular system. A commonly used method for tool identification is template matching. This method requires the pre-establishment of a tool image template library. The template is then compared pixel-by-pixel in the target image to determine the similarity of grayscale distribution between the template and the actual image area, thus identifying the specific tool. This method is simple to implement and easy to deploy, making it suitable for use in modular systems.

[0004] However, the task of tool identification faces many challenges in the confined and narrow space of a makeshift hospital. First, most maintenance tools are made of metal, which has a severe specular reflection phenomenon. Especially under bright lighting or direct natural light, problems such as local overexposure and loss of texture can easily occur. This can lead to misjudgments in the process of identifying tools using traditional template matching methods. Summary of the Invention

[0005] To reduce misjudgments during template matching algorithm identification, this application provides a machine vision-based method for identifying maintenance tools inside modular shelters.

[0006] This application provides a machine vision-based method for identifying maintenance tools inside a modular shelter, employing the following technical solution:

[0007] A machine vision-based method for identifying maintenance tools inside a mobile cabin includes: acquiring an image to be processed, converting the image to be processed from a first color space to a second color space containing a luminance channel, acquiring the original luminance value of each pixel, and normalizing the original luminance value to obtain the standard luminance of the pixel.

[0008] Initial bright and dark area masks are constructed based on preset initial bright and dark area thresholds. The texture richness of the bright area mask is calculated, and the bright area threshold is updated iteratively. The texture richness of the dark area mask is calculated, and the dark area threshold is updated iteratively. Based on the optimal bright and dark area thresholds obtained after iteration, the optimal bright and dark area masks are obtained, and the initial bright and dark area masks are iterated. Different gamma corrections are applied to the standard brightness of the pixels under the optimal bright and dark area masks to obtain the optimal brightness of the pixels. The image to be processed is reconstructed based on the optimal brightness of each pixel to obtain the enhanced image, and tool recognition is performed based on the enhanced image.

[0009] In this application, bright and dark area masks are first constructed based on initial bright and dark area thresholds, respectively. A texture richness calculation and iterative optimization mechanism is introduced to effectively distinguish between real high-exposure / vignetting areas and the normal texture areas of the tool itself. By dynamically updating the bright and dark area thresholds based on texture information, the problem of missegmentation caused by fixed thresholds is reduced. Subsequently, different gamma corrections are applied to the pixels under the masks based on the optimal bright and dark area masks to suppress highlight areas and enhance vignette areas. The corrected brightness components are fused with the original hue and saturation information to reconstruct the image, resulting in an enhanced image that maintains color fidelity while possessing a more balanced brightness distribution and clearer edge and texture features. This provides high-quality input feature maps for subsequent deep learning object detection algorithms (such as YOLO), significantly improving the accuracy and robustness of tool recognition and reducing false detections and missed detections caused by lighting interference such as reflections and vignettes.

[0010] Optionally, an initial bright area threshold and an initial dark area threshold are set based on the global brightness mean and global brightness standard deviation of the brightness channel.

[0011] By calculating the global mean of the luminance channel, the overall luminance level of the image can be obtained; while the global standard deviation reflects the dispersion of the luminance distribution, i.e., the amplitude of luminance variation. Initial bright area thresholds and dark area thresholds are set based on these two statistics, allowing the threshold selection to be dynamically adjusted according to the overall luminance of different images.

[0012] Optionally, the step of calculating the texture richness of the bright area mask includes: for any pixel in the corresponding region of the bright area mask, the second-order gradient value of the pixel is calculated as the local contrast; a local observation window is constructed based on the pixel, the fusion complexity index of the pixel is calculated based on the information entropy of the gray value distribution in the observation window and the local contrast, and the average of the texture complexity index of each pixel in the bright area is used as the texture richness.

[0013] By calculating local contrast and grayscale entropy, the texture richness of the corresponding area of ​​the bright area mask is quantitatively evaluated, which can accurately distinguish between highly reflective areas and normal bright texture areas on the tool surface, thus providing a reliable basis for iterative optimization of the bright area threshold.

[0014] Optionally, the result of a weighted sum of the information entropy of the grayscale distribution in the observation window and the local contrast can be used as the fusion complexity index.

[0015] The weighting coefficients can be adjusted according to different scenarios and lighting conditions, making them highly adaptable and suitable for application scenarios with variable environments, such as mobile cabins.

[0016] Optionally, the steps for constructing a local observation window include: for any pixel, constructing a square region with a preset side length centered on that pixel, and using the square region as a local observation window.

[0017] Optionally, the steps for updating and iterating the bright area threshold include: in response to the texture richness of the corresponding region of the bright area mask being higher than a preset texture threshold, increasing the bright area threshold by a preset step size until the iteration stop condition is met or the preset number of iterations is reached.

[0018] Optionally, the iteration stopping condition is: the difference in texture richness between the corresponding regions of the bright area mask before and after the iteration is less than a preset difference threshold.

[0019] The introduced texture richness difference determination mechanism can objectively reflect the degree of change in structural information of the bright area mask before and after iteration. When the difference value is less than the preset threshold, it indicates that the texture features of the region have basically stabilized, and continuing the iteration can no longer significantly improve the results. At this time, early termination can achieve the best balance between computation and accuracy.

[0020] Optionally, the step of performing gamma correction on the standard brightness of pixels under the optimal bright area mask includes: constructing a suppressed gamma index based on the mean and standard deviation of the original brightness of pixels in the corresponding region of the bright area mask; for any pixel, using the original brightness of the pixel as the base and the suppressed gamma index as the exponent as the result of a power function as the corrected brightness, and the corrected brightness is negatively correlated with the suppressed gamma index.

[0021] By suppressing the brightness of pixels under the optimal bright area mask through the suppression gamma index, the overexposure of pixels is reduced, the texture of the image is highlighted, and the brightness of pixels is optimized.

[0022] Optionally, the first color space is the RGB color space; the second color space is the HSV color space.

[0023] Optionally, the step of reconstructing the image to be processed based on the optimal brightness of each pixel to obtain the enhanced image includes: mapping the corrected brightness of each pixel to the range of [0,255], merging it with other channel values ​​in the image to be processed, and converting it to the RGB color space to obtain the enhanced image.

[0024] This application has the following technical advantages:

[0025] By adjusting the brightness of overexposed and vignetted areas in the image to varying degrees, the texture and details in the image to be processed are effectively preserved, thereby improving the accuracy of subsequent recognition by the tool. Attached Figure Description

[0026] Figure 1 This is a flowchart of the machine vision-based method for identifying maintenance tools inside a modular shelter, according to an embodiment of this application.

[0027] Figure 2 This is a flowchart of step S2 in the machine vision-based method for identifying maintenance tools inside a modular shelter, as described in this application embodiment. Detailed Implementation

[0028] This application discloses a machine vision-based method for identifying maintenance tools inside a mobile shelter. The method converts an image in RGB color space to an HSV color space that includes a luminance channel. Initial bright and dark area masks are constructed based on preset initial bright and dark area thresholds. The initial bright area thresholds are then iteratively updated based on the texture richness of the corresponding regions of the bright area masks; similarly, the initial dark area thresholds are iteratively updated based on the texture richness of the corresponding regions of the dark area masks. Optimal bright and dark area masks are then obtained. Gamma correction is applied to the pixels using the optimal bright and dark area masks to obtain an enhanced image. The tool is then identified based on this enhanced image. By applying different gamma corrections to different regions, highly exposed and dark areas in the image are reduced, thereby improving the accuracy of subsequent tool identification.

[0029] Reference Figure 1 The machine vision-based method for identifying maintenance tools inside a mobile cabin includes steps S1-S5.

[0030] S1: Obtain the image to be processed, and convert the image to be processed from the first color space to the second color space containing the luminance channel. Obtain the original luminance value of each pixel and normalize the original luminance value to obtain the standard luminance of the pixel.

[0031] In this embodiment, firstly, images of the maintenance tool scene are acquired using an industrial camera deployed inside the shelter. To ensure image quality, a high-resolution industrial camera is used, and it is fixed in place to ensure a stable shooting angle. The acquired raw images are in RGB format.

[0032] Because luminance and chromaticity information are coupled in the RGB color space, direct luminance adjustment is not feasible. Therefore, it is necessary to convert the original RGB image to the HSV color space. The HSV space decomposes the image into three independent channel components: hue (H), saturation (S), and lightness (V). This application mainly focuses on processing the lightness component.

[0033] To avoid numerical overflow or uneven enhancement during subsequent calculations, the original brightness of the pixels needs to be normalized. The brightness value is linearly scaled to the [0,1] range to obtain the standard brightness of the pixel. Meanwhile, for ease of distinction, the brightness of the pixel before normalization is used as the original brightness in this embodiment.

[0034] S2: Construct initial bright area masks and dark area masks based on preset initial bright area thresholds and initial dark area thresholds.

[0035] This step is mainly used to accurately identify and segment highly reflective areas (bright areas) and vignetting areas (dark areas) in the image, providing a basis for subsequent targeted brightness adjustments.

[0036] Reference Figure 2 Step S2 includes steps S21-S22.

[0037] S21: Based on the global brightness mean and global brightness standard deviation of the brightness channel, set the initial bright area threshold and the initial dark area threshold.

[0038] Statistical analysis was performed on the entire luminance channel to calculate its global mean and standard deviation. Based on these two statistics, initial bright area thresholds and initial dark area thresholds were set. This was based on the fact that, typically, the luminance values ​​of highly reflective areas are much higher than the mean, while those of dark corner areas are also much higher than the mean.

[0039] Specifically, the formula for calculating the initial bright area threshold can be expressed as: In the formula, Indicates the initial bright area threshold; This represents the average original brightness of all pixels in the image to be processed; This represents the standard deviation of the mean of the original brightness of all pixels in the image to be processed.

[0040] The formula for calculating the initial dark area threshold can be expressed as: In the formula, Indicates the initial bright area threshold; This represents the average original brightness of all pixels in the image to be processed; This represents the standard deviation of the mean of the original brightness of all pixels in the image to be processed.

[0041] S22: Construct initial bright area masks and initial dark area masks based on initial bright area thresholds and initial dark area thresholds. Specifically, set the mask value of pixels whose original brightness is greater than the initial bright area threshold to 1, and set the mask value of the remaining pixels to 0, thus obtaining the initial bright area mask. Similarly, set the mask value of pixels whose original brightness is less than the initial dark area threshold to 1, and set the mask value of the remaining pixels to 0, thus obtaining the initial dark area mask.

[0042] S3: Calculate the texture richness of the bright area mask and update the bright area threshold iteratively; calculate the texture richness of the dark area mask and update the dark area threshold iteratively; based on the optimal bright area threshold and optimal dark area threshold obtained after the iteration, obtain the optimal bright area mask and optimal dark area mask and iterate on the initial bright area mask and dark area mask.

[0043] To distinguish between real overexposed / underexposed areas and the light and dark textures inherent in the tool itself, this application introduces local texture analysis, which calculates the texture richness of the area under the bright area mask.

[0044] The steps for calculating the texture richness of a bright area mask include: for any pixel in the corresponding region of the bright area mask, the second-order gradient value of the pixel is calculated as the local contrast; a local observation window is constructed based on the pixel, and the fusion complexity index of the pixel is calculated based on the information entropy of the gray value distribution in the observation window and the local contrast; the mean of the texture complexity index of each pixel in the bright area is used as the texture richness.

[0045] In this embodiment, the Laplacian operator is used to calculate the second-order gradient of the original brightness of the image to be processed, so as to obtain the local contrast of each pixel.

[0046] Subsequently, a square region is constructed centered on the pixel, serving as the observation window for that pixel. In this embodiment, the side length of the square region is 5. The distribution probability of grayscale values ​​within the observation window is statistically analyzed, and the information entropy is calculated to measure the texture complexity of the local region.

[0047] Specifically, the formula for calculating the information entropy of the grayscale distribution in the observation window corresponding to a pixel can be expressed as: In the formula, The information entropy represents the grayscale distribution within the observation window corresponding to a pixel. The grayscale value in the observation window indicates The probability of [the outcome]. This formula is a standard technique in this field and will not be described in detail here.

[0048] The fusion complexity index of pixels is calculated based on the information entropy of the gray value distribution in the observation window and the local contrast.

[0049] In this embodiment, the result of the weighted sum of information entropy and local contrast is used as the fusion complexity index, where the weights of information entropy and local contrast are both 0.5.

[0050] The average value of the fusion complexity index of each pixel is used as the texture richness corresponding to the bright area mask.

[0051] The texture richness corresponding to the dark area mask is the same as that of the bright area mask, and will not be repeated here.

[0052] After calculating the texture richness, the bright area threshold can be adjusted based on the texture richness, and then the adjusted bright area threshold can be used to update and iterate the threshold.

[0053] Specifically, the steps for updating and iterating the bright area threshold include: in response to the texture richness of the corresponding region of the bright area mask being higher than the preset texture threshold, increasing the bright area threshold by a preset step size until the iteration stop condition is met or the preset number of iterations is reached.

[0054] In this embodiment, the preset texture threshold is 0.3. For a bright area mask, if the texture richness corresponding to the bright area mask is greater than the texture threshold, it means that the area corresponding to the bright area mask contains a lot of texture, that is, the area under the bright area mask contains a lot of pixels that are not high-exposure pixels. In order to improve the accuracy of subsequent processing of the standard brightness of pixels, it is necessary to increase the bright area threshold to reduce the area corresponding to the bright area mask.

[0055] In this embodiment, the method for adjusting the bright area threshold is as follows: increase the bright area threshold by a preset step size until the iteration stop condition is met or the preset number of iterations is reached.

[0056] In this embodiment, assuming a step size of 5, when the calculated texture richness of the bright area mask exceeds the texture threshold, the initial bright area threshold is summed with 5, and this result is used as the adjusted bright area threshold for iterative updates. To prevent the bright area threshold from becoming too large, a threshold protection boundary is set. The threshold protection boundary is obtained by subtracting a preset parameter from the maximum brightness value in the image to be processed. When the calculated bright area threshold exceeds the protection boundary, the value corresponding to the protection boundary is used as the bright area threshold.

[0057] Specifically, the formula for adjusting the bright area threshold can be expressed as:

[0058] In the formula, This indicates the bright area threshold after one adjustment, and is mainly used here to describe the change in the bright area threshold before and after adjustment. This indicates the threshold value of the bright area before adjustment; Indicates preset parameters; Indicates the protection boundary; This represents a function that takes the minimum value. This is the preset step size.

[0059] In this embodiment, the iteration stopping condition is: the difference in texture richness between the bright area mask and the corresponding area before and after the iteration is less than a preset difference threshold. In this embodiment, the preset difference threshold is set to 0.01.

[0060] The steps for updating and iterating the dark area threshold include: in response to the texture richness of the corresponding region of the dark area mask being higher than the preset texture threshold, decreasing the bright area threshold by a preset step size until the iteration stop condition is met or the preset number of iterations is reached.

[0061] This step is similar to the bright area threshold adjustment step described above. In an image, both vignetting and overexposed areas can lead to texture loss. If a certain area has excessively high texture richness, it indicates that the area contains normal areas and should be adjusted. The difference is that the dark area threshold needs to be reduced.

[0062] Once the iterations of both the dark area threshold and the bright area threshold have stopped, construct the optimal bright area mask and the optimal dark area mask based on the bright area threshold and the dark area threshold.

[0063] S4: Perform different gamma corrections on the standard brightness of the pixels under the optimal bright area mask and the optimal dark area mask to obtain the optimal brightness of the pixels.

[0064] For any pixel under the optimal bright area mask, perform suppressed gamma correction.

[0065] The correction process can be expressed by the following formula: In the formula, Represents pixels Optimal brightness after correction; Represents pixels Standard brightness; This represents the suppression correction index.

[0066] The formula for calculating the suppression correction index can be expressed as:

[0067] In the formula, Indicates the suppression correction index; This represents the average original brightness of each pixel under the optimal bright area mask; This represents the standard deviation of the brightness under the optimal bright area mask; Represents the hyperbolic tangent function; Indicates the first regulating factor. Indicates the second regulatory factor; The median represents the original brightness of each pixel in the image to be processed. In the application environment of the method in this application, there will be no situation without light, so there is no need to consider the case where the denominator is 0 during the calculation process; of course, a hyperparameter can also be set in the distribution to place the denominator part as 0.

[0068] when A higher value indicates a brighter area, approaching overexposure, requiring stronger gamma compression. The larger, The larger, The larger it is, the better. The larger the value, the stronger the suppression of brightness.

[0069] For any pixel under the optimal dark area mask, perform enhanced gamma correction.

[0070] The correction process can be expressed by the following formula: In the formula, Represents pixels Corrected brightness; Represents pixels Standard brightness; This indicates the enhanced correction index.

[0071] The formula for calculating the enhancement correction index can be expressed as:

[0072] In the formula, Indicates the suppression correction index; This represents the median of the dataset consisting of the original brightness of each pixel in the image to be processed. This represents the average original brightness of each pixel under the optimal dark area mask; The standard deviation of the brightness under the optimal dark area mask; It is the third regulating factor; It is the fourth regulatory factor; It is the hyperbolic tangent function.

[0073] When the overall brightness of the dark area The lower the value, the darker the area is, requiring stronger enhancement. The lower, The smaller, The larger, therefore Consequently, it decreases. According to the formula for the correction process, the standard brightness value ranges from 0 to 1. Decreasing the brightness will increase the standard brightness of the pixel. When the average brightness of the dark area... near If the dark area is not actually dark, then no excessive enhancement is needed. Approaching 0 It approaches 1 (i.e., it does not increase).

[0074] When the standard deviation of the brightness distribution in the dark area A larger value indicates a significant brightness variation in the area, potentially containing some brighter pixels. Therefore, the enhancement level should be reduced to avoid noise amplification or localized overexposure. Thus, a factor is added to the formula... Positively correlated terms, so when When it is large, It will increase relatively, thus weakening the degree of enhancement. This normalizes the standard deviation to the range [0,1]. When the value is large, the contribution is 1 (maximum), at which point the enhancement intensity will be appropriately suppressed.

[0075] S5: Reconstruct the image to be processed based on the optimal brightness of each pixel to obtain an enhanced image, and perform tool recognition based on the enhanced image.

[0076] The processed optimal luminance component Mapped to the range [0,255], then merged with the original hue H and saturation S, converted back to RGB space, and an enhanced image is generated for subsequent recognition tasks.

[0077] Further, deep learning object detection algorithms such as YOLO (You Only Look Once) can be used. Due to the significant improvement in the quality of the input image, the contours, textures, and other features of the tool are more prominent, providing higher-quality feature maps for the YOLO algorithm. This significantly improves the accuracy and robustness of tool recognition, laying a solid foundation for subsequent tasks such as automatic inventory and intelligent grasping.

[0078] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A machine vision-based method for identifying maintenance tools inside a modular shelter, characterized in that, include: The image to be processed is acquired and converted from the first color space to the second color space containing the luminance channel. The original luminance value of each pixel is acquired and the original luminance value is normalized to obtain the standard luminance of the pixel. Initial bright area masks and dark area masks are constructed based on preset initial bright area thresholds and initial dark area thresholds; Calculate the texture richness of the bright area mask and update the bright area threshold iteratively; Calculate the texture richness of the dark area mask and update the dark area threshold iteratively; Based on the optimal bright area threshold and optimal dark area threshold obtained after iteration, the optimal bright area mask and optimal dark area mask are obtained, and the initial bright area mask and dark area mask are iterated. Different gamma corrections are applied to the standard brightness of the pixels under the optimal bright area mask and optimal dark area mask to obtain the optimal brightness of the pixels. The image to be processed is reconstructed based on the optimal brightness of each pixel to obtain the enhanced image, and tool recognition is performed based on the enhanced image. The steps for calculating the texture richness of the bright area mask include: for any pixel in the corresponding region of the bright area mask, the second-order gradient value of the pixel is calculated as the local contrast. A local observation window is constructed based on the pixel. The fusion complexity index of the pixel is calculated based on the information entropy of the gray value distribution in the observation window and the local contrast. The average of the texture complexity index of each pixel in the bright area is used as the texture richness.

2. The machine vision-based method for identifying maintenance tools inside a modular shelter according to claim 1, characterized in that, Based on the global brightness mean and global brightness standard deviation of the brightness channel, the initial bright area threshold and the initial dark area threshold are set.

3. The method for identifying maintenance tools inside a modular shelter based on machine vision according to claim 1, characterized in that, The result of the weighted sum of the information entropy of the gray-scale distribution and the local contrast in the observation window is used as the fusion complexity index.

4. The method for identifying maintenance tools inside a modular shelter based on machine vision according to claim 1, characterized in that, The steps for constructing a local observation window include: for any pixel, constructing a square region with a preset side length centered on that pixel, and using this square region as the local observation window.

5. The machine vision-based method for identifying maintenance tools inside a modular shelter according to claim 1, characterized in that, The steps for updating and iterating the bright area threshold include: in response to the texture richness of the corresponding region of the bright area mask being higher than the preset texture threshold, increasing the bright area threshold by a preset step size until the iteration stop condition is met or the preset number of iterations is reached.

6. The machine vision-based method for identifying maintenance tools inside a modular shelter according to claim 5, characterized in that, The iteration stops when the difference in texture richness between the bright area mask and the corresponding area before and after the iteration is less than a preset difference threshold.

7. The method for identifying maintenance tools inside a modular shelter based on machine vision according to claim 1, characterized in that, The steps for gamma correction of the standard brightness of pixels under the optimal bright area mask include: constructing a suppressed gamma index based on the mean and standard deviation of the original brightness of pixels in the corresponding region of the bright area mask; for any pixel, the result of a power function with the original brightness of the pixel as the base and the suppressed gamma index as the exponent is used as the corrected brightness, and the corrected brightness is negatively correlated with the suppressed gamma index.

8. The machine vision-based method for identifying maintenance tools inside a modular shelter according to claim 1, characterized in that, The first color space is the RGB color space; the second color space is the HSV color space.

9. The machine vision-based method for identifying maintenance tools inside a modular shelter according to claim 1, characterized in that, The steps for reconstructing the image to be processed based on the optimal brightness of each pixel to obtain the enhanced image include: mapping the corrected brightness of each pixel to the range of [0,255], merging it with other channel values ​​in the image to be processed, and converting it to the RGB color space to obtain the enhanced image.

Images