A mobile tool detection method and system for subway operation clearance scenarios

Abstract

The present application relates to the technical field of computer vision and artificial intelligence, and especially relates to a mobile tool detection method and system for a subway operation clearing scene, the method labels and separates the contour of a tool image, generates an occlusion sample enhanced dataset by transforming, stacking and adhering to a specified background, trains a target detection network, adjusts the bounding box regression loss weight according to the contour geometric features, performs global and local light and shadow anti-disturbance processing on the image to be detected, feeds the image into the network for preliminary detection, selectively compensates for the dynamic resolution and performs local re-detection according to the area scale, clusters the result space, merges the same clustering areas, and outputs the tool category and position coordinates. The present application constructs a tool detection process that integrates data enhancement, light and shadow anti-disturbance, dynamic resolution compensation and spatial clustering, realizes collaborative processing of training and detection, and solves the defects of insufficient detection accuracy in complex industrial scenes caused by the single detection process of the existing mobile tool detection.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and in particular to a mobile tool detection method and system for subway operation clearing scenarios. Background Technology

[0002] With the continuous expansion of urban rail transit construction, daily inspections, equipment maintenance, and construction operations on subway lines are becoming increasingly frequent. The clearing of tools and equipment after construction work has become a crucial step in ensuring the safe operation of rail transit. Currently, on-site tool and equipment inventory methods are gradually shifting from manual verification to intelligent visual inspection. This involves using mobile terminals to collect on-site images and combining them with target detection algorithms to automatically identify and locate tools such as screwdrivers, wrenches, pliers, and safety helmets. This enables tool and equipment category identification and location labeling, thereby meeting the application needs of mobile and intelligent on-site inspection and clearing.

[0003] Existing mobile tool inspection systems mostly adopt a single-target inspection process. Due to the lack of data construction and inspection process design for complex industrial scenarios, the inspection accuracy in complex industrial scenarios is insufficient. Summary of the Invention

[0004] To overcome the above shortcomings, this invention provides a mobile tool inspection method and system for subway operation clearing scenarios, aiming to improve the problem that existing mobile tool inspection methods mostly adopt a single target inspection process, which easily leads to insufficient inspection accuracy in complex industrial scenarios.

[0005] In a first aspect, the present invention provides the following technical solution: a mobile terminal tool detection method for subway operation clearing scenarios, comprising the following steps: S1. Perform contour masking on the tool image, extract the tool contour and separate it from the background. After random scaling, rotation and random stacking of multiple instances, the separated tool contour is pasted into the background of the glossy tile and grid lines to generate an enhanced training dataset containing mutually occluded samples. S2. Train the target detection network using the augmented training dataset generated in S1, and adjust the weights of the bounding box regression loss term in the joint loss function according to the geometric features of the tool outline. S3. Perform light and shadow anti-disturbance processing on the image to be detected, including global illumination processing and local illumination processing, to obtain an enhanced image; S4. Input the enhanced image obtained in S3 into the target detection network trained in S2 for initial detection. Determine whether to perform dynamic resolution compensation based on the scale information of the candidate detection region. Perform local cropping, resolution resampling and re-detection on the candidate detection region that performs dynamic resolution compensation. Use the re-detection result as the final detection result of the corresponding candidate detection region. Use the initial detection result as the final detection result of the candidate detection region that does not perform dynamic resolution compensation. S5. Perform spatial clustering processing on the final detection results obtained in S4, merge candidate detection regions belonging to the same cluster, and output the tool detection results including tool category labels and location coordinates.

[0006] By adopting the above technical solution, a tool detection process that includes data augmentation, light and shadow anti-disturbance processing, dynamic resolution compensation, and spatial clustering is constructed. This enables collaborative processing between the training and detection phases, thereby improving the problem that most existing mobile tool detection processes use a single-target detection process. Due to the lack of data construction and detection process design for complex industrial scenarios, the detection accuracy in complex industrial scenarios is insufficient.

[0007] Further, in S1, the step of extracting the tool outline and separating it from the background includes: Generate a binary mask for the corresponding tool based on the contour mask annotation results; The image region corresponding to the tool is determined based on the binary mask of the tool; Extract the foreground pixels corresponding to the tool based on the binary mask of the tool, and remove the background pixels in the image area to obtain the tool outline image; Record the category label and contour coordinates corresponding to the tool contour image to generate tool contour data.

[0008] Furthermore, in S1, the step of randomly scaling, rotating, and randomly stacking multiple instances of the separated tool outline and then attaching it to the glossy tile and grid background includes: Random scaling and random rotation are performed on the tool contour image in the tool contour data to generate a transformed tool contour image. Acquire background images of glossy tiles and grid lines, and then paste multiple transformed tool outline images onto the glossy tiles and grid line background images at random positions. During the pasting process, control the overlapping relationship between the tool outline images to generate a combined image containing mutual occlusion. Generate an enhanced training dataset from combined images containing mutual occlusion.

[0009] Further, in S2, the step of adjusting the weights of the bounding box regression loss term in the joint loss function based on the geometric features of the tool outline includes: Determine the minimum bounding rectangle corresponding to the tool profile based on the tool profile, and calculate the geometric feature data of the tool profile based on the minimum bounding rectangle. The weight data corresponding to the bounding box regression loss term is determined based on the aspect ratio in the geometric feature data of the tool outline, where the tool outline with a larger aspect ratio corresponds to a higher weight data. The weighted data is applied to the bounding box regression loss term in the joint loss function to generate a weighted joint loss function. The loss value of the object detection network is calculated based on the weighted joint loss function, and the network parameters of the object detection network are updated based on the loss value.

[0010] Furthermore, in S3, the step of performing light and shadow anti-disturbance processing on the image to be detected, including global illumination processing and local illumination processing, includes: Calculate global brightness statistics based on the image to be detected, and perform global illumination processing on the image to be detected based on the global brightness statistics to generate a globally illuminated image; The image to be detected is divided into multiple local image regions, and local illumination processing is performed on each local image region to generate a locally illuminated image. The global illumination processed image and the local illumination processed image are weighted and fused based on the local contrast information of the image to be detected to generate a fused image.

[0011] Furthermore, in S4, the step of determining whether to perform dynamic resolution compensation based on the scale information of the candidate detection region includes: The location coordinates of each candidate detection area are extracted based on the initial detection results, and the scale information of each candidate detection area is calculated based on the location coordinates. When the scale information of the candidate detection region meets the preset conditions, it is determined to perform dynamic resolution compensation on the candidate detection region. The preset conditions include that the pixel area of ​​the candidate detection region is less than the area threshold or the aspect ratio of the candidate detection region is greater than the aspect ratio threshold. Output the candidate detection regions for which dynamic resolution compensation is performed.

[0012] Furthermore, in S4, the steps of locally cropping, resampling, and re-detecting the candidate detection region for which dynamic resolution compensation is performed include: Based on the position coordinates of the candidate detection regions that have undergone dynamic resolution compensation, the enhanced image is locally cropped to generate local image regions. Perform resolution resampling on a local image region to generate a resampled image; The resampled image is input into the target detection network for re-detection, generating a re-detection result; The detection results corresponding to the candidate detection regions for which dynamic resolution compensation is performed are updated based on the results of the re-detection.

[0013] Further, in S5, the step of performing spatial clustering processing on the final detection results obtained in S4 and merging candidate detection regions belonging to the same cluster includes: The location coordinates of each candidate detection area are extracted based on the final detection results, and the spatial distance between the candidate detection areas is calculated based on the location coordinates of each candidate detection area. Generate clustering results corresponding to candidate detection regions based on the spatial distance between them; Based on the clustering results, candidate detection regions belonging to the same cluster are identified, and a merging process is performed on the candidate detection regions belonging to the same cluster to generate merged candidate detection regions; Tool and equipment detection results are generated based on the merged candidate detection areas.

[0014] Secondly, the present invention provides a mobile tool detection system for subway operation clearing scenarios, including an image acquisition terminal and a processor, wherein the processor runs the following modules: The data augmentation module is used to perform contour masking on tool images, extract tool contours and separate them from the background, and then apply the separated tool contours to the background of specular tiles and grid lines after random scaling, rotation and random stacking of multiple instances, generating an augmented training dataset containing mutually occluded samples. The training module is used to receive the augmented training dataset generated by the data augmentation module, train the target detection network using the augmented training dataset, and adjust the weight of the bounding box regression loss term in the joint loss function according to the geometric features of the tool outline to obtain the trained target detection network. The light and shadow processing module is used to acquire the image to be detected acquired by the image acquisition terminal, and to perform light and shadow anti-disturbance processing on the image to be detected, including global illumination processing and local illumination processing, to obtain an enhanced image; The detection module is used to receive the enhanced image output by the light and shadow processing module and the trained target detection network output by the training module, input the enhanced image into the trained target detection network for initial detection, determine whether to perform dynamic resolution compensation based on the scale information of the candidate detection region, perform local cropping, resolution resampling and re-detection on the candidate detection region that performs dynamic resolution compensation, and use the re-detection result as the final detection result of the corresponding candidate detection region. For the candidate detection region that does not perform dynamic resolution compensation, the initial detection result is used as the final detection result of the corresponding candidate detection region. The spatial clustering module is used to receive the final detection result output by the detection module, perform spatial clustering processing on the final detection result, merge candidate detection regions belonging to the same cluster, and output the tool detection result including tool category label and location coordinates.

[0015] The present invention has the following beneficial effects: 1. In this invention, a tool detection process that includes data augmentation, light and shadow anti-disturbance processing, dynamic resolution compensation, and spatial clustering is constructed to achieve collaborative processing between the training and detection stages. This improves upon the existing mobile tool detection process, which mostly uses a single-target detection process and lacks data construction and detection process design for complex industrial scenarios, resulting in insufficient detection accuracy in complex industrial scenarios.

[0016] 2. In this invention, by adjusting the weights of the bounding box regression loss term in the joint loss function according to the geometric features of the tool contour, the target detection network training process adopts bounding box regression constraints corresponding to the geometric features of the tool contour. This improves the problem that existing target detection network training mostly uses fixed bounding box regression loss weights, which leads to large positioning deviations for slender tools due to the use of the same bounding box regression constraints for targets with different geometric shapes. 3. In this invention, by performing global illumination processing and local illumination processing on the image to be detected, and performing weighted fusion based on local contrast information, an enhanced image for target detection is generated. This improves the problem that existing mobile tool detection mostly uses images without layered illumination processing for direct target detection, which leads to unstable target image features due to uneven illumination and local highlight areas in complex industrial scenes. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the overall process of a mobile terminal tool detection method for subway operation clearing scenarios provided by the present invention; Figure 2 This is a schematic diagram of the enhanced training dataset generation process provided by the present invention; Figure 3 This is a schematic diagram of the target detection network training process provided by the present invention; Figure 4 This is a schematic diagram of the light and shadow anti-disturbance processing flow provided by the present invention; Figure 5 This is a schematic diagram of the tool testing process provided by the present invention; Figure 6 This is a schematic diagram of the module architecture of a mobile tool detection system for subway operation clearing scenarios provided by the present invention. Detailed Implementation

[0018] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1: In the first embodiment of the present invention, the present invention provides a mobile tool detection method for subway operation clearing scenarios, such as... Figure 1 As shown, it includes the following steps: like Figure 2 As shown, S1, the tool image is masked and the tool contour is extracted and separated from the background. The separated tool contour is randomly scaled, rotated and randomly stacked with multiple instances and then pasted into the background of the glossy tile and grid lines to generate an enhanced training dataset containing mutually occluded samples. Furthermore, in S1, the step of extracting the tool outline and separating it from the background includes: Generate a binary mask for the corresponding tool based on the contour mask annotation results; The image region corresponding to the tool is determined based on the binary mask of the tool; Extract the foreground pixels corresponding to the tool based on the binary mask of the tool, and remove the background pixels in the image area to obtain the tool outline image; Record the category label and contour coordinates corresponding to the tool contour image to generate tool contour data.

[0020] Specifically, when performing contour masking on tool images, polygons are used to outline the edges of the tools to generate a fine mask region, and then a corresponding binary mask is generated based on this mask region. ;in Representing coordinates This is the foreground pixel. Represents background pixels; based on a binary mask Determine the image region corresponding to the tool, and extract the original image of the tool. With binary mask Perform a pixel-by-pixel bitwise AND operation to extract the foreground pixels corresponding to the tool. Simultaneously, background pixels in the image region are removed to obtain an outline image containing only the foreground of the tool. Record the outline image of the tool. Corresponding category tags and contour coordinate sequence Generate tool and fixture contour data This contour data will serve as input for subsequent data augmentation.

[0021] Furthermore, in S1, the step of randomly scaling, rotating, and randomly stacking multiple instances of the separated tool outline and then fitting it to the glossy tile and grid background includes: Random scaling and random rotation are performed on the tool contour image in the tool contour data to generate a transformed tool contour image. Acquire background images of glossy tiles and grid lines, and then paste multiple transformed tool outline images onto the glossy tiles and grid line background images at random positions. During the pasting process, control the overlapping relationship between the tool outline images to generate a combined image containing mutual occlusion. Generate an enhanced training dataset from combined images containing mutual occlusion.

[0022] Specifically, the tool contour images in the tool contour data. Perform random scaling and random rotation operations, with the random scaling ratio ranging from [value missing]. The random rotation angle range is Generate the transformed tool outline image ; Obtain a background image containing highlight tiles and grid line noise. Multiple transformed tool outline images Adhere to the background image at random positions. In the above process, a non-linear fusion method is used at the junction of the tool outline edge and the background during the pasting process. The transition is achieved by weighting the foreground and background pixel values, and the weighting coefficient is determined according to the distance from the pixel to the outline edge, so that the stitching boundary between the pasted tool and the background is naturally transitioned. During the pasting process, the occlusion order of each tool is randomly assigned. The tools drawn later are covered on the tools drawn earlier according to the occlusion order, and the visible outline of the occluded tools is preserved in the covered area, generating a combined image containing mutual occlusion. Based on combined images containing mutual occlusion The corresponding instance-level annotation information is used to generate an enhanced training dataset. Each training sample contains a combined image and the category label and bounding box coordinates of each tool instance.

[0023] like Figure 3 As shown, S2 uses the augmented training dataset generated in S1 to train the target detection network, and adjusts the weights of the bounding box regression loss term in the joint loss function according to the geometric features of the tool outline. Furthermore, in S2, the steps of adjusting the weights of the bounding box regression loss term in the joint loss function based on the geometric features of the tool outline include: Determine the minimum bounding rectangle corresponding to the tool profile based on the tool profile, and calculate the geometric feature data of the tool profile based on the minimum bounding rectangle. The weight data corresponding to the bounding box regression loss term is determined based on the aspect ratio in the geometric feature data of the tool outline, where the tool outline with a larger aspect ratio corresponds to a higher weight data. The weighted data is applied to the bounding box regression loss term in the joint loss function to generate a weighted joint loss function. The loss value of the object detection network is calculated based on the weighted joint loss function, and the network parameters of the object detection network are updated based on the loss value.

[0024] Specifically, in the construction of the joint loss function, the total loss function is: ;in For classifying losses, For bounding box regression loss, The weighting coefficients are used for the bounding box regression loss term; the minimum bounding rectangle corresponding to the tool contour is determined based on the tool contour, and the width of the minimum bounding rectangle is used as the basis for the weighting coefficients. and height Calculate the geometric feature data of the tool's outline, including the aspect ratio. According to aspect ratio Dynamic calculation of weight coefficients ;in Based on the weights, in this scheme The preferred value is 1.0. As a regulating factor, in this scheme The preferred value is 0.5, which ensures that tool outlines with larger aspect ratios correspond to higher bounding box regression loss weights; the weight coefficients are... The bounding box regression loss term applied to the joint loss function generates a weighted joint loss function. ,according to The loss value of the object detection network is calculated, and the network parameters are updated using the backpropagation algorithm. During training, the bounding box regression loss adopts the CIoU loss function. ;in The intersection-union ratio (IU) of the predicted bounding box and the ground truth bounding box. To predict the Euclidean distance between the center point of the bounding box and the center point of the ground truth bounding box, It is the length of the diagonal of the smallest bounding rectangle that encloses the predicted bounding box and the ground truth bounding box. To balance the parameters, ; For aspect ratio consistency parameters, .

[0025] like Figure 4 As shown, S3, the image to be detected undergoes light and shadow anti-disturbance processing, including global illumination processing and local illumination processing, to obtain an enhanced image; Furthermore, in S3, the step of performing light and shadow anti-perturbation processing on the image to be detected, including global illumination processing and local illumination processing, includes: Calculate global brightness statistics based on the image to be detected, and perform global illumination processing on the image to be detected based on the global brightness statistics to generate a globally illuminated image; The image to be detected is divided into multiple local image regions, and local illumination processing is performed on each local image region to generate a locally illuminated image. The global illumination processed image and the local illumination processed image are weighted and fused based on the local contrast information of the image to be detected to generate a fused image.

[0026] Specifically, the image to be detected Calculate global brightness statistics, including the global brightness mean. and global brightness standard deviation Based on global brightness statistics, global illumination processing is performed on the image to be detected. By using histogram matching or gamma correction, the brightness distribution of the image to be detected is mapped to a preset standard brightness reference, generating a globally illuminated image. ; the image to be detected Divided into The local image region in this scheme Preferably, 8 is used to perform local illumination processing on each local image region, specifically contrast-limited adaptive histogram equalization processing, where the contrast limiting threshold is preferably 2.0 in this scheme, generating a locally illuminated image. ; Calculate the local contrast information of the image to be detected, for each pixel location. The standard deviation of pixel values ​​within a neighborhood window centered on it is used as the local contrast value. Generate a fusion weight map based on local contrast information. In this plan The preferred value is 0.1 for global illumination processing images. and local illumination processing images Perform weighted fusion to generate a fused image. The fused image is output as an enhanced image in the subsequent detection stage.

[0027] like Figure 5As shown in S4, the enhanced image obtained in S3 is input into the target detection network trained in S2 for initial detection. Based on the scale information of the candidate detection region, it is determined whether to perform dynamic resolution compensation. For the candidate detection region that performs dynamic resolution compensation, local cropping, resolution resampling and re-detection are performed. The re-detection result is used as the final detection result of the corresponding candidate detection region. For the candidate detection region that does not perform dynamic resolution compensation, the initial detection result is used as the final detection result of the corresponding candidate detection region. Furthermore, in S4, the step of determining whether to perform dynamic resolution compensation based on the scale information of the candidate detection region includes: The location coordinates of each candidate detection area are extracted based on the initial detection results, and the scale information of each candidate detection area is calculated based on the location coordinates. When the scale information of the candidate detection region meets the preset conditions, dynamic resolution compensation is performed on the candidate detection region. The preset conditions include that the pixel area of ​​the candidate detection region is less than the area threshold or the aspect ratio of the candidate detection region is greater than the aspect ratio threshold. Output the candidate detection regions for which dynamic resolution compensation is performed.

[0028] Specifically, it will enhance the image. The trained object detection network is input for initial detection, and the network outputs the initial detection results, including the location coordinates of each candidate detection region. The system calculates category confidence and category labels; it also extracts the location coordinates of each candidate detection region based on the initial detection results and calculates the scale information of each candidate detection region, including the pixel area of ​​the candidate detection region. and aspect ratio ;in When the scale information of a candidate detection region meets a preset condition, dynamic resolution compensation is performed on that candidate detection region. The preset condition is the pixel area of ​​the candidate detection region. Less than the area threshold or aspect ratio of the candidate detection region Greater than the aspect ratio threshold In this plan Preferably, it is the total pixel area of ​​the input image. , The preferred version is 3.0; the output determines the candidate detection region and its position coordinates for dynamic resolution compensation.

[0029] Furthermore, in S4, the steps of locally cropping, resampling, and re-detecting the candidate detection region for which dynamic resolution compensation is performed include: Based on the position coordinates of the candidate detection regions that have undergone dynamic resolution compensation, the enhanced image is locally cropped to generate local image regions. Perform resolution resampling on a local image region to generate a resampled image; The resampled image is input into the target detection network for re-detection, generating a re-detection result; The detection results corresponding to the candidate detection regions for which dynamic resolution compensation is performed are updated based on the results of the re-detection.

[0030] Specifically, for determining candidate detection regions for dynamic resolution compensation, their position coordinates are used as the basis. Enhanced image Perform local cropping, with the cropping area being a rectangular region centered on the candidate detection region and with sides expanded by a factor of 1.2, to generate a local image region. For local image regions Resolution resampling is performed, using bilinear interpolation to magnify the resolution of local image regions to 3 to 4 times the original value. In this scheme, the magnification factor is preferably 3, generating a resampled image. Resample the image The target detection network is input for re-detection, generating re-detection results. The re-detection results include the updated candidate detection region location coordinates, category confidence, and category label. The original detection data of the candidate detection region in the initial detection results is replaced based on the re-detection results. For candidate detection regions that have not undergone dynamic resolution compensation, their initial detection results are retained.

[0031] S5. Perform spatial clustering processing on the final detection results obtained in S4, merge candidate detection regions belonging to the same cluster, and output the tool detection results including tool category labels and location coordinates. Furthermore, in S5, spatial clustering is performed on the final detection results obtained in S4, and the steps to merge candidate detection regions belonging to the same cluster include: The location coordinates of each candidate detection area are extracted based on the final detection results, and the spatial distance between the candidate detection areas is calculated based on the location coordinates of each candidate detection area. Generate clustering results corresponding to candidate detection regions based on the spatial distance between them; Based on the clustering results, candidate detection regions belonging to the same cluster are identified, and a merging process is performed on the candidate detection regions belonging to the same cluster to generate merged candidate detection regions; Tool and equipment detection results are generated based on the merged candidate detection areas.

[0032] Specifically, the position coordinates of each candidate detection region are extracted based on the final detection results. For any two candidate detection regions... and Calculate the spatial distance between the two using the intersection-union ratio. As a measure of spatial distance ;in and These are the bounding boxes of the two candidate detection regions; calculate pairwise bounding boxes for all candidate detection regions. ,by Greater than the clustering threshold An undirected graph is constructed for the edges, and each connected component in the graph is taken as a cluster. In this scheme... The preferred value is 0.5; merging is performed on candidate detection regions belonging to the same cluster. The merging method is to take the average coordinates of all candidate detection regions within the same cluster as the coordinates of the merged candidate detection region, and take the category label corresponding to the highest confidence level within the same cluster as the category label of the merged candidate detection region to generate the merged candidate detection region; the final tool detection result is generated based on the merged candidate detection region, including the category label and location coordinates of each tool instance.

[0033] Example 2: In a scenario where tools and equipment are being cleared and inventoried after track maintenance work in a subway tunnel, workers place screwdrivers, wrenches, insulating rods, sockets, and other tools in a tunnel maintenance area with glossy tiled floors and grid lines. An industrial tablet PC's rear-facing camera captures images of the tool and equipment area. In this scenario, the tunnel lighting is uneven, the tiled floor produces localized high-gloss reflections and glare, and some small tools, such as screwdriver tips, are mixed in with the reflective background, making manual inventory easy to miss and time-consuming. To solve these problems, this invention provides a mobile tool and equipment detection system for subway maintenance clearing scenarios, such as... Figure 6 As shown. The system includes a rear-facing camera on an industrial tablet PC serving as an image acquisition terminal, and an embedded processor built into the industrial tablet PC serving as a processor. The processor runs data augmentation, training, lighting and shadow processing, detection, and spatial clustering modules. The specific implementation process of the system is as follows: During the offline training phase, the data augmentation module first performs contour masking on the tool images and extracts the tool contours to separate them from the background. After random scaling, rotation, and random stacking of multiple instances, the data is then pasted onto the background of the glossy tiles and grid lines to generate an augmented training dataset containing mutually occluded samples. This ensures that the training data covers high-density stacking and extreme occlusion situations that are difficult to collect in real-world scenarios. Then, the training module uses this augmented training dataset to train a lightweight object detection network and dynamically adjusts the weights of the bounding box regression loss term according to the aspect ratio of the tool contours. The longer the aspect ratio of the slender tool, the higher the regression loss weight, resulting in the trained object detection network.

[0034] In the online inference phase, the image acquisition terminal first captures the image to be detected and transmits it to the lighting and shadow processing module. The lighting and shadow processing module performs lighting and shadow anti-disturbance processing on the image to be detected, including global illumination processing and local illumination processing, to eliminate large-area shadows and suppress local reflections and glare on the tiles, respectively. It also obtains an enhanced image based on local contrast weighted fusion, so that the texture of small tools in dark areas and the edges of tools in high-reflection areas are clearly presented. Then, the detection module inputs the enhanced image into the trained target detection network for initial detection. For candidate detection regions with pixel area smaller than the threshold or aspect ratio greater than the threshold, dynamic resolution compensation is performed. After local cropping and resolution resampling, the detection is performed again, and the re-detection result replaces the initial detection result, thereby accurately detecting targets such as small screwdriver tips that are easily missed in the initial detection. Finally, the spatial clustering module calculates the intersection-union ratio (IUGR) between each candidate detection region in the final detection result. Candidate detection regions with IUGR greater than the threshold are merged into the same cluster, eliminating duplicate detection boxes for the same tool, and outputting the tool detection result including tool category label and location coordinates. The operators compare the test results displayed on the industrial tablet computer screen with the actual list of tools and equipment they are carrying to quickly confirm whether any tools or equipment have been omitted and complete the clearance confirmation.

[0035] Finally, it should be noted that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A mobile terminal tool detection method for subway operation clearing scenarios, characterized in that, Includes the following steps: S1. Perform contour masking on the tool image, extract the tool contour and separate it from the background. After random scaling, rotation and random stacking of multiple instances, the separated tool contour is pasted into the background of the glossy tile and grid lines to generate an enhanced training dataset containing mutually occluded samples. S2. Train the target detection network using the augmented training dataset generated in S1, and adjust the weights of the bounding box regression loss term in the joint loss function according to the geometric features of the tool outline. S3. Perform light and shadow anti-disturbance processing on the image to be detected, including global illumination processing and local illumination processing, to obtain an enhanced image; S4. Input the enhanced image obtained in S3 into the target detection network trained in S2 for initial detection. Determine whether to perform dynamic resolution compensation based on the scale information of the candidate detection region. Perform local cropping, resolution resampling and re-detection on the candidate detection region that performs dynamic resolution compensation. Use the re-detection result as the final detection result of the corresponding candidate detection region. Use the initial detection result as the final detection result of the candidate detection region that does not perform dynamic resolution compensation. S5. Perform spatial clustering processing on the final detection results obtained in S4, merge candidate detection regions belonging to the same cluster, and output the tool detection results including tool category labels and location coordinates.

2. The mobile terminal tool detection method for subway operation clearing scenarios according to claim 1, characterized in that, In S1, the step of extracting the tool outline and separating it from the background includes: Generate a binary mask for the corresponding tool based on the contour mask annotation results; The image region corresponding to the tool is determined based on the binary mask of the tool; Extract the foreground pixels corresponding to the tool based on the binary mask of the tool, and remove the background pixels in the image area to obtain the tool outline image; Record the category label and contour coordinates corresponding to the tool contour image to generate tool contour data.

3. The mobile terminal tool detection method for subway operation clearing scenarios according to claim 1, characterized in that, In S1, the step of randomly scaling, rotating, and randomly stacking multiple instances of the separated tool outline and then attaching it to the glossy tile and grid background includes: Random scaling and random rotation are performed on the tool contour image in the tool contour data to generate a transformed tool contour image. Acquire background images of glossy tiles and grid lines, and then paste multiple transformed tool outline images onto the glossy tiles and grid line background images at random positions. During the pasting process, control the overlapping relationship between the tool outline images to generate a combined image containing mutual occlusion. Generate an enhanced training dataset from combined images containing mutual occlusion.

4. The mobile terminal tool detection method for subway operation clearing scenarios according to claim 1, characterized in that, In S2, the step of adjusting the weights of the bounding box regression loss term in the joint loss function based on the geometric features of the tool outline includes: Determine the minimum bounding rectangle corresponding to the tool profile based on the tool profile, and calculate the geometric feature data of the tool profile based on the minimum bounding rectangle. The weight data corresponding to the bounding box regression loss term is determined based on the aspect ratio in the geometric feature data of the tool outline, where the tool outline with a larger aspect ratio corresponds to a higher weight data. The weighted data is applied to the bounding box regression loss term in the joint loss function to generate a weighted joint loss function. The loss value of the object detection network is calculated based on the weighted joint loss function, and the network parameters of the object detection network are updated based on the loss value.

5. A mobile tool detection method for subway operation clearing scenarios according to claim 1, characterized in that, In S3, the step of performing light and shadow anti-disturbance processing on the image to be detected, including global illumination processing and local illumination processing, includes: Calculate global brightness statistics based on the image to be detected, and perform global illumination processing on the image to be detected based on the global brightness statistics to generate a globally illuminated image; The image to be detected is divided into multiple local image regions, and local illumination processing is performed on each local image region to generate a locally illuminated image. The global illumination processed image and the local illumination processed image are weighted and fused based on the local contrast information of the image to be detected to generate a fused image.

6. The mobile terminal tool detection method for subway operation clearing scenarios according to claim 1, characterized in that, In S4, the step of determining whether to perform dynamic resolution compensation based on the scale information of the candidate detection region includes: The location coordinates of each candidate detection area are extracted based on the initial detection results, and the scale information of each candidate detection area is calculated based on the location coordinates. When the scale information of the candidate detection region meets the preset conditions, it is determined to perform dynamic resolution compensation on the candidate detection region. The preset conditions include that the pixel area of ​​the candidate detection region is less than the area threshold or the aspect ratio of the candidate detection region is greater than the aspect ratio threshold. Output the candidate detection regions for which dynamic resolution compensation is performed.

7. A mobile tool detection method for subway operation clearing scenarios according to claim 1, characterized in that, In S4, the steps of locally cropping, resampling, and re-detecting the candidate detection region after dynamic resolution compensation include: Based on the position coordinates of the candidate detection regions that have undergone dynamic resolution compensation, the enhanced image is locally cropped to generate local image regions. Perform resolution resampling on a local image region to generate a resampled image; The resampled image is input into the target detection network for re-detection, generating a re-detection result; The detection results corresponding to the candidate detection regions for which dynamic resolution compensation is performed are updated based on the results of the re-detection.

8. A mobile tool detection method for subway operation clearing scenarios according to claim 1, characterized in that, In S5, the step of performing spatial clustering processing on the final detection results obtained in S4 and merging candidate detection regions belonging to the same cluster includes: The location coordinates of each candidate detection area are extracted based on the final detection results, and the spatial distance between the candidate detection areas is calculated based on the location coordinates of each candidate detection area. Generate clustering results corresponding to candidate detection regions based on the spatial distance between them; Based on the clustering results, candidate detection regions belonging to the same cluster are identified, and a merging process is performed on the candidate detection regions belonging to the same cluster to generate merged candidate detection regions; Tool and equipment detection results are generated based on the merged candidate detection areas.

9. A mobile tool and equipment inspection system for subway operation clearing scenarios, characterized in that, It includes an image acquisition terminal and a processor, wherein the processor runs the following modules: The data augmentation module is used to perform contour masking on tool images, extract tool contours and separate them from the background, and then apply the separated tool contours to the background of specular tiles and grid lines after random scaling, rotation and random stacking of multiple instances, generating an augmented training dataset containing mutually occluded samples. The training module is used to receive the augmented training dataset generated by the data augmentation module, train the target detection network using the augmented training dataset, and adjust the weight of the bounding box regression loss term in the joint loss function according to the geometric features of the tool outline to obtain the trained target detection network. The light and shadow processing module is used to acquire the image to be detected acquired by the image acquisition terminal, and to perform light and shadow anti-disturbance processing on the image to be detected, including global illumination processing and local illumination processing, to obtain an enhanced image; The detection module is used to receive the enhanced image output by the light and shadow processing module and the trained target detection network output by the training module, input the enhanced image into the trained target detection network for initial detection, determine whether to perform dynamic resolution compensation based on the scale information of the candidate detection region, perform local cropping, resolution resampling and re-detection on the candidate detection region that performs dynamic resolution compensation, and use the re-detection result as the final detection result of the corresponding candidate detection region. For the candidate detection region that does not perform dynamic resolution compensation, the initial detection result is used as the final detection result of the corresponding candidate detection region. The spatial clustering module is used to receive the final detection result output by the detection module, perform spatial clustering processing on the final detection result, merge candidate detection regions belonging to the same cluster, and output the tool detection result including tool category label and location coordinates.

Images