Manhole cover disease and ownership collaborative identification method and system based on multi-label detection

Abstract

The present application relates to a kind of manhole cover disease and ownership collaborative identification method and system based on multi-label detection, which comprises: obtaining the multi-frame road surface image of same manhole cover at different collection time and corresponding positioning information;Each frame image is input into multi-label target detection model, and the bounding box of manhole cover instance in each frame and the prediction result of multiple independent attributes are output, and the attribute at least includes disease state and visual ownership identification, the head network of the model independently outputs confidence for each attribute channel, and each channel is not mutually constrained;According to the positioning information, the detection results of same manhole cover are associated and consistent verification across frame, determine the target manhole cover instance that there is disease;Arc-shaped arrangement text recognition is carried out to the bounding box region of target manhole cover instance, and ownership unit text information is extracted, and the ownership unit of target manhole cover instance is determined by integrating visual ownership identification prediction result and text information;According to disease state prediction result, ownership unit and positioning information, generate maintenance work order.

Description

Technical Field

[0001] This invention belongs to the field of image recognition technology, and in particular relates to a method and system for collaborative identification of manhole cover defects and ownership based on multi-label detection. Background Technology

[0002] Manhole covers are critical exposed nodes in urban underground pipe networks. If damage, missing parts, displacement, or unevenness between the manhole cover and the road surface are not detected and addressed promptly, they can easily lead to vehicle accidents and pedestrian falls, directly threatening public safety and causing asset damage. Taking a certain city as an example, in 2023, over 30,000 cases of structural defects in municipal manholes were addressed. Of these, damage to the manhole cover and surrounding area, displacement, and unevenness between the manhole cover and the ground accounted for approximately 95%, resulting in a high number of citizen hotline complaints and direct economic losses. Therefore, timely and accurate identification of manhole cover defects and determining their ownership is a crucial step in ensuring the safe operation of the city.

[0003] Traditional manhole cover maintenance relies primarily on two methods, neither of which can meet the demands of modern, sophisticated management. The first is manual inspection, where inspectors visually assess defects and identify manhole cover markings to confirm ownership. This method poses high safety risks on busy main roads, has limited coverage, inconsistent defect assessment standards, and is prone to overlooking minor defects. Manhole cover markings are often worn or damaged, making them difficult to read, leading to misrecognition, omissions, and confusion regarding ownership. Furthermore, centralized backend processing suffers from significant time lag. The second method is automatic recognition based on traditional image processing techniques such as edge detection, template matching, and color analysis. This approach is highly sensitive to complex road conditions such as lighting, shadows, water accumulation, and occlusion, exhibiting poor robustness and a high rate of false positives and false negatives. It typically only determines the presence or absence of a manhole cover or distinguishes a very limited number of defect types. Furthermore, it requires manual redesign of features when the environment changes, resulting in high maintenance costs.

[0004] In recent years, with the development of deep learning technology, automatic identification of manhole cover defects based on object detection models has gradually become a research hotspot. For example, patent application CN111914634A proposes an automatic manhole cover category detection method and system resistant to interference from complex scenes. This method uses ResNet-101 as the network skeleton to construct an R-FCN manhole cover detection model, which detects manhole covers in surveillance video images and identifies three categories: intact, damaged, and missing. However, this method is based on the R-FCN framework, and its detection accuracy is limited by the quality of the regional proposal network. It can only determine the coarse condition of the manhole cover and cannot simultaneously identify attributes such as the ownership unit of the manhole cover.

[0005] For example, patent application CN118334595A proposes an intelligent detection method and system for manhole cover hazards. This method utilizes a deformable large kernel attention mechanism and a target dynamic detection head to construct a manhole cover hazard detection model, and performs detection and identification on a set of manhole cover hazard images, achieving improved detection accuracy compared to traditional methods. However, this method still follows the traditional single-label classification paradigm. For multiple independent attributes of the same manhole cover target (such as damage status, ownership unit, surface pattern, etc.), it is still necessary to reason separately or connect multiple dedicated models, which significantly increases the recognition time and makes it difficult to meet the real-time processing requirements of mobile platforms.

[0006] In recent years, vehicle-mounted mobile inspection platforms have emerged, enabling rapid and continuous acquisition of road surface images through high-definition cameras, providing a new approach for main road inspection. However, significant shortcomings remain in the identification and information extraction stages after acquisition. On one hand, manhole covers require simultaneous identification of multiple independent attributes, such as damage status, ownership unit, and surface patterns. Existing Softmax single-label classification models are constrained by class exclusivity, requiring separate reasoning for each attribute or the concatenation of multiple dedicated models, which exponentially increases recognition time and fails to meet the real-time requirements of mobile platforms. Furthermore, in reality, the vast majority of manhole covers are normal, with few damaged samples, making manual review of massive amounts of images highly susceptible to missed detections. On the other hand, damaged manhole covers and logos often occupy less than 2% of the area in the entire road surface image, leading to severe under-detection by general detection models. Moreover, ownership text is often arranged in an arc along the edge of the manhole cover, making it difficult for conventional text recognition to read accurately. In addition, in the traditional workflow, after manually identifying the disease, it is necessary to manually check the location information, sort out the disease type and ownership, fill in the form to generate a work order and send it to the maintenance unit. The whole process has many steps, is slow, and manual input is prone to errors. The results after the maintenance is completed also need to be updated manually in real time. There is a lack of a systematic automatic closed-loop mechanism.

[0007] In summary, existing technologies cannot efficiently and safely complete mobile data collection while simultaneously achieving accurate identification of manhole cover defects and automatic determination of ownership, and forming a closed-loop management system from identification to treatment. There is an urgent need for an integrated solution that can adapt to real-time operation of mobile platforms and synchronously identify defects and ownership information. Summary of the Invention

[0008] In view of the shortcomings of the existing technology, the purpose of the invention is to provide a method and system for collaborative identification of manhole cover defects and ownership based on multi-label detection. This method and system can realize mobile non-contact inspection of manhole covers on main roads, parallel identification of multiple attributes of defect status and ownership unit, accurate reading of arc-shaped ownership text, and automatic generation of maintenance work orders, effectively improving the efficiency, accuracy and intelligence level of manhole cover defect identification.

[0009] In a first aspect, the present invention proposes a method for collaborative identification of manhole cover defects and ownership based on multi-label detection, comprising: acquiring a road surface image sequence collected by a mobile acquisition platform while traveling along a road, wherein the road surface image sequence includes multiple frames of road surface images of the same manhole cover at multiple acquisition times from far to near, and each frame of road surface image corresponds to the location information of the acquisition time; inputting the multiple frames of road surface images into a multi-label target detection model for inference, and the multi-label target detection model outputting the bounding box of each manhole cover instance in each frame of road surface image and the prediction results of multiple independent attributes; wherein the multiple independent attributes include at least the defect status attribute and the visual ownership identification attribute of the manhole cover; the head network of the multi-label target detection model is configured to detect each attribute Each channel outputs confidence independently, and the outputs of each attribute channel are independent of each other. Based on the location information, the detection results of the same manhole cover in the multi-frame road surface images are correlated across frames, and the correlation results are verified for consistency. Based on the consistency verification results, the target manhole cover instance with defects is determined. Text recognition is performed on the image region corresponding to the bounding box of the target manhole cover instance for arc-shaped text, and the ownership unit text information is extracted. Based on the prediction results of the visual ownership identifier attribute of each manhole cover instance and the ownership unit text information, the ownership unit of the target manhole cover instance is determined. Based on the prediction results of the defect status attribute, the ownership unit, and the location information, a maintenance work order for the target manhole cover instance is generated.

[0010] According to a second aspect of the present disclosure, a storage medium is provided, the storage medium including a stored program, wherein the above-described method is executed by a processor when the program is running.

[0011] According to a third aspect of the present disclosure, a collaborative identification system for manhole cover defects and ownership based on multi-label detection is provided, comprising: an image acquisition module for acquiring a sequence of road surface images collected by a mobile acquisition platform while traveling along a road, the road surface image sequence including multiple frames of road surface images of the same manhole cover at multiple acquisition times from far to near, each frame of road surface image corresponding to the location information of the acquisition time; and a target detection module for inputting the multiple frames of road surface images into a multi-label target detection model for inference, the multi-label target detection model outputting the bounding box of each manhole cover instance in each frame of road surface image and prediction results of multiple independent attributes; the multiple independent attributes include at least the defect status attribute and the visual ownership identification attribute of the manhole cover; the head network of the multi-label target detection model is configured to independently input for each attribute channel. The system outputs confidence scores, and the outputs of each attribute channel are independent of each other; the target manhole cover instance determination module is used to perform cross-frame association of detection results belonging to the same manhole cover in the multi-frame road surface images based on the positioning information, and to perform consistency verification on the associated detection results, and to determine the target manhole cover instance with defects based on the consistency verification results; the ownership unit determination module is used to perform text recognition for arc-shaped text in the image area corresponding to the bounding box of the target manhole cover instance, extract the ownership unit text information, and determine the ownership unit of the target manhole cover instance based on the prediction results of the visual ownership identifier attribute of each manhole cover instance and the ownership unit text information; the maintenance work order generation module is used to generate a maintenance work order for the target manhole cover instance based on the prediction results of the defect status attribute, the ownership unit and the positioning information.

[0012] According to a fourth aspect of the present disclosure, a collaborative identification system for manhole cover defects and ownership based on multi-label detection is provided, comprising: a processor; and a memory connected to the processor, configured to provide the processor with instructions for processing the following steps: acquiring a sequence of road surface images collected by a mobile acquisition platform while it is traveling along a road, the sequence of road surface images including multiple frames of road surface images of the same manhole cover at multiple acquisition times from far to near, each frame of road surface image corresponding to the location information of the acquisition time; inputting the multiple frames of road surface images into a multi-label target detection model for inference, and having the multi-label target detection model output the bounding box of each manhole cover instance in each frame of road surface image and prediction results of multiple independent attributes; the multiple independent attributes include at least the defect status attribute and the visual ownership identification attribute of the manhole cover; the multi-label... The head network of the target detection model is configured to output confidence scores independently for each attribute channel, and the outputs of each attribute channel are independent of each other. Based on the location information, the detection results of the same manhole cover in the multi-frame road surface images are correlated across frames, and the correlation results are verified for consistency. Based on the consistency verification results, the target manhole cover instance with defects is determined. Text recognition is performed on the image region corresponding to the bounding box of the target manhole cover instance for arc-shaped text, and the ownership unit text information is extracted. Based on the prediction results of the visual ownership identifier attribute of each manhole cover instance and the ownership unit text information, the ownership unit of the target manhole cover instance is determined. Based on the prediction results of the defect status attribute, the ownership unit, and the location information, a maintenance work order for the target manhole cover instance is generated.

[0013] This application achieves mobile, non-contact inspection of manhole covers on main roads by acquiring road surface image sequences collected by a mobile acquisition platform while it travels along the road. These images include multiple frames of road surface images of the same manhole cover from far to near, with each frame corresponding to the location information at the time of acquisition. This avoids the safety risks and coverage limitations of manual inspection. By inputting multiple frames of road surface images into a multi-label target detection model for inference, and independently outputting confidence scores for each attribute channel with no mutual constraints, this application achieves parallel output of multiple independent attributes such as damage status and visual ownership identification in a single forward inference. This avoids the efficiency bottleneck of traditional single-label models, which require multiple inferences or concatenation of multiple models due to class exclusivity, significantly reducing recognition time. Furthermore, by associating detection results belonging to the same manhole cover across frames based on location information, the detection results of multiple frames of the same manhole cover collected from far to near can be accurately identified. By merging and then verifying the consistency of the associated detection results, motion blur and occasional false detections in single-frame detection are effectively suppressed, improving the reliability of the detection results. By performing text recognition on arc-shaped text in target manhole cover instances with defects, the problem of manhole cover ownership text being difficult to accurately read by conventional OCR along the arc-shaped edge of the manhole cover is solved. The ownership unit is comprehensively determined by combining the prediction results of the visual ownership identification attributes of each manhole cover instance. The cross-verification of visual identification and text recognition overcomes the deficiency of insufficient accuracy of single recognition methods in the case of worn or dirty identification. By automatically generating maintenance work orders based on the prediction results of defect status attributes, ownership unit, and location information, the automated flow from defect identification to work order output is realized, providing a structured data foundation for subsequent accurate work order dispatch and closed-loop treatment, and improving the overall intelligence and precision of urban manhole cover defect management. Attached Figure Description

[0014] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. It is obvious that the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings.

[0015] Figure 1 This is a flowchart of the collaborative identification method for manhole cover defects and ownership based on multi-tag detection, as described in Embodiment 1 of this application; Figure 2 This is a flowchart illustrating the overall technical process of the collaborative identification method for manhole cover defects and ownership based on multi-tag detection, as described in Embodiment 1 of this application. Figure 3 This is a schematic diagram of an image acquisition platform according to Embodiment 1 of this application; Figure 4This is a schematic diagram of the structure of the multi-label target detection model according to Embodiment 1 of this application; Figure 5 This is a schematic diagram of the structure of the C3k2 module embedded with a channel attention module and a spatial attention module according to Embodiment 1 of this application; Figure 6 This is a schematic diagram of the ADown module according to Embodiment 1 of this application; Figure 7 This is a schematic diagram of the structure of the multi-scale fusion module according to Embodiment 1 of this application; Figure 8 This is a flowchart of collaborative identification of manhole cover defects and ownership as described in Embodiment 1 of this application; Figure 9 This is a schematic diagram of arc-shaped ownership text correction according to Embodiment 1 of this application; Figure 10 This is the inspection flowchart according to Embodiment 1 of this application; Figure 11 This is a schematic diagram of a collaborative identification system for manhole cover defects and ownership based on multi-tag detection, as described in Embodiment 2 of this application. Figure 12 This is a schematic diagram of a collaborative identification system for manhole cover defects and ownership based on multi-label detection, as described in Embodiment 3 of this application. Detailed Implementation

[0016] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this disclosure.

[0017] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0018] Example 1

[0019] According to this embodiment, a method embodiment for collaborative identification of manhole cover defects and ownership based on multi-tag detection is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here. Figure 1 A flowchart illustrating the method is shown. Figure 2 A flowchart illustrating the overall technical process of this method is shown. (Reference) Figure 1 and Figure 2 As shown, the method includes: S1: Obtain a sequence of road surface images collected by the mobile acquisition platform while it is traveling along the road. The sequence of road surface images includes multiple frames of road surface images of the same manhole cover at multiple acquisition times, from far to near. Each frame of road surface image corresponds to the location information of the acquisition time.

[0020] In this embodiment, as Figure 3 As shown, the mobile data acquisition platform utilizes a municipal inspection vehicle as its mobile carrier. An industrial-grade high-definition camera (i.e., an industrial camera) is mounted on the roof, with the lens tilted downwards at approximately 25° and slightly tilted to the right. Its field of view primarily covers the roadway directly in front of the vehicle, while also taking into account the adjacent non-motorized vehicle lane on the right, thus achieving full coverage of areas prone to manhole covers. The industrial camera uses a global shutter, achieving a resolution of 2448×2048 and a frame rate cap of 60fps. Synchronous supplementary lighting is installed on both sides of the camera to ensure image quality in low-light environments such as dusk and night. A GPS / INS combined positioning module is installed inside the vehicle, providing a positioning accuracy of approximately ±0.5m and an output frequency of 100Hz. It is connected to an edge computing device for online inference. The camera, positioning module, and edge computing device are synchronized via hardware trigger signals, ensuring that each frame of the road image carries the location information at the time of acquisition, including latitude, longitude, station number, and timestamp.

[0021] During road image acquisition, the edge computing device obtains the current location information of the mobile acquisition platform in real time and controls the industrial camera to acquire images of the road surface accordingly. When the mobile acquisition platform passes the location of a manhole cover, the industrial camera continuously acquires multiple frames of road surface images of the same manhole cover from far to near at multiple acquisition times. Each frame of road surface image is assigned the location information of the acquisition time and stored in association to form a road surface image sequence.

[0022] Thus, by combining industrial cameras mounted on a mobile platform with positioning information, continuous acquisition of road surface images is achieved, ensuring high-definition images of each manhole cover are captured from multiple angles. By associating and storing each frame of road surface image with the positioning information at the time of acquisition, a data foundation is provided for subsequent cross-frame association and spatial positioning. This enables mobile, non-contact inspection of manhole covers on main roads, allowing for continuous acquisition of high-definition images without requiring inspection personnel to be on the road. This avoids the traffic safety risks of manual inspections on main roads and overcomes the poor image acquisition caused by limitations in manual positioning and movement speed.

[0023] S2: Input the multi-frame road surface images into the multi-label target detection model for inference, and output the bounding box of each manhole cover instance in each frame of road surface image and the prediction results of multiple independent attributes; the multiple independent attributes include at least the manhole cover's damage status attribute and visual ownership identification attribute; the head network of the multi-label target detection model is configured to output confidence for each attribute channel independently, and the outputs of each attribute channel are not mutually constrained.

[0024] In this embodiment, multiple frames of road surface images of the same manhole cover obtained in step S1, from far to near, are independently input into the multi-label target detection model. Each frame of road surface image serves as an independent input sample, and the model performs forward inference calculations. The model performs detection on each frame of road surface image separately, and the inference process of each frame does not interfere with each other.

[0025] For each frame of the input road surface image, the multi-label object detection model outputs the bounding boxes of each detected manhole cover instance in that frame, along with predictions for multiple independent attributes. The bounding boxes provide the position coordinates of each manhole cover in the image, including the center point coordinates and its width and height. For each manhole cover instance, the model outputs predictions for multiple independent attributes, each including a corresponding category label and confidence score.

[0026] The multiple independent attributes include at least the manhole cover's condition attribute and visual ownership identification attribute. The condition attribute characterizes the physical condition of the manhole cover, including categories such as normal, damaged, missing, displaced, settlement difference, damage to the manhole perimeter or frame, and looseness. The visual ownership identification attribute characterizes the identification information on the manhole cover surface, including the ownership unit's logo category and surface pattern category. Logo categories include electricity, communication, water supply, drainage, gas, and heating, while pattern categories include checkered patterns, radial patterns, circular patterns, grid patterns, anti-slip raised dots, and double-layer patterns. The aforementioned condition attribute and visual ownership identification attribute are physically independent; a manhole cover can simultaneously possess a certain condition, a certain logo, and a certain pattern, and there is no mutual exclusion among these three attributes.

[0027] To achieve parallel output across multiple attributes, the head network of the multi-label object detection model is configured to output confidence scores independently for each attribute channel. The number of output channels of the head network is redefined as the total number of categories for all independent attributes. Let the number of disease state categories be... The number of Logo categories is The number of pattern categories is Then the number of output channels of the head network is set to Each output channel corresponds to a specific attribute category, such as "damaged", "electrical", "checkered", etc., and the head network calculates a confidence score for each channel separately.

[0028] Meanwhile, the outputs of each attribute channel are independent of each other. Specifically, the head network abandons the traditional Softmax activation function, which forces the sum of all channel outputs to be 1, causing channels to mutually exclude each other; an increase in the confidence of one channel will inevitably suppress the confidence of other channels. This embodiment uses a Sigmoid activation function that acts independently on each channel, calculating the activation value for each channel independently. The output values ​​of each channel are between 0 and 1, and there is no constraint relationship between channels that sum to 1. Therefore, the same manhole cover instance can output a high confidence of "damaged" in the damage status channel, a high confidence of "electricity" in the Logo channel, and a high confidence of "checkered pattern" in the pattern channel. The prediction results of multiple independent attributes can be high simultaneously without suppressing each other. For attribute labels with physical mutual exclusion, such as "damaged" and "intact" belonging to mutually exclusive categories under the same damage status attribute, the final category is determined by taking the maximum confidence of the mutually exclusive channels during the model inference stage.

[0029] Thus, the head network of the multi-label target detection model achieves parallel output of multiple attributes such as disease status, logo, and pattern in a single forward inference by independently outputting confidence for each attribute channel and without mutual constraints between channels. This avoids the efficiency bottleneck of traditional single-label models that require separate inference for each attribute or concatenation of multiple dedicated models due to the mutual exclusion of Softmax categories. It significantly shortens the overall recognition time of multiple frames of images and meets the real-time processing requirements of mobile platforms.

[0030] S3: Based on the positioning information, the detection results of the same manhole cover in the multi-frame road surface images are correlated across frames, and the correlation results are verified for consistency. Based on the consistency verification results, the target manhole cover instance with defects is determined.

[0031] In this embodiment, during the cross-frame association stage, since the detection results of the same manhole cover in multiple frames are spatially adjacent, the detection results of multiple frames that are spatially adjacent and temporally continuous are grouped into the same manhole cover instance using GPS or INS positioning information and timestamps. Thus, the multi-frame detection results of the same manhole cover collected from far to near are accurately merged.

[0032] Subsequently, the system performs consistency verification on the associated multi-frame detection results, statistically analyzing the detected defects of the manhole cover in each frame. Only when the same defect is consistently detected in at least a preset number of frames (e.g., but not limited to 3 frames) is it confirmed, effectively suppressing false detections caused by motion blur, occlusion, or occasional noise in single-frame detection. After multi-frame voting confirmation, the system identifies the manhole cover instance determined to have defects as the target manhole cover instance.

[0033] Thus, by accurately merging multiple detection results of the same manhole cover collected from far to near using location information, and then suppressing false detections in a single frame through consistency verification, the reliability of the detection results is improved.

[0034] S4: Perform text recognition on the image region corresponding to the bounding box of the target manhole cover instance for arc-shaped text, extract the ownership unit text information, and determine the ownership unit of the target manhole cover instance based on the prediction results of the visual ownership identification attributes of each manhole cover instance and the ownership unit text information.

[0035] In this embodiment, the system crops the manhole cover area image from the corresponding road surface image based on the bounding box of the target manhole cover instance determined in step S3. Then, it performs text recognition on the cropped manhole cover area image for arc-shaped text to extract the ownership unit text information. Various implementation architectures can be used for text recognition of arc-shaped text. For example, a sequential architecture of "detection + correction + recognition" can be used, where the arc-shaped text area is first located and detected, then spatially transformed to correct the arc-shaped text into horizontal text, and finally character recognition is performed on the horizontal text; a "detection + end-to-end recognition" architecture can also be used, where after locating and detecting the arc-shaped text area, the arc-shaped text is directly recognized using an end-to-end recognition model without explicit geometric correction steps; a segmentation-based text recognition architecture can also be used, achieving arc-shaped text recognition through pixel-level text area segmentation and character classification. Those skilled in the art can choose any of the above architectures, or their improved or combined architectures, depending on the actual application scenario and accuracy requirements, as long as it can achieve text recognition and extraction of text content from arc-shaped text. The text information of the ownership unit is, for example, information containing the keyword of the ownership unit, such as "Beixin Infrastructure".

[0036] Subsequently, the system obtains the predicted results of the visual ownership identification attributes of each manhole cover instance output by the multi-label object detection model in step S2, and combines them with the ownership unit text information extracted in this step to determine the ownership unit of the target manhole cover instance. The visual ownership identification attributes and the ownership unit text information are independent of each other in terms of information source: the former comes from the recognition of the logo and surface pattern on the manhole cover surface by the multi-label object detection model in S2, and the latter comes from the reading of the text on the manhole cover surface by the text recognition model in this step. Because these two information sources are independent, when one recognition method is affected by wear, stains, or other interference and has a low confidence level, the other method may still provide accurate ownership clues. The system comprehensively compares and verifies both, forming a complementary relationship between the visual identification and text recognition results, and outputs the final ownership unit.

[0037] Thus, by extracting the text information of the ownership unit through text recognition for arc-shaped text, and by cross-verifying the visual ownership mark with text recognition, the shortcomings of single recognition methods in terms of insufficient accuracy when the mark is worn or stained are overcome, making the ownership judgment result more reliable.

[0038] S5: Based on the prediction results of the disease status attributes, the ownership unit, and the location information, generate a maintenance work order for the target manhole cover instance.

[0039] In this embodiment, the predicted damage status attribute output by the multi-label target detection model in step S2 is used to describe the specific damage type of the target manhole cover instance. The ownership unit determined in step S4 is used to determine the responsible department corresponding to the manhole cover. The location information obtained in step S1 is used to determine the geographical coordinates and road station number of the manhole cover. The system integrates the above information to generate a structured maintenance work order including fields such as damage status, ownership unit, and geographical location.

[0040] Taking a real-world identification as an example: The multi-label target detection model infers from a certain frame of road surface image and outputs a prediction result with the defect status attribute of "damaged" and a prediction result with the ownership unit determined in step S4 as "Beixin Infrastructure". The location information obtained in step S1 is 116.397128 degrees east longitude, 39.916527 degrees north latitude, and road chainage K12+340. The system automatically generates a maintenance work order with the following content: Work order number is WO-20260813-00342, generation time is 14:32:18 on August 13, 2026, location information is 116.397128 degrees east longitude, 39.916527 degrees north latitude and road chainage K12+340 meters, ownership unit is Beixin Infrastructure, geographical location is the east side of the intersection of a certain road and a certain street in a certain city, and the defect status is damaged.

[0041] It should be noted that the maintenance work order is not limited to including the condition of the manhole cover, the owner, and the geographical location. Other fields can be added according to the needs of the actual management system (for example, adding a field for attaching on-site photos to simultaneously upload the original real-shot images of the manhole cover defects as well as the recognition images with model detection bounding boxes, adding a field for historical maintenance records to trace the previous maintenance of the manhole cover, etc.). Those skilled in the art can make adaptive adjustments to the work order fields according to the management requirements of different owners or different cities.

[0042] Thus, by automatically integrating multi-tag recognition results, ownership units, and precise location information to generate maintenance work orders, an automated workflow from defect identification to work order output is achieved, avoiding the problems of multiple steps, slow workflow, and high error rates associated with manual form filling. Furthermore, the structured maintenance work orders provide a complete data foundation for subsequent accurate dispatching and closed-loop processing, effectively shortening the response time for defect handling.

[0043] As described in the background section, traditional manhole cover maintenance mainly relies on two methods: manual inspection and automatic recognition based on traditional image processing. The former has high safety risks, limited coverage, and inconsistent standards, while the latter is extremely sensitive to complex road conditions such as lighting, shadows, water accumulation, and obstruction, and has poor robustness. Although deep learning-based target detection methods have improved in recent years, they still have shortcomings such as the class mutual exclusion constraint of single-label classification leading to the inability to recognize multiple attributes in parallel, serious missed detection of small targets, difficulty in accurately reading curved ownership text, and the lack of automated closed-loop conversion of recognition results into maintenance work orders.

[0044] In view of this, this application achieves mobile, non-contact inspection of manhole covers on main roads by acquiring road surface image sequences collected by a mobile acquisition platform while traveling along the road. These images include multiple frames of road surface images of the same manhole cover from far to near, with each frame corresponding to the location information at the time of acquisition. This avoids the safety risks and coverage limitations of manual inspection. By inputting multiple frames of road surface images into a multi-label target detection model for inference, and independently outputting confidence scores for each attribute channel, with no constraints on the outputs of each attribute channel, this application achieves parallel output of multiple independent attributes such as damage status and visual ownership identification in a single forward inference. This avoids the efficiency bottleneck of traditional single-label models, which require multiple inferences or serialization of multiple models due to class mutual exclusion, and significantly reduces recognition time. By associating detection results belonging to the same manhole cover across frames based on location information, multiple frames of detection results of the same manhole cover collected from far to near can be... Accurate merging and consistency verification of the associated detection results effectively suppress motion blur and occasional false detections in single-frame detection, improving the reliability of the detection results. By performing text recognition on arc-shaped text in target manhole cover instances with defects, the problem of manhole cover ownership text being difficult to accurately read by conventional OCR along the arc-shaped edge of the manhole cover is solved. The ownership unit is comprehensively determined by combining the prediction results of the visual ownership identification attributes of each manhole cover instance. The cross-verification of visual identification and text recognition overcomes the shortcomings of single recognition methods in terms of insufficient accuracy when the identification is worn or dirty. By automatically generating maintenance work orders based on the prediction results of the defect status attributes, ownership unit, and location information, the automated flow from defect identification to work order output is realized, providing a structured data foundation for subsequent accurate dispatching and closed-loop treatment, and improving the overall intelligence and precision of urban manhole cover defect management.

[0045] Optionally, the operation of acquiring a sequence of road surface images collected by the mobile acquisition platform while it travels along the road includes: acquiring the current location information of the mobile acquisition platform in real time, and calculating the distance between the mobile acquisition platform and the nearest manhole cover based on the current location information and a pre-loaded list of manhole cover coordinates; when the distance is greater than a preset distance threshold, acquiring road surface images at a first frame rate; when the distance is less than or equal to the preset distance threshold, acquiring road surface images at a second frame rate until the mobile acquisition platform passes the nearest manhole cover; the second frame rate is higher than the first frame rate; and storing each acquired frame of road surface image in association with the location information at the time of acquisition as the road surface image sequence.

[0046] In this embodiment, the system achieves adaptive acquisition of road surface image sequences through the following steps: First, the system acquires the current location information of the mobile data acquisition platform in real time. The mobile data acquisition platform is equipped with a GPS / INS combined positioning module, which calculates the vehicle's latitude and longitude coordinates, heading angle, and speed information in real time at a high output frequency (e.g., 100Hz). Edge computing devices then read this location information in real time through a data interface.

[0047] Meanwhile, the system pre-loads a list of manhole cover coordinates within the target area. This list records the precise geographical coordinates (e.g., latitude and longitude coordinates) of all manhole covers within the target area, as well as basic information such as the road station number and ownership unit corresponding to each manhole cover. The list of manhole cover coordinates can be imported from the maintenance system or GIS database before the inspection task begins and stored in the local storage of the edge computing device.

[0048] The system calculates the spatial distance between the mobile data acquisition platform and the nearest manhole cover in real time based on the current location information and the coordinates of each manhole cover in the manhole cover coordinate list. Specifically, the system iterates through the locations of all uninspected manhole covers in the manhole cover coordinate list, calculates the Euclidean distance or spherical distance between each manhole cover and the vehicle's current location, and then takes the minimum value as the distance to the nearest manhole cover.

[0049] The system dynamically adjusts the image acquisition frame rate based on the calculated distance and the vehicle's current speed. When the distance exceeds a preset threshold (e.g., 20m), it indicates that the vehicle is still far from the manhole cover. In this case, the system acquires road surface images at the first frame rate (e.g., 5fps) to conserve computing resources and storage space for the edge computing device. The first frame rate can be set to a lower frame rate, sufficient to maintain continuous acquisition of road surface images at normal vehicle speeds.

[0050] When the distance is less than or equal to a preset distance threshold (e.g., 20m), and the vehicle's current speed is greater than a preset speed threshold (used to exclude stationary or extremely low-speed congested states), it indicates that the vehicle has approached the manhole cover and is in an effective driving state. At this time, the system automatically switches to a second frame rate (e.g., 30fps) to acquire road surface images. The second frame rate is higher than the first frame rate, ensuring that multiple frames of images of the manhole cover are captured at a higher temporal resolution as the vehicle passes the manhole cover from a distance, thereby increasing the number of image frames available for subsequent identification. The system continues to acquire images at the second frame rate until the mobile acquisition platform passes the nearest manhole cover. The criteria for determining whether a manhole cover has been passed can be based on the distance between the vehicle's positioning information and the manhole cover position increasing from a small distance to a larger distance, and then increasing again after the vehicle passes the manhole cover, or it can be based on the distance between the vehicle's positioning information and the manhole cover position exceeding a preset passing threshold (e.g., 5m). If the vehicle speed drops below the preset speed threshold during the second frame rate acquisition period, the system can automatically revert to the first frame rate or pause acquisition until the vehicle resumes effective driving, thereby avoiding a large accumulation of redundant data.

[0051] Finally, the system associates and stores each captured road surface image frame with its location information at the time of capture. For example, each road surface image frame and its corresponding location information are associated and saved as a record in the storage medium of the edge computing device. Thus, all the captured road surface image frames form a road surface image sequence according to the capture time, and each frame carries its location information at the time of capture, which facilitates cross-frame association based on location information in subsequent steps.

[0052] Therefore, by calculating the distance and driving status of the vehicle to the nearest manhole cover in real time, a low frame rate is used to save computing power when the vehicle is at a long distance or stationary, and a high frame rate is automatically switched to intensively capture the same manhole cover when the vehicle is effectively moving at close range. This achieves dynamic adjustment of the frame rate according to changes in spatial location and motion state, ensuring that multiple frames of images of the same manhole cover from far to near can be completely captured. By associating and storing each frame of road image with the positioning information at the time of acquisition, a precise data foundation is provided for subsequent cross-frame association and spatial positioning. Thus, while ensuring the complete acquisition of key targets, the computing power and storage resource allocation of the edge computing device are also maximized and optimized.

[0053] Optionally, the multi-label target detection model is an improved YOLO model. The improvements to the baseline YOLO model include: adding a P2 feature layer to the feature pyramid of the neck network, where the output feature map resolution of the P2 feature layer is higher than the feature map resolution output by the highest spatial resolution feature layer of the baseline YOLO model; embedding channel attention modules and spatial attention modules into at least a portion of the C3k2 modules in the backbone network and the neck network; and replacing at least a portion of the stride convolutions in the downsampling path of the neck network with ADown modules, where the ADown modules are used to divide the input features into two subsets in the channel dimension, process them separately, and then... The processing results of the two subsets are concatenated along the channel dimension. An improved head network is adopted, which includes a multi-scale fusion module, a P2 detection head, a P3 detection head, a P4 detection head, and a P5 detection head. The multi-scale fusion module is used to: map the feature maps of the four different scales output by the neck network to the spatial resolution of the target scale for each detection head, thereby obtaining four sets of aligned features; concatenate the four sets of aligned features and perform convolution and normalization to generate a set of spatial attention weight matrices; and perform weighted fusion of the four sets of aligned features according to the spatial attention weight matrices to obtain the fused feature map of the target scale and input it into the detection head of the corresponding scale.

[0054] In this embodiment, the multi-label object detection model adopts an improved YOLO model. YOLO is an object detection model with single-stage detection as its core, achieving high detection accuracy while maintaining real-time inference speed. The baseline YOLO model follows the standard three-component architecture of the YOLO series, including a backbone network, a neck network, and a head network.

[0055] Specifically, YOLO's backbone network employs a cross-stage local aggregation network structure, extracting multi-scale features from the input image through progressive downsampling. The backbone network typically comprises five stages, each progressively reducing the feature map resolution and increasing the number of channels through convolution and aggregation operations, outputting feature maps at multiple scales. YOLO integrates the C3k2 module into the backbone network, which enhances feature representation capabilities through multi-branch convolution and feature aggregation.

[0056] YOLO's neck network employs a feature pyramid network structure, fusing multi-scale feature maps output from the backbone network. The baseline YOLO model typically includes three feature layers in the neck network: P3, P4, and P5, corresponding to feature maps with downsampling ratios of 8, 16, and 32, respectively. Layer P3 has the highest resolution, suitable for detecting small targets; layer P5 has the lowest resolution but the largest receptive field, suitable for detecting large targets. The neck network achieves full fusion of multi-scale features through top-down and bottom-up paths.

[0057] The YOLO head network contains detection heads corresponding to the feature layers. In the baseline design, each detection head includes a conventional single-label classification branch and a bounding box regression branch, used to output the target's class probability and bounding box location. The above description of the baseline YOLO model is only used to illustrate the model background on which this embodiment is based, so as to more clearly understand the improvements of this invention in multi-label parallel detection, etc. This invention is not limited to the specific architecture described above, and other similar target detection models in related technologies can also be improved accordingly based on the concept of this invention.

[0058] This embodiment makes the following four structural improvements based on the baseline YOLO model, combined with... Figure 4 As shown, it specifically includes: Firstly, a P2 feature layer is added to the feature pyramid of the neck network. The baseline YOLO model's neck network contains three feature layers: P3, P4, and P5. However, small targets such as damaged areas of manhole covers and logos in road images often account for less than 2% of the total area of ​​the image. These small targets occupy only a few pixels in the P3 feature layer (downsampling factor of 8), and their detailed information is easily lost after multiple downsampling steps. Therefore, this embodiment adds a high-resolution feature layer, the P2 feature layer, to the neck network. The output feature map resolution of the P2 feature layer is higher than that of the smallest feature layer (P3 feature layer) in the baseline YOLO model. For example, with an input image resolution of 1280×1280, the output feature map resolution of the P3 feature layer is 160×160, while the output feature map resolution of the P2 feature layer reaches 320×320, with a grid density four times that of the P3 layer. This high-resolution feature map provides more refined spatial location clues for small target detection, fundamentally solving the problem of small targets being lost due to progressive downsampling.

[0059] Secondly, channel attention modules and spatial attention modules are embedded in at least a portion of the C3k2 modules in the backbone and neck networks. While high-resolution features bring detailed information, they also introduce more background noise. Furthermore, deep features face the problem of weakening key semantics when propagated across layers. Therefore, this embodiment upgrades at least a portion of the standard C3k2 modules in the backbone and neck networks to C3k2-CBAM modules. Figure 5 A schematic diagram of the C3k2 module, which embeds a channel attention module and a spatial attention module, is shown. This module, based on the original C3k2 structure, sequentially connects the channel attention sub-module and the spatial attention sub-module to form a complete attention feature enhancement unit.

[0060] In the channel attention submodule, the input features (C, H, and W represent the number of channels, height, and width of the input feature map, respectively.) First, the channel-weighted feature F' is obtained through the channel attention module: ; in, This indicates element-wise multiplication. The channel attention weight is calculated using the following formula: ; in, MLP() represents the Sigmoid activation function, and MLP() represents a multilayer perceptron, which is a fully connected neural network with shared weights. This represents the global average pooling operation. This represents the global max pooling operation. This channel attention mechanism extracts the global spatial description of the feature map through global average pooling and global max pooling, respectively. After transformation by a shared multilayer perceptron, the values ​​are added together, and then activated by a sigmoid function to generate channel attention weights. This can adaptively focus on the channel dimensions that are more important for detecting minor defects on the manhole cover surface, while suppressing the responses of irrelevant channels.

[0061] Subsequently, the channel-weighted feature F' enters the spatial attention submodule to obtain the final enhanced feature F'': ; Among them, spatial attention weights Features generated by concatenating features after max and average pooling along the channel dimension, followed by 7×7 convolution: ; in, Represents the Sigmoid activation function. This indicates a convolution operation with a kernel size of 7×7, used to perform spatial dimension feature fusion on the concatenated feature maps, and outputs a spatial attention map with 1 channel.

[0062] This spatial attention mechanism performs global average pooling and global max pooling along the channel dimension, concatenates the resulting two 2D spatial attention maps along the channel dimension, fuses them through a 7×7 convolution, and then generates spatial attention weights through sigmoid activation. This adaptively focuses on spatial locations more important for detecting minute defects, enhancing the response to key areas such as manhole cover surface damage and logos, while suppressing the response to background areas. Through the cascaded enhancement of channel attention and spatial attention, the C3k2-CBAM module effectively mitigates the risks of false alarms and memory overflow while improving the accuracy of small target detection.

[0063] Thirdly, at least some stride convolutions in the downsampling path of the neck network are replaced with ADown modules. In the neck network, downsampling operations are required between feature maps of different scales to achieve multi-scale fusion. The baseline YOLO model typically uses ordinary convolutions with a stride of 2 for downsampling. However, ordinary convolutions process all channels simultaneously during downsampling, causing the loss of detailed information such as edges and textures during cross-layer propagation. Therefore, this embodiment replaces at least some stride convolutions in the downsampling path of the neck network with ADown modules. Figure 6 The diagram shows the structure of the ADown module. The ADown module employs a strategy of channel decoupling and dual-branch parallelism to avoid the loss of edge and texture details in high-resolution features when propagating to deeper layers.

[0064] Specifically, the ADown module first divides the input feature X into two subsets along the channel dimension. and : ; The left branch uses 3×3 convolution to perform spatial downsampling and extract local correlations: ; The right branch first preserves significant high-frequency features through max pooling, and then performs channel alignment through 1×1 convolution: ; Finally, the two branches of features are concatenated along the channel dimension to output: ; in, and The spatial dimensions are all half of the input feature X, and the number of channels is also half of the input feature X; the output is... and The result after concatenation along the channel dimension has a spatial size that is half that of the input feature X, and the number of channels is restored to the original number of channels of the input feature X.

[0065] Through the above-mentioned dual-branch parallel processing, the ADown module achieves spatial downsampling while preserving high-frequency detail information through the max pooling branch and extracting local correlations through the convolution branch, enabling the key information of small targets to participate in the computation of subsequent deeper networks without loss.

[0066] Fourthly, an improved head network is adopted. This improved head network includes a multi-scale fusion module, a P2 detection head, a P3 detection head, a P4 detection head, and a P5 detection head. Figure 4 This diagram illustrates the overall structure of a multi-label object detection model, showing the process of adaptive spatial weighted fusion of the feature maps (P2, P3, P4, and P5) after processing by the neck network and then fed into the multi-scale fusion module in the detection head. The baseline YOLO model's detection head directly detects the feature maps at each scale output by the neck network without deep fusion of features at different scales. This embodiment adds a multi-scale fusion module to the detection head, performing adaptive spatial weighted fusion of the feature maps at the four scales (P2, P3, P4, and P5), and then feeding them into the corresponding detection heads for detection, thus breaking down the semantic barriers between feature layers at different scales.

[0067] Specifically, in combination Figure 7 As shown, when generating fusion features at the target scale Pi, the multi-scale fusion module first combines features from all scales. After passing through the scale alignment module, the spatial resolution is uniformly mapped to the target scale Pi, resulting in alignment features. Subsequently, the four sets of aligned features are concatenated, and the weights are used to generate a normalized spatial attention weight matrix for the network output. : ; Here, Softmax() represents the spatially normalized exponential activation function. Finally, the scale-aligned features are multiplied element-wise by their corresponding weight matrices. Then, perform element-wise addition (⊕) to obtain the final fused output. : ; in, This represents the fusion feature at the target scale Pi. Through the aforementioned rigorous, pixel-by-pi adaptive weighted fusion, extremely small targets can simultaneously utilize the fine positional information provided by shallow features and the rich semantic information provided by deep features, while significantly suppressing complex background noise, ensuring that the final regressed bounding box coordinates are extremely accurate. The fused feature map is then input into a detection head at the corresponding scale for detection.

[0068] The following section describes the complete data processing procedure for detecting input road surface images using the improved YOLO model in the application phase.

[0069] In the application phase, the multiple road surface images acquired in S1 are first input into the improved YOLO model. Each road surface image undergoes progressive processing by the backbone network: the input image sequentially passes through multiple stages of convolution and downsampling to gradually extract multi-scale features. At least some C3k2 module locations in the backbone and neck network, the replaced C3k2-CBAM attention modules process the feature maps, enhancing the response to minor defects on the manhole cover surface and suppressing background noise through cascaded enhancement of channel attention and spatial attention.

[0070] The multi-scale feature maps output from the backbone network then enter the neck network. In the neck network, the feature maps at four scales (P2, P3, P4, and P5) are fused using a feature pyramid structure. During the fusion process, at least some stride convolutions in the downsampling path are replaced by the ADown module to ensure that edge and texture details in high-resolution features are not weakened during cross-layer propagation.

[0071] The four scale feature maps output by the neck network are then fed into the multi-scale fusion module in the head network. The multi-scale fusion module performs the following fusion process sequentially for the target scales corresponding to the four detection heads (P2, P3, P4, and P5): Taking the P2 detection head as an example, for the P2 scale, the module maps the feature maps of the four scales output by the neck network to the P2 scale spatial resolution, obtaining four sets of aligned features; these four sets of aligned features are concatenated and then processed by convolution and normalization to generate a spatial attention weight matrix; based on this weight matrix, the four sets of aligned features are weighted and fused pixel-by-pixel to obtain the fused feature map at the P2 scale, which is then input into the P2 detection head. The P2 detection head detects this fused feature map and outputs the bounding boxes of each manhole cover instance detected at this scale, as well as the prediction results of multiple independent attributes. Similarly, for the P3, P4, and P5 detection heads, the multi-scale fusion module repeats the above process, mapping the feature maps of the four scales to the corresponding target scales, generating fused feature maps for each scale, and inputting them into the P3, P4, and P5 detection heads respectively. Each detection head independently outputs the detection results at its own scale, namely the bounding boxes of each manhole cover instance detected on the feature map of that scale, as well as the prediction results of multiple independent attributes. Since the feature maps at different scales have different receptive fields and resolutions, the P2 detection head focuses on the fine localization of small targets, while the P5 detection head is responsible for the recall of large targets. The four detection heads complement each other at different scales.

[0072] Finally, the system integrates the outputs of the four detection heads. Specifically, since the same manhole cover instance may be detected on feature maps at different scales, there is overlap between the detection results output by the four detection heads. For each scale, the detection head outputs all detected manhole cover instances at that scale, along with their corresponding bounding boxes and independent attribute predictions. The system collects all outputs from the four detection heads to form a candidate detection result set, which contains all candidate detection boxes from the four scales P2, P3, P4, and P5.

[0073] The system applies a non-maximum suppression algorithm (NMS) to the candidate detection result set to remove redundant detection boxes. The processing logic of the NMS algorithm is as follows: First, for each candidate detection box, extract its output multiple independent attribute confidence scores, and take the maximum confidence score as the comprehensive target confidence score of the candidate detection box. Then, sort all candidate detection boxes according to their confidence scores from high to low. The confidence score reflects the degree of confidence that the detection box contains the manhole cover target. Second, select the detection box with the highest confidence score as the retained box, add it to the final detection result list, and calculate the intersection-union ratio (IUR) of the detection box with all other detection boxes in the candidate set. When the IUR of a detection box with the retained boxes is greater than a preset threshold, it indicates that the two detection boxes are highly overlapping in spatial location and have low confidence. The system determines the detection box as redundant and removes it from the candidate set. When the IUR is less than or equal to the preset threshold, it indicates that the two detection boxes correspond to different manhole cover instances, and the system retains the detection box. The system iterates through all detection boxes in the candidate set according to the above steps until the candidate set is empty. After nonmaximum suppression processing, repeated detections of the same manhole cover instance at different scales are merged into a single detection result.

[0074] After non-maximum suppression processing, the system outputs the final detection results for each manhole cover instance in each frame of the road image. Each final detection result corresponds to a manhole cover instance, including the bounding box of the manhole cover instance and the prediction results of multiple independent attributes. When there are multiple spatially separated manhole cover targets in the road image, the model can output multiple detection results, each independently identifying the corresponding manhole cover instance and its attributes. The prediction results of multiple independent attributes of the same manhole cover instance (including damage status attribute, logo attribute, and pattern attribute) are output in parallel in this final detection result, and the attributes do not interfere with each other.

[0075] In this embodiment, to enable the improved YOLO model to accurately identify manhole cover instances and their multiple independent attributes in road surface images, the model needs to be sufficiently trained. The training process includes three main steps: preparation of training samples, design of the loss function, and optimization of model parameters.

[0076] (a) Preparation of training samples First, we prepare training samples and build a high-quality multi-label manhole cover image database.

[0077] Extensive on-site photography was conducted across various administrative regions in multiple cities using image acquisition equipment. The primary subjects were the motor vehicle lanes of main urban roads, while also considering adjacent non-motorized vehicle lanes, sidewalks, and roads within residential areas. This ensured the model could fully adapt to the real-world application scenario of inspection vehicles "primarily targeting motor vehicle lanes while also considering adjacent non-motorized vehicle lanes," and various road environments. Shooting spanned daytime, dusk, nighttime, and dawn, covering typical weather conditions such as sunny, cloudy, rainy, and foggy. Perspectives included overhead, oblique, and slight distortion views, thus covering as many real-world inspection conditions as possible in terms of lighting, weather, occlusion, and scene diversity. Approximately 10,000 original images were obtained, with each image potentially containing one or more manhole cover targets.

[0078] Based on the acquired original images, the original images are first manually cleaned to remove invalid samples that are severely blurred, overexposed, completely obscured by the target, or do not contain manhole covers.

[0079] Subsequently, multi-label annotation was performed on the retained valid images. Since each manhole cover target needs to be labeled with independent attributes across three dimensions—damage status, logo, and pattern—while the traditional YOLO format only supports single-category annotation and cannot meet the needs of multi-attribute annotation, this embodiment extends the annotation format. The traditional YOLO annotation row format is “cls xy wh”, where cls represents the category index, and x, y, w, and h are the center coordinates and width and height of the bounding box, respectively. For example, the annotation row “0 0.45 0.52 0.12 0.10” indicates that the category index corresponding to this bounding box is 0, the corresponding manhole cover status is “good” (the manhole cover is in good condition), the center coordinates are (0.45, 0.52), the width is 0.12, and the height is 0.10. x, y, w, and h have all been normalized, with values ​​between 0 and 1.

[0080] This embodiment extends the format to a space-separated numerical index format "cls_1 cls_2 cls_3 ... xy wh", where the last four fields are the center coordinates and width / height of the bounding box, and the preceding fields can contain any number of category index fields to describe multiple independent attributes of the same target. During annotation, a rectangular bounding box is assigned to each manhole cover target, and a preset numerical mapping table maps the attributes of the three dimensions to globally unique numerical indices. These three dimensions include: damage status attributes (normal, damaged, missing, displaced, settlement difference, damage around the manhole or frame, loosening, etc., totaling 7 categories), logo attributes (electricity, communication, water supply, drainage, gas, heating, etc., totaling 6 categories), and pattern attributes (square pattern, radial pattern, circular pattern, grid pattern, anti-slip raised dot pattern, double-layer pattern, etc., totaling 6 categories). To ensure the uniformity of tensor dimensions during data loading, for targets with insufficient labels, "-1" is introduced as a placeholder for padding at the end, with the maximum number of attributes per frame as the upper limit. Meanwhile, to ensure absolute consistency in feature alignment during loss calculation in the deep learning data loading layer, the system introduces a multi-label forced alignment mechanism: regardless of the order in which labels are selected in the front-end annotation interface, the underlying processing logic forces the effective category index values ​​to be arranged in ascending order, strictly guaranteeing that when writing to the file, the label index value that comes first is always less than the label index value that comes later. For example, in the format “0 914 -1 0.45 0.52 0.12 0.10”, “0”, “9”, and “14” are arranged in strict ascending order, representing the numerical indices of the corresponding disease state, Logo attribute, and pattern type, respectively. “-1” is a placeholder, and the last four digits represent the center coordinates and width / height of the bounding box. Similarly, “0 15 -1 -1 0.45 0.52 0.12 0.10” indicates that the target has only two valid attribute labels, with the remaining digits padded with “-1”. To ensure consistency and accuracy in labeling, all labelers must undergo training on standardized multi-attribute judgment rules and pass cross-validation and expert review mechanisms. Ambiguous samples are subject to multiple rounds of review and labeling.

[0081] In the data loading layer, since the number of multi-labels for different targets may vary, this embodiment designs a dynamic multi-class slicing algorithm based on reverse indexing: when the data loader reads each label row, it takes the last four values ​​as the bounding box coordinate field and uses all fields before the last four fields as the category vector; for category vectors that are not long enough, a preset padding value (e.g., -1) is used to pad the tail, so that the category vectors of each target in the same batch are of consistent length, and the number of categories is capped at the maximum number of labels per box of the project. This structure fundamentally avoids subsequent NMS conflicts caused by multi-attribute box labeling in traditional solutions.

[0082] After completing the basic annotations, data augmentation was used to further expand the sample size and enrich the data distribution, increasing the number of annotated images from approximately 10,000 to approximately 30,000. Augmentation strategies included: random angle rotation (angle controlled within...). To maintain label validity, random scaling and cropping were implemented, along with random perturbations of brightness / contrast / saturation, hue shifting, Gaussian noise injection, and motion blur and rain / fog effect simulation. It should be noted that because the circular logo on the manhole cover is used as one of the criteria for determining ownership, and many organizations' logos are not symmetrical horizontally or vertically, horizontal and vertical flipping is prohibited during the enhancement process to avoid compromising the logo's recognizability.

[0083] Subsequently, the enhanced complete dataset was stratified and sampled according to scene and attribute labels, divided into training, validation, and test sets in a ratio of approximately 7:2:1, with each subset maintaining a roughly balanced distribution in terms of defect type, ownership unit, pattern type, and road scene. The resulting manhole cover database covers typical urban lighting variations, multiple road paving types, common defects, major ownership units, and typical surface patterns, providing a robust, widely distributed, and attribute-complete training foundation for subsequent multi-label detection models.

[0084] (II) Design of the loss function After preparing the training samples, they are input into the improved YOLO model for forward propagation. Following the processing flow of the aforementioned application phase, the model sequentially undergoes feature extraction from the backbone network, multi-scale feature fusion from the neck network, and multi-scale fusion and detection output from the detection head to obtain the prediction results. The prediction results include the bounding boxes of each manhole cover instance, as well as the category prediction results and confidence scores for multiple independent attributes.

[0085] Based on this, the total loss between the model's predictions and the ground truth labels is calculated. The number of output channels of the detection head's classification branch is redefined as the total number of categories for all independent attributes. Let the attribute to be detected contain... Such state, Logo and If a pattern is used, the number of output channels for the classification branch is set as follows: ; In this embodiment, when , , Sometimes, This achieves a flattened and unified physical dimension across all attribute channels. At this point, for the input target, the original output of the detection head classification branch can be represented as a... dimensional vector At the same time, the original Softmax activation function in the classification branch output layer is discarded and replaced with the Sigmoid activation function. The Softmax activation function is defined as: ; in, The original output value (logit) of the i-th channel of the classification branch output layer, i=1,2,⋯, j is the summation index variable. ; This represents the total number of output channels for the classification branch. This function forces the sum of all channel outputs to be 1, i.e. This leads to mutual exclusion between categories, where an increase in the confidence level of one channel inevitably suppresses the confidence levels of other channels. In contrast, this embodiment uses a Sigmoid function that acts independently on each channel: ; The Sigmoid function calculates the activation value independently for each classification channel, and its original output for the i-th channel is... Furthermore, there is no constraint that the sum of the channels is one, so that the scores of each channel do not suppress each other, and the same target can output high confidence on multiple attribute channels at the same time.

[0086] The corresponding multi-label classification loss uses binary cross entropy, and the loss for a training sample is calculated as follows: ; in, This is the multi-label classification loss value for the training sample. For the first The actual label of each attribute channel (1 if it exists, 0 if it does not exist). This represents the confidence score output by the Sigmoid function. This loss function drives each channel to independently learn the binary classification decision boundary for its corresponding attribute, thus enabling joint detection of multiple labeled attributes in a single forward inference. Simultaneously, some labels are inherently mutually exclusive, such as "damaged manhole cover" and "intact manhole cover." During training, the loss function is retained using the full Sigmoid function plus BCE, allowing the model to learn independent binary classification boundaries for each channel. During inference, mutually exclusive labels are judged with the highest confidence score, forcing physical mutual exclusion.

[0087] The regression loss employs methods such as intersection-over-union loss and distribution loss to constrain the positional accuracy of the bounding box. The total loss is a weighted sum of the classification and regression losses described above.

[0088] (III) Optimization of Model Parameters After calculating the total loss, the backpropagation algorithm is used to calculate the gradient of each loss term with respect to the model parameters, and the optimizer is used to update the model parameters to reduce the total loss. The optimizer can employ mainstream optimization algorithms such as stochastic gradient descent (SGD) or AdamW, and the learning rate can use a cosine annealing decay strategy, for example, an initial learning rate of 0.01 for SGD followed by cosine annealing. During training, training can be terminated early based on performance on the validation set to prevent overfitting. After iterative optimization over multiple training epochs, such as 300 epochs, the model gradually converges, and the prediction accuracy gradually improves.

[0089] Thus, through the complete training process described above, including data collection, cleaning and multi-label annotation, data augmentation, dataset partitioning, and multi-label loss function design, the improved YOLO model can learn the ability to detect manhole cover instances and identify multiple independent attributes from labeled road surface image samples. After training, the model has the ability to simultaneously output the bounding boxes of each manhole cover instance and prediction results of multiple independent attributes such as damage status, logo, and pattern upon inputting a road surface image, providing a reliable model foundation for the automated identification of manhole cover damage and ownership.

[0090] Optionally, the operation of performing text recognition for arc-shaped text in the image region corresponding to the bounding box of the target manhole cover instance and extracting ownership unit text information includes: cropping the manhole cover region image from the corresponding road surface image based on the bounding box of the target manhole cover instance; locating the arc-shaped text region in the manhole cover region image using a Fourier contour embedding network and outputting a contour that fits the boundary of the arc-shaped text; extracting the corresponding arc-shaped text region image from the manhole cover region image based on the contour, and performing spatial geometric transformation on the arc-shaped text region image to flatten the arc-shaped text in the arc-shaped text region image into horizontal text, generating a horizontal text image; recognizing the horizontal text image using a text recognition model and outputting the text character sequence and confidence score as the ownership unit text information.

[0091] In this embodiment, after identifying the target manhole cover instance with defects in step S3, the system crops the manhole cover area image from the corresponding road surface image based on the bounding box of the target manhole cover instance. This manhole cover area image serves as the input data source for subsequent arc-shaped character recognition and ownership determination. Figure 8 The complete flowchart of the collaborative identification of manhole cover defects and ownership is shown, in which the arc-shaped text recognition link is the core component of the ownership unit determination process.

[0092] For the cropped manhole cover area image, the system first performs detection and localization of arc-shaped text. Specifically, since the ownership unit text on the manhole cover surface is usually arranged in a fan shape or ring shape along the edge of the manhole cover, conventional horizontal text detection methods are difficult to accurately fit its curvature. Therefore, this embodiment uses a Fourier Contour Embedding Network (FCENet) to locate the arc-shaped text region in the manhole cover area image. FCENet transforms text contours of arbitrary complex shapes (including rotation, tilt, and severe curvature) in the spatial domain into fixed-length coefficient vectors in the frequency domain for compact encoding. Specifically, the closed-loop contour of the text is regarded as a parameterized closed-loop curve in the complex domain. The curve is composed of a set of Fourier coefficients. Approximate expression using truncated Fourier series: ; in, The plural form of the closed-loop outline of the text, its actual part With the imaginary part These correspond to the horizontal and vertical coordinates of the contour points in the image space, respectively; t is the normalized arc length parameter, with a value range of [0,1], representing the relative circumference progress along the closed-loop contour; K is the truncation order of the Fourier series, used to filter out high-frequency noise and ensure that the length of the output feature vector is fixed, with a value range of [3,10]. The Fourier coefficients of the text contour are the core target variables that the model network directly regresses and predicts during the training phase. is the complex exponential basis function of the Fourier transform.

[0093] During the actual inference phase, the Fourier coefficient sequence output by the FCENet network is obtained. Then, the inverse Fourier transform is applied to N uniformly discrete sampling points. ( The coordinates of the discrete polygons that reconstruct the text outline are calculated separately on each side: ; in, For the model at discrete sampling points The predicted complex coordinates of the reconstructed text outline; The actual output of the network is the sequence of predicted Fourier coefficients; N is the total number of uniformly sampled points during contour reconstruction, which determines the number of vertices of the final generated polygon. The discrete sampling parameters, where n is an integer step size index from 0 to N-1, are used to divide the continuous contour curve into N discrete vertices. This is achieved by extracting... By sequentially connecting the real and imaginary parts, a closed polygon that tightly surrounds arbitrarily deformed text can be obtained, and the output contour that fits the boundary of the curved text can be obtained.

[0094] Subsequently, based on the contour output by FCENet, the system extracts the corresponding arc-shaped text region image from the manhole cover area image. Specifically, since the text in this arc-shaped text region image is arranged along an arc, directly feeding it into a conventional text recognition model would result in a significant drop in recognition accuracy. Therefore, this embodiment performs a spatial geometric transformation on the arc-shaped text region image, flattening the arc-shaped text distribution into horizontal text, generating a horizontal text image. The specific implementation of this spatial geometric transformation will be described in detail in subsequent embodiments.

[0095] Finally, the horizontal text image is recognized using a text recognition model. In this embodiment, the text recognition model adopts the SVTR (Single Visual Transformer) pure visual model. SVTR abandons recursive sequence modeling and explicit geometric correction, and directly extracts features from the original text image through alternating stacked local attention modules and global attention modules. The local attention module divides the input feature map into multiple non-overlapping local windows, and extracts fine-grained visual features of characters through multi-head attention within the window. The calculation method is as follows: ; Where X is the input feature map sequence of this module; X' is the output feature map after processing by this module; This is a local feature blending operation; This is a layer normalization operation used to stabilize training; This is a window-based local multi-head self-attention function used to extract local stroke details of characters; This is a residual connection used to preserve the original image information and prevent gradient vanishing.

[0096] Then, the feature map is first spatially downsampled using a global attention module. Global multi-head attention is then calculated on the downsampled feature map to obtain long-distance inter-character dependencies. Finally, it is upsampled back to its original size. The calculation method is as follows: ; Where X' is the input feature map of this module (i.e., the output X' of the previous local module); X'' is the final output feature map after processing by this module; This is a global semantic fusion operation; This is a spatial downsampling operation used to compress spatial resolution and reduce global computational complexity. This is a global multi-head self-attention function used to capture context dependencies across character and long text sequences; For upsampling operations, the feature map is restored to the same original size as X'; This is a residual connection.

[0097] By employing a multi-layered stacking of alternating local and global attention, the model can simultaneously capture character stroke details and overall semantics, enabling high-precision recognition of text regions of arbitrary direction and shape. It exhibits excellent robustness in recognizing complex structures such as Chinese characters. The SVTR model outputs a text character sequence and its corresponding confidence score, serving as the text information for the ownership unit.

[0098] Thus, by using FCENet Fourier contour detection to accurately locate the ownership text arranged in a fan shape along the edge of the manhole cover, by using spatial geometric transformation to non-destructively flatten the arc-shaped text into horizontal text, and by using SVTR visual Transformer to complete highly robust character recognition, the three together constitute a complete recognition path for the arc-shaped text on the manhole cover. This solves the problem that the ownership text arranged in an arc along the edge of the manhole cover is difficult to be accurately read by conventional OCR, and provides a reliable text information basis for the subsequent comprehensive determination of ownership units.

[0099] Optionally, the operation of performing spatial geometric transformation on the arc-shaped text region image to flatten the arc-shaped text in the arc-shaped text region image into horizontal text and generate a horizontal text image includes: performing principal component analysis on the text pixel coordinates in the arc-shaped text region image to extract the principal direction vector, and rotating and straightening the arc-shaped text region image according to the principal direction vector; dividing the text pixels in the straightened arc-shaped text region image into multiple bins along the horizontal direction, and extracting the upper boundary extreme point, lower boundary extreme point, and center point in each bin; performing quadratic polynomial fitting on the upper boundary extreme point, the lower boundary extreme point, and the center point respectively to obtain the upper edge fitting curve, the lower edge fitting curve, and the center skeleton fitting curve; calculating the tangent slope at multiple sampling points of the center skeleton fitting curve, and determining the normal direction at each sampling point according to the tangent slope; extracting pixels along the normal direction between the upper edge fitting curve and the lower edge fitting curve, and linearly mapping the extracted pixels to a standard canvas of a preset height to generate the corresponding horizontal text image.

[0100] In this embodiment, an adaptive trilinear fitting method is proposed for the spatial geometric transformation of arc-shaped text region images. This method completely abandons the dependence of traditional thin-plate spline spatial transformation algorithms on the construction of a global reference center and polar coordinate system, and achieves adaptive correction from local features. Figure 9 As shown, it specifically includes: First, after acquiring the image of the arc-shaped text region, principal component analysis (PCA) is performed on the text pixel coordinates to extract the principal direction vector of the current arc-shaped text region. Based on this, the affine transformation matrix is ​​calculated, and the image is rotated and aligned to reduce the curvature extrema of subsequent fitting.

[0101] Secondly, the aligned text pixel coordinates are binned horizontally to extract the upper boundary extreme points, lower boundary extreme points, and center point within each bin. A quadratic polynomial is then used to independently fit the upper edge line of each text region. Bottom edge line and the central skeleton line A "three-line" model is constructed that closely reflects the actual physical deformation of the text. Let the pixel coordinates extracted by binning in the local coordinate system be... Its general polynomial fitting equation is: ; in, , , These are the polynomial fitting coefficients. Three sets of heterogeneous coefficients applicable to the upper edge line, lower edge line, and center skeleton line are calculated using the least squares method. Taking the center skeleton line as an example, its specific fitting coefficient is denoted as... , , This allows for a precise depiction of the text's true curved topology.

[0102] Furthermore, to achieve lossless straightening of text strokes and avoid vertical shear caused by conventional vertical sampling, this embodiment strictly samples along the normal orthogonal vector of the central skeleton line. In the equation of the central skeleton line... any x-coordinate At that point, calculate the slope (first derivative) k of its tangent line: ; Based on the geometric orthogonality theorem, the slope of the normal direction for orthogonal sampling along the central skeleton line is obtained. for: ; Finally, using the slope of the normal direction mentioned above This method guides the inverse remapping of pixels while eliminating the "near-to-far" distortion caused by 3D perspective projection. Traditional methods often result in inconsistent character scales after unfolding when dealing with such distortions. This embodiment accurately captures local scale changes caused by perspective by constructing independent upper and lower edge polynomials. During the remapping stage, the algorithm dynamically calculates the upper edge line along the normal line determined by the slope. With the bottom edge line The algorithm first extracts the target pixels between the upper and lower boundaries and then forces a linear mapping to a standard canvas size with a fixed height (e.g., 64 pixels). Finally, using bilinear interpolation, the irregularly sampled arc-shaped pixel stream is losslessly reconstructed into a standard rectangular text image. This mechanism achieves pixel-level adaptive height normalization, forcibly aligning the upper and lower boundaries of all characters, resulting in a highly consistent text scale in the final output.

[0103] Thus, principal component analysis is used to initially align the text region; a three-line polynomial fitting method accurately captures the true curved topology of the arc-shaped text; pixel sampling and inverse remapping along the normal direction of the central skeleton line achieves lossless flattening of the arc-shaped text; and adaptive height normalization eliminates the problem of inconsistent character scales caused by perspective distortion. This method does not require manual specification of prior knowledge of the center point, avoiding feature mapping errors caused by the absence or deviation of the central reference object in traditional polar coordinate unfolding methods, and significantly reducing the difficulty of feature extraction and inference in subsequent text recognition models.

[0104] Optionally, the operation of determining the ownership unit of the target manhole cover instance based on the prediction results of the visual ownership identification attributes of each manhole cover instance and the ownership unit text information includes: matching the ownership unit text information with a preset keyword library to obtain the text recognition ownership result and the corresponding text confidence score; obtaining the ownership unit category prediction result and pattern category prediction result in the visual ownership identification attributes output by the multi-label target detection model; verifying the text recognition ownership result based on the pattern category prediction result: if the ownership unit corresponding to the text recognition ownership result and the pattern category prediction result belong to a preset mutually exclusive combination, then the text recognition ownership result is determined to be invalid; otherwise, the text recognition ownership result is determined to be valid; when the text recognition ownership result is determined to be valid, the text recognition ownership result is taken as the final ownership unit; otherwise, the ownership unit category prediction result output by the multi-label target detection model is taken as the final ownership unit.

[0105] In this embodiment, in step S2, the multi-label target detection model predicts the disease status and visual ownership identifiers (including ownership unit category and pattern category). In step S4, the ownership unit text information is extracted through arc-shaped character recognition. Based on this, the system fuses and comprehensively judges the ownership identification results from the two sources mentioned above.

[0106] First, the system matches the ownership unit text information output by the SVTR model in step S4 with a preset keyword database. The keyword database stores keywords corresponding to each ownership unit; for example, "electricity" corresponds to an electricity company, "gas" to a gas company, and "Beixin Infrastructure" to Beixin Infrastructure Company. The system searches the text character sequence for the existence of these keywords. If a keyword is matched, the ownership unit corresponding to that keyword is taken as the text recognition ownership result, and the system outputs the text confidence score corresponding to the match.

[0107] Simultaneously, the system acquires the visual ownership identification attributes output by the multi-label target detection model in step S2, including the predicted results of ownership unit categories (e.g., electricity, communication, water supply, drainage, gas, heating, etc.) and pattern category prediction results (e.g., checkered pattern, radial pattern, circular pattern, grid pattern, anti-slip raised dot pattern, double-layer pattern, etc.). These prediction results have been output in parallel with the disease status attributes in step S2.

[0108] Since text recognition and visual ownership identification are two independent identification paths, when one identification method suffers from low confidence due to wear, stains, or other interference, the other method may still provide accurate ownership clues. The system employs a cross-validation mechanism to comprehensively determine the results from both sources. Specifically, the system verifies the text recognition ownership results based on the pattern category prediction results. During verification, the system queries a pre-defined mutually exclusive combination library, which records the correspondence between a specific ownership unit and a pattern type that cannot appear in it. For example, if a power well cover in a certain area does not have a checkered pattern, then "power" and "checkered pattern" are recorded as mutually exclusive combinations in the library; similarly, if a gas well cover in a certain area does not have a radial pattern, then "gas" and "radial pattern" are recorded as mutually exclusive combinations in the library.

[0109] If the ownership unit corresponding to the text recognition ownership result and the pattern category prediction result belong to the aforesaid preset mutually exclusive combination, it is determined that the text recognition ownership result is invalid, which indicates that the text recognition may have a misreading caused by wear, stain or occlusion; otherwise, it is determined that the text recognition ownership result is valid. When it is determined that the text recognition ownership result is valid and the corresponding text confidence is greater than or equal to a first preset threshold, the system takes the text recognition ownership result as the final ownership unit; when it is determined that the text recognition ownership result is invalid, or the corresponding text confidence is less than the first preset threshold, the system acquires the ownership unit category prediction result output by the multi-label object detection model and the confidence thereof; if the confidence of the ownership unit category prediction result is greater than or equal to a second preset threshold, the system takes the ownership unit category prediction result as the final ownership unit; if the confidence of the ownership unit category prediction result is less than the second preset threshold, it indicates that both the text recognition and the visual ownership identifier have high uncertainty, and the system marks the ownership unit of the target manhole cover instance as "unknown" or "to be determined", and generates a manual review prompt message. It should be noted that, the first preset threshold is the minimum score standard for measuring the reliability of the arc-shaped text recognition result, and the specific value thereof may be set according to the requirement on ownership determination accuracy in practical applications, for example, it is preferably between 0.70 and 0.85; the second preset threshold is the minimum score standard for measuring the reliability of a visual ownership identifier such as a Logo classification prediction result, and it is preferably between 0.60 and 0.80. By setting the aforesaid dual confidence thresholds for checking, the system can effectively filter out low-confidence noise point data caused by extremely severe working conditions such as severe manhole cover wear and mud coverage, avoid blind misjudgment of the automated system, and ensure high accuracy of the ownership recognition result and the security of the service closed loop.

[0110] Taking an actual recognition as an example: the SVTR model recognizes a horizontal text image of a certain manhole cover, outputs a text character sequence of "Beixin Foundation", the system maps it to "Beixin Foundation" through keyword matching, with a text confidence of 0.72; meanwhile, the YOLO model outputs an ownership unit category prediction result of "electric power" and a pattern category prediction result of "square pattern". The system queries a preset mutually exclusive combination library, and finds that "electric power" and "square pattern" are a mutually exclusive combination locally, so it is determined that the text recognition ownership result ("Beixin Foundation") is valid, and finally "Beixin Foundation" is determined as the ownership unit of the manhole cover.

[0111] Thus, the text recognition result is mapped to the ownership unit through keyword matching, the text recognition ownership result is subjected to blacklist exclusion verification through the pattern category prediction result, the text recognition result is preferentially adopted when the text recognition result is valid, otherwise the system falls back to the prediction result of the visual ownership identifier, which realizes dual mutual verification of text recognition and visual identifier, overcomes the defect of insufficient accuracy of a single recognition means in the case of identifier wear and stain, and makes the ownership discrimination result more reliable.

[0112] Optionally, the operation of generating a maintenance work order for the target manhole cover instance based on the prediction results of the defect status attributes, the ownership unit, and the location information includes: determining the defect severity level of the target manhole cover instance based on the prediction results of the defect status attributes; generating a maintenance work order for the target manhole cover instance based on the prediction results of the defect status attributes, the defect severity level, the ownership unit, and the location information; and the method further includes: submitting the maintenance work order to the maintenance system for review and then dispatching it to the maintenance terminal corresponding to the ownership unit; in response to the false alarm feedback information returned by the maintenance system, adding the road surface image corresponding to the target manhole cover instance and the prediction results output by the multi-label target detection model as negative samples to the training dataset for iterative optimization of the multi-label target detection model; and in response to the maintenance completion information returned by the maintenance system, adding the target manhole cover instance to the next round of inspection path.

[0113] In this embodiment, in step S2, the multi-label target detection model predicts the attributes of the damage status; in step S4, it makes a comprehensive determination of the ownership unit; and in step S1, it acquires the location information of the acquisition time corresponding to each frame of the road surface image. Based on this, the system integrates the above information to generate a maintenance work order and drives the subsequent closed-loop processing flow.

[0114] During the maintenance work order generation phase, the system first determines the severity level of the target manhole cover instance based on the predicted results of the defect status attributes output by the multi-label target detection model in step S2. Specifically, the system rates the defect status according to preset severity grading rules: defects posing a serious threat to public safety, such as missing manhole covers or large-area damage, are rated as "urgent"; defects with a minor impact, such as slight elevation differences or damage around the manhole, are rated as "moderate". The severity level is used for prioritizing subsequent work order processing and determining response time limits.

[0115] Then, based on the predicted results of the defect status attributes, the defect severity level, the ownership unit determined in step S4, and the location information obtained in step S1, the system automatically generates a structured maintenance work order for the target manhole cover instance. The maintenance work order is a structured data record containing complete information such as defect description, location information, responsible unit, and urgency rating. The defect description field in the work order is filled with the specific defect status category output by the multi-label target detection model in step S2; the location information field is filled with the latitude and longitude coordinates and road station number obtained in step S1; the responsible unit field is filled with the ownership unit output after comprehensive judgment in step S4; and the urgency rating field is filled with the defect severity level determined in this step.

[0116] Subsequently, the system submits the generated maintenance work order to the maintenance system via a data interface. Upon receiving the work order, the maintenance system has professionals conduct a secondary verification of its contents. If the verification passes, the maintenance system dispatches the work order to the maintenance terminal corresponding to the responsible unit, and maintenance personnel receive the work order and proceed to the site to perform the repair work. If, during the review process, professionals discover a false alarm in the work order content—that is, the system's identification result does not match the actual condition of the manhole cover—the maintenance system returns a false alarm feedback message.

[0117] During the information feedback phase, in response to false alarm feedback from the maintenance system, the system adds the road surface image corresponding to the target manhole cover instance and the prediction results output by the multi-label object detection model as negative samples to the training dataset. Specifically, false alarm samples include: the system misclassifying a normal manhole cover as having defects (false positive), and the system misclassifying one defect state as another (class misclassification). These road surface images and corresponding prediction results are packaged as negative samples and added to the training dataset for subsequent iterative optimization training of the multi-label object detection model. By continuously introducing false alarm samples for model retraining, the system can gradually reduce the false alarm rate and improve the accuracy and robustness of identification.

[0118] The system responds to the maintenance system's return of repair completion information and adds the target manhole cover instance to the next round of inspection routes. After completing the repair work on-site, maintenance personnel upload repair effect pictures and completion time to the maintenance system via the maintenance terminal; the maintenance system returns the repair completion information to this system. Upon receiving the repair completion information, the system marks the manhole cover location as repaired and adds it to the next round of inspection routes so that the effectiveness of the repair can be verified during the next inspection. Simultaneously, the system can trace all events related to the construction, maintenance, and replacement of this manhole cover, forming a complete lifecycle archive for the manhole cover.

[0119] Thus, by automatically determining the severity level based on the predicted damage status and generating structured maintenance work orders, an automated workflow from damage identification to work order output is achieved, avoiding the problems of multiple steps, slow workflow, and susceptibility to errors associated with manual form filling. By submitting maintenance work orders to the management system and having them reviewed by professionals before being dispatched to the corresponding authorized maintenance terminals, the accuracy and traceability of work order dispatch are ensured. By collecting false alarm samples and adding them to the training dataset as negative samples, continuous model optimization and self-evolution are achieved. By adding repaired manhole covers to the next round of inspection paths to verify the effectiveness of repairs, a closed-loop process of "inspection → identification → work order → repair → verification" is formed. This closed-loop mechanism significantly shortens the response time for damage handling and improves the intelligence and precision of urban manhole cover damage management.

[0120] To more clearly illustrate the application effect of this invention in actual road inspection scenarios, a complete implementation case is given below.

[0121] Taking the intelligent inspection of manhole covers along the main roads of a certain city as an example, this case presents a complete implementation of this application in actual road inspection, covering the entire process of hardware deployment, model training and deployment, online inference and closed-loop processing. Figure 10 The inspection flowchart corresponding to this implementation case is shown, fully demonstrating the entire process from road surface perception, defect identification, work order generation to maintenance closure. Combined with... Figure 1 As shown, the specific steps include the following: 1. Hardware Platform Setup. A municipal inspection van was selected as the mobile platform. An industrial-grade high-definition camera (2448×2048 resolution, global shutter, frame rate cap 60fps) was fixedly mounted on the roof. The lens was set at a downward angle of approximately 25° and slightly tilted to the right to ensure its field of view primarily covered the motor vehicle lanes while also covering the right-hand non-motorized vehicle lanes. Synchronous supplementary lights were installed on both sides of the camera to ensure image quality at dusk and night. A GPS / INS combined positioning module (positioning accuracy approximately ±0.5m, output frequency 100Hz) was installed inside the vehicle and connected to an edge computing device for online inference. The camera, positioning module, and edge device were synchronized in time via hardware trigger signals, and each frame of image carried latitude, longitude, station number, and timestamp.

[0122] 2. Adaptive Acquisition Strategy. The system pre-loads a list of known manhole cover coordinates for the road section. During driving, the edge device calculates the distance 'd' between the vehicle and the nearest manhole cover in real time: when 'd' > 20m, it acquires images at a low frame rate of 5fps to save computing power; when 'd' ≤ 20m, it automatically switches to dense acquisition at 30fps, continuously acquiring 8-15 frames of multi-directional images of the same manhole cover from far to near. Multi-frame results for the same manhole cover are comprehensively verified: only when the same defect is consistently detected in at least 3 frames is it confirmed, thereby suppressing motion blur and occasional false detections.

[0123] 3. Dataset and Model Training. Approximately 10,000 original road surface images were collected from a certain urban area. After manual cleaning to remove severely blurred, overexposed, completely occluded, and uncovered images, a modified multi-label LabelImg was used for annotation. Each manhole cover frame was assigned three independent attributes in the format "cls_1 cls_2 ... cls_k xywh": 7 types of status (normal, damaged, missing, displaced, settlement difference, damage to the manhole perimeter / frame, loose), 6 types of logo (electricity, communication, water supply, drainage, gas, heating), and 6 types of pattern (square pattern, radial pattern, circular pattern, grid pattern, anti-slip raised dot pattern, double-layer pattern). Therefore, the number of output channels for the classification branch is: ; Considering the logo's asymmetry, horizontal and vertical flipping were ruled out; only this method was used. Enhancements were applied including random rotation, scaling and cropping, brightness / contrast / saturation perturbations, Gaussian noise and motion blur, and rain / fog simulation. The sample size was expanded to approximately 30,000 images, which were hierarchically divided into training / validation / test sets in a 7:2:1 ratio. An improved YOLO model (which can be called the P2_C3k2-CBAM_Robust-FASFFHead network) was used, with an input resolution of 1280×1280, a batch size of 8, and SGD optimizer (initial learning rate 0.01, cosine annealing). Training was performed for 300 epochs on a single RTX 4090 image. The classification branch of the detection head used Sigmoid activation combined with binary cross-entropy loss to achieve parallel forward output of the three attributes in a single pass. On the test set, the overall detection mAP@0.5 for manhole covers improved from approximately 0.82 to approximately 0.93 compared to the baseline without P2 branch and Robust-FASFFHead. Among them, the recall rate for damaged areas with an area ratio of less than 2% and small targets such as logos improved from approximately 0.71 to approximately 0.89. The inference time per frame was approximately 22ms, which meets the real-time requirements.

[0124] 4. Ownership Recognition Link. When the YOLO model determines that a manhole cover has defects, the OCR link is automatically triggered (it is not triggered if there are no defects, in order to save computing power): First, the manhole cover area is cropped from the original image according to the bounding box, and the FCENet text detection model is input to locate the ownership text area arranged in a fan shape along the edge of the cover; then, based on local features, an adaptive trilinear fitting spatial geometric transformation method is used to extract the upper and lower boundaries and the central skeleton line of the text and remap it along the normal direction to flatten the arc-shaped text into horizontal text lines without loss; finally, it is sent to SVTR pure vision Transformer to recognize the character sequence and obtain the text and confidence score.

[0125] 5. Multi-model cross-verification and fusion judgment. The ownership conclusion is cross-verified by the YOLO model's Logo / pattern recognition and OCR text recognition: the text side uses keyword matching (e.g., recognizing "electricity" maps it to the power company), while the pattern side is used as a blacklist exclusion item. For example, if a power manhole cover in a certain area does not have a square pattern, and the OCR judges it as "electricity" with low confidence, but the pattern detects a square pattern, then the recognition is deemed invalid and discarded. At this time, the system will automatically backtrack and adopt the ownership Logo prediction result output by the YOLO visual network, achieving dual-path mutual protection. After comprehensive judgment, the final conclusion of "Disease Type + Ownership Unit" is output.

[0126] 6. Closed-Loop Processing and Feedback. For defects confirmed to belong to the unit, the system automatically fills in the time, latitude / longitude / station number, defect type, and severity, generating a structured maintenance work order and submitting it to the maintenance system. Professional personnel conduct a second verification and then dispatch the work order to the corresponding unit's maintenance personnel with one click. If a false alarm is detected, it is rejected, and the false alarm sample is collected as retraining material for the model. After repair, the system uploads the effect diagram and completion time on-site. The system can trace all records of the manhole cover's construction, repair, and replacement, and add the location to the next round of inspection routes to verify the effectiveness of the repair, thus completing a closed-loop process of "inspection → identification → work order → repair → verification → feedback." In this case, during a test on an approximately 12km inspection route, a single vehicle inspection covered approximately 340 manhole covers. The manual verification pass rate for defect detection and ownership determination was approximately 94%, reducing the response time by approximately 70% compared to the original manual inspection-back-end interpretation process.

[0127] In summary, this application has the following beneficial effects: (1) To achieve mobile, non-contact, and routine inspection of manhole covers on main roads, taking into account both safety and efficiency. This application utilizes a vehicle-mounted mobile platform and a distance-adaptive acquisition mechanism, relying on GPS / INS positioning and vehicle speed information to calculate the relative distance between the vehicle and the manhole cover in real time. When approaching a manhole cover, it automatically switches to high frame rate dense acquisition and automatically reduces the frequency for other road sections, taking into account both the integrity of the acquisition of key targets and the economy of overall computing power. High-definition images can be continuously acquired without the need for inspection personnel to go on the road. This avoids the traffic safety risks of manual inspection on main roads and overcomes the poor image acquisition caused by the limited position and movement speed of manual inspection. A single inspection can cover hundreds of manhole covers, and the inspection efficiency and coverage are greatly improved compared with traditional methods.

[0128] (2) Achieve parallel recognition of multiple attributes such as defects, ownership, and patterns in a single step, improving both recognition efficiency and detection accuracy. This application uses a YOLO multi-label detection head reconstruction method for parallel output of multiple attributes. On the data side, the labeling format is expanded from "single category + bounding box" to "multi-category index + bounding box". A dynamic multi-class slicing algorithm based on inverse index is designed to achieve batch loading of multiple attributes in the same frame, fundamentally avoiding NMS conflicts caused by multi-attribute box labeling. On the model side, the Softmax of the classification branch is replaced with a channel-by-channel independent Sigmoid and combined with binary cross-entropy loss to mathematically eliminate the class mutual exclusion under the single-label mechanism, enabling the three attributes of defects, logo, and patterns to be output in parallel in a single forward inference, avoiding the time increase caused by attribute-by-attribute inference or multi-model chaining. Combined with small target enhancement for minor defects of manhole covers. The robust network P2_C3k2-CBAM_Robust-FASFFHead provides finer location anchors by adding a high-resolution P2 detection branch, introducing C3k2-CBAM attention to adaptively suppress shallow background noise and enhance the response to small defects, replacing stride convolution with the ADown module to achieve lossless downsampling of detailed information, and using Robust-FASFFHead to perform adaptive spatial weighted fusion of multi-scale features. This significantly improves the recall rate of extremely small targets such as damage and logos with an area ratio of less than 2%, while suppressing the risks of false alarms and memory overflow caused by high-resolution feature maps. The detection accuracy and robustness under complex road conditions are improved simultaneously, which can meet the real-time processing requirements of mobile platforms.

[0129] (3) Automatic identification of ownership units is realized, providing a reliable basis for accurate order dispatch and responsibility traceability. This application uses a dedicated recognition link for arc-shaped ownership text and a YOLO-OCR multi-model mutual verification decision strategy. It uses FCENet Fourier contour detection to accurately locate the ownership text arranged in a fan shape along the edge of the cover, uses adaptive trilinear fitting spatial geometric transformation to nonlinearly flatten the arc-shaped text, and uses SVTR visual Transformer to complete highly robust character recognition, thus forming a complete recognition path for arc-shaped text on the manhole cover. This solves the problem that the arc-shaped arrangement of ownership text along the edge of the manhole cover is difficult to be accurately read by conventional OCR. Through the fusion decision strategy of "keyword matching + pattern blacklist exclusion + disease-triggered call", the ownership conclusion is automatically obtained by the logo, pattern and arc text through multi-model mutual verification. Even if the manhole cover marking is worn or stained, making it difficult to identify by a single method, a reliable result can be obtained through cross-validation and blacklist exclusion. When text recognition is blocked or the trigger pattern blacklist is determined to be invalid, the system can smoothly fall back to the visual ownership result output by the multi-label detection model for backup. This strategy only triggers OCR when a defect is detected, ensuring the accuracy of ownership determination while minimizing computing power consumption. It overcomes the drawbacks of manual identification, such as errors in recording and omissions, and confusion in ownership, laying the foundation for accurate order dispatch and accountability in the future.

[0130] (4) Achieve a closed-loop process from inspection to treatment, significantly improving the intelligence and precision of governance. This application automatically converts the identification results into structured work orders containing time, location, disease type and ownership unit, and drives the closed-loop operation of "dispatch → repair → verification → false alarm feedback". By connecting with the back-end automatic work orders, a fully automated flow from identification to treatment is formed, which not only greatly shortens the response time of disease treatment and reduces manual input errors, but also allows false alarm samples to be continuously fed back for model retraining, realizing the system to continuously self-optimize as it runs, and significantly improving the intelligence and precision of urban manhole cover disease governance.

[0131] In addition, refer to Figure 1 As shown, according to a second aspect of this embodiment, a storage medium is provided. The storage medium includes a stored program, wherein, when the program is executed, a processor performs the method described in any of the above embodiments.

[0132] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0133] Example 2

[0134] Figure 11 This illustration shows a collaborative identification system for manhole cover defects and ownership based on multi-tag detection, corresponding to the method described in Embodiment 1. (Reference) Figure 11As shown, the system includes: an image acquisition module 1110, used to acquire a sequence of road surface images collected by a mobile acquisition platform while it travels along a road. The road surface image sequence includes multiple frames of road surface images of the same manhole cover at multiple acquisition times, from far to near, and each frame of road surface image corresponds to the location information of the acquisition time; and a target detection module 1120, used to input the multiple frames of road surface images into a multi-label target detection model for inference, and the multi-label target detection model outputs the bounding box of each manhole cover instance in each frame of road surface image and the prediction results of multiple independent attributes; the multiple independent attributes include at least the manhole cover's damage status attribute and visual ownership identification attribute; the head network of the multi-label target detection model is configured to output confidence scores independently for each attribute channel, and the outputs of each attribute channel are not mutually constrained; The manhole cover instance determination module 1130 is used to perform cross-frame association of detection results belonging to the same manhole cover in the multi-frame road surface images based on the positioning information, and to perform consistency verification on the associated detection results, and determine the target manhole cover instance with defects based on the consistency verification results; the ownership unit determination module 1140 is used to perform text recognition for arc-shaped text in the image area corresponding to the bounding box of the target manhole cover instance, extract the ownership unit text information, and determine the ownership unit of the target manhole cover instance based on the prediction results of the visual ownership identification attributes of each manhole cover instance and the ownership unit text information; the maintenance work order generation module 1150 is used to generate a maintenance work order for the target manhole cover instance based on the prediction results of the defect status attributes, the ownership unit and the positioning information.

[0135] It should be noted that the multi-tag detection-based collaborative identification system for manhole cover defects and ownership provided in this embodiment can realize all the functions and steps in the above method embodiments, solve the same technical problems, and achieve the same technical effects. The similarities will not be repeated here.

[0136] Example 3

[0137] Figure 12 This illustration shows a collaborative identification system for manhole cover defects and ownership based on multi-tag detection, corresponding to the method described in Embodiment 1. (Reference) Figure 12As shown, the system includes: a processor 1210; and a memory 1220, connected to the processor 1210, for providing the processor 1210 with instructions to process the following steps: acquiring a sequence of road surface images collected by a mobile acquisition platform while it travels along a road, the sequence of road surface images including multiple frames of road surface images of the same manhole cover at multiple acquisition times from far to near, each frame of road surface image corresponding to the location information of the acquisition time; inputting the multiple frames of road surface images into a multi-label target detection model for inference, the multi-label target detection model outputting the bounding box of each manhole cover instance in each frame of road surface image and the prediction results of multiple independent attributes; the multiple independent attributes include at least the manhole cover's damage status attribute and visual ownership identification attribute; the multi-label target detection model's header... The network is configured to output confidence scores independently for each attribute channel, and the outputs of each attribute channel are independent of each other. Based on the location information, the detection results of the same manhole cover in the multi-frame road surface images are correlated across frames, and the correlation results are verified for consistency. Based on the consistency verification results, the target manhole cover instance with defects is determined. Text recognition is performed on the image region corresponding to the bounding box of the target manhole cover instance for arc-shaped text, and the ownership unit text information is extracted. Based on the prediction results of the visual ownership identifier attribute of each manhole cover instance and the ownership unit text information, the ownership unit of the target manhole cover instance is determined. Based on the prediction results of the defect status attribute, the ownership unit, and the location information, a maintenance work order for the target manhole cover instance is generated.

[0138] It should be noted that the multi-tag detection-based collaborative identification system for manhole cover defects and ownership provided in this embodiment can realize all the functions and steps in the above method embodiments, solve the same technical problems, and achieve the same technical effects. The similarities will not be repeated here.

[0139] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0140] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0141] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for collaborative identification of manhole cover defects and ownership based on multi-label detection, characterized in that, include: The system acquires a sequence of road surface images collected by a mobile data acquisition platform while it is traveling along a road. The sequence of road surface images includes multiple frames of road surface images of the same manhole cover at multiple acquisition times, from far to near. Each frame of road surface image corresponds to the location information of the acquisition time. The multi-frame road surface images are input into a multi-label target detection model for inference. The multi-label target detection model outputs the bounding box of each manhole cover instance in each frame of the road surface image and the prediction results of multiple independent attributes. The multiple independent attributes include at least the damage status attribute and the visual ownership identification attribute of the manhole cover. The head network of the multi-label target detection model is configured to output confidence for each attribute channel independently, and the outputs of each attribute channel are not mutually constrained. Based on the location information, the detection results of the same manhole cover in the multi-frame road surface images are correlated across frames, and the correlation results are verified for consistency. Based on the consistency verification results, the target manhole cover instance with defects is determined. The image region corresponding to the bounding box of the target manhole cover instance is subjected to text recognition for arc-shaped text, the ownership unit text information is extracted, and the ownership unit of the target manhole cover instance is determined based on the prediction results of the visual ownership identification attributes of each manhole cover instance and the ownership unit text information. Based on the prediction results of the disease status attributes, the ownership unit, and the location information, a maintenance work order for the target manhole cover instance is generated.

2. The method according to claim 1, characterized in that, The operation of acquiring road surface image sequences collected by a mobile acquisition platform while it is traveling along a road includes: The current location information of the mobile acquisition platform is obtained in real time, and the distance between the mobile acquisition platform and the nearest manhole cover is calculated based on the current location information and the pre-loaded list of manhole cover coordinates. When the distance is greater than a preset distance threshold, road surface images are acquired at a first frame rate; when the distance is less than or equal to the preset distance threshold, road surface images are acquired at a second frame rate until the mobile acquisition platform passes the nearest manhole cover; the second frame rate is higher than the first frame rate. Each captured road surface image is associated with and stored along with the location information at the time of capture, forming the road surface image sequence.

3. The method according to claim 1, characterized in that, The multi-label target detection model is an improved YOLO model. The improvements of the improved YOLO model based on the baseline YOLO model include: A P2 feature layer is added to the feature pyramid of the neck network. The output feature map resolution of the P2 feature layer is higher than that of the minimum feature layer output feature map of the baseline YOLO model. Embed channel attention modules and spatial attention modules in at least a portion of the C3k2 modules of the backbone network and neck network; At least some stride convolutions in the downsampling path of the neck network are replaced with ADown modules. The ADown modules are used to divide the input features into two subsets in the channel dimension, process them separately, and then concatenate the processing results of the two subsets in the channel dimension. An improved head network is employed, comprising a multi-scale fusion module, a P2 detection head, a P3 detection head, a P4 detection head, and a P5 detection head. The multi-scale fusion module is used to: map the feature maps of four different scales output by the neck network to the spatial resolution of the target scale for each detection head, obtaining four sets of aligned features; concatenate the four sets of aligned features and perform convolution and normalization to generate a spatial attention weight matrix; and perform weighted fusion of the four sets of aligned features according to the spatial attention weight matrix to obtain a fused feature map of the target scale, which is then input into the detection head of the corresponding scale.

4. The method according to claim 1, characterized in that, The operation of performing text recognition on the image region corresponding to the bounding box of the target manhole cover instance, specifically for arc-shaped text, and extracting the ownership unit text information, includes: The manhole cover area image is cropped from the corresponding road surface image based on the bounding box of the target manhole cover instance; The arc-shaped text region in the manhole cover area image is located by using a Fourier contour embedding network, and the contour that fits the boundary of the arc-shaped text is output. Based on the outline, the corresponding arc-shaped text region image is extracted from the manhole cover region image, and the arc-shaped text region image is subjected to spatial geometric transformation to flatten the arc-shaped text in the arc-shaped text region image into horizontal text, thereby generating a horizontal text image. The horizontal text image is identified using a text recognition model, and the text character sequence and confidence score are output as the ownership unit text information.

5. The method according to claim 4, characterized in that, The operation of performing a spatial geometric transformation on the arc-shaped text region image to flatten the arc-shaped text in the image into horizontal text and generate a horizontal text image includes: Principal component analysis is performed on the text pixel coordinates in the arc-shaped text region image to extract the principal direction vector, and the arc-shaped text region image is rotated and aligned according to the principal direction vector; The text pixels in the straightened arc-shaped text region image are divided into multiple bins along the horizontal direction, and the upper boundary extreme point, lower boundary extreme point and center point of each bin are extracted. Perform quadratic polynomial fitting on the upper boundary extreme point, the lower boundary extreme point, and the center point respectively to obtain the upper edge fitting curve, the lower edge fitting curve, and the center skeleton fitting curve; The tangent slope is calculated at multiple sampling points of the fitted curve of the central skeleton, and the normal direction at each sampling point is determined based on the tangent slope. Pixels are extracted between the upper edge fitting curve and the lower edge fitting curve along the normal direction, and the extracted pixels are linearly mapped to a standard canvas of a preset height to generate the corresponding horizontal text image.

6. The method according to claim 1, characterized in that, The operation of determining the ownership unit of the target manhole cover instance based on the prediction results of the visual ownership identifier attributes of each manhole cover instance and the ownership unit text information includes: The ownership unit text information is matched with a preset keyword library to obtain the text recognition ownership result and the corresponding text confidence score; Obtain the prediction results of ownership unit category and pattern category in the visual ownership identification attribute output by the multi-label target detection model; The ownership result of the character recognition is verified based on the pattern category prediction result: if the ownership unit corresponding to the character recognition ownership result and the pattern category prediction result belong to a preset mutually exclusive combination, the ownership result of the character recognition is determined to be invalid; otherwise, the ownership result of the character recognition is determined to be valid. If the text recognition ownership result is deemed valid, the text recognition ownership result shall be taken as the final ownership unit; otherwise, the ownership unit category prediction result output by the multi-label target detection model shall be taken as the final ownership unit.

7. The method according to claim 1, characterized in that, The operation of generating a maintenance work order for the target manhole cover instance based on the prediction results of the defect status attributes, the ownership unit, and the location information includes: Based on the prediction results of the disease status attributes, the severity level of the disease of the target manhole cover instance is determined; Based on the prediction results of the disease status attributes, the severity level of the disease, the ownership unit, and the location information, a maintenance work order for the target manhole cover instance is generated; Furthermore, the method also includes: The maintenance work order is submitted to the management system for review and then dispatched to the maintenance terminal corresponding to the ownership unit. In response to the false alarm feedback information returned by the maintenance system, the road surface image corresponding to the target manhole cover instance and the prediction result output by the multi-label target detection model are added as negative samples to the training dataset for iterative optimization of the multi-label target detection model; In response to the maintenance completion information returned by the maintenance system, the target manhole cover instance is added to the next round of inspection path.

8. A storage medium, characterized in that, The storage medium includes a stored program, wherein, when the program is executed, the method described in any one of claims 1 to 7 is performed by a processor.

9. A collaborative identification system for manhole cover defects and ownership based on multi-tag detection, characterized in that, include: The image acquisition module is used to acquire road surface image sequences collected by the mobile acquisition platform while it is traveling along the road. The road surface image sequence includes multiple frames of road surface images of the same manhole cover at multiple acquisition times from far to near. Each frame of road surface image corresponds to the location information of the acquisition time. The target detection module is used to input the multi-frame road surface images into a multi-label target detection model for inference. The multi-label target detection model outputs the bounding box of each manhole cover instance in each frame of road surface image and the prediction results of multiple independent attributes. The multiple independent attributes include at least the damage status attribute and the visual ownership identification attribute of the manhole cover. The head network of the multi-label target detection model is configured to output confidence for each attribute channel independently, and the outputs of each attribute channel are not mutually constrained. The target manhole cover instance determination module is used to perform cross-frame association of the detection results of the same manhole cover in the multi-frame road surface images based on the positioning information, and to perform consistency verification on the associated detection results, and determine the target manhole cover instance with defects based on the consistency verification results. The ownership unit determination module is used to perform text recognition on the image region corresponding to the bounding box of the target manhole cover instance for arc-shaped text, extract the ownership unit text information, and determine the ownership unit of the target manhole cover instance based on the prediction results of the visual ownership identification attributes of each manhole cover instance and the ownership unit text information. The maintenance work order generation module is used to generate a maintenance work order for the target manhole cover instance based on the prediction results of the defect status attributes, the ownership unit, and the location information.

10. A collaborative identification system for manhole cover defects and ownership based on multi-label detection, characterized in that, include: processor; A memory, connected to the processor, for providing the processor with instructions to perform the following processing steps: The system acquires a sequence of road surface images collected by a mobile data acquisition platform while it is traveling along a road. The sequence of road surface images includes multiple frames of road surface images of the same manhole cover at multiple acquisition times, from far to near. Each frame of road surface image corresponds to the location information of the acquisition time. The multi-frame road surface images are input into a multi-label target detection model for inference. The multi-label target detection model outputs the bounding box of each manhole cover instance in each frame of the road surface image and the prediction results of multiple independent attributes. The multiple independent attributes include at least the damage status attribute and the visual ownership identification attribute of the manhole cover. The head network of the multi-label target detection model is configured to output confidence for each attribute channel independently, and the outputs of each attribute channel are not mutually constrained. Based on the location information, the detection results of the same manhole cover in the multi-frame road surface images are correlated across frames, and the correlation results are verified for consistency. Based on the consistency verification results, the target manhole cover instance with defects is determined. The image region corresponding to the bounding box of the target manhole cover instance is subjected to text recognition for arc-shaped text, the ownership unit text information is extracted, and the ownership unit of the target manhole cover instance is determined based on the prediction results of the visual ownership identification attributes of each manhole cover instance and the ownership unit text information. Based on the prediction results of the disease status attributes, the ownership unit, and the location information, a maintenance work order for the target manhole cover instance is generated.

Images