A sewer pipeline inspection data processing method and system based on time sequence correlation and credibility fusion

Abstract

The application discloses a kind of based on timing association and credibility fusion sewer inspection data processing method and system, belongs to belong to electric digital data processing technical field.The method is by constructing the special detection dataset and special identification dataset of sewer inspection record, train character detection model and character recognition model;Inspection video record is segmented sampling, and combined with the local contrast weight of character preset area and character edge geometric feature Effective graphic data frame is filtered;Subsequently, character area detection and character content reading are carried out on effective graphic data frame, to form the candidate inspection data containing character area position, character content and confidence, finally, combined with the timing logical relationship of running distance or time Abnormal data correction is carried out, and the sewer inspection data processing result is output.The application can improve the accuracy, credibility and timing consistency of continuous sewer inspection data processing.

Description

Technical Field

[0001] This invention belongs to the field of electronic digital data processing technology, specifically relating to a method and system for processing drainage pipeline inspection data based on the fusion of time-series correlation and reliability. Background Technology

[0002] With the continuous improvement of digital management of urban drainage pipe networks, pipe inspection robots and camera equipment are commonly used to collect images or video data of the pipe interiors during drainage pipe inspections. The inspection process also records pipe numbers, inspection locations, travel distances, time information, and other maintenance-related information. This data provides a basis for pipe network asset management, fault analysis, and maintenance decisions. Therefore, it is necessary to process the data generated during inspections, converting the business information it contains into inspection data that can be correlated and used according to the inspection time relationship, thereby improving the efficiency of inspection data organization and management.

[0003] In existing technologies, character reading from images or videos typically employs traditional image processing algorithms or deep learning-based character recognition models. Traditional methods usually read character content from images through edge detection, character region extraction, and template matching, but their recognition accuracy is easily affected by the acquisition environment. In drainage pipeline inspection scenarios, factors such as insufficient lighting inside the pipeline, high ambient humidity, water mist obscuring the image, interference from pipe wall textures, and image changes caused by equipment movement reduce the distinguishability between graphic characters and the background, making it difficult to accurately locate character regions and thus affecting the accuracy of character data reading. Furthermore, existing deep learning-based methods often rely on a large number of manually labeled samples for training, while drainage pipeline inspection records have unique characteristics such as specific scenarios, variable character distribution, and significant variations in image quality, leading to insufficient adaptability of general character recognition models in practical applications. In addition, existing graphic character reading methods typically process single images, lacking correlation analysis of character information between multiple image frames in continuous inspection records. When some image frames have blurred or obscured characters or fluctuating recognition results during video acquisition, incomplete, discontinuous, or erroneous character reading results are likely to occur, making it difficult to meet the requirement of continuous character data reading in drainage pipeline inspection records.

[0004] Therefore, there is an urgent need to improve existing technologies to solve the problems of inaccurate character area identification, insufficient adaptability to complex environments, and poor stability of continuous data reading in the process of reading graphic character data in drainage pipeline inspection records. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a drainage pipeline inspection data processing method and system based on the fusion of time sequence correlation and reliability, so as to solve the problems of difficult accurate identification of character areas, insufficient adaptability to complex environments, and poor stability of continuous data reading in the process of reading graphic character data in drainage pipeline inspection records.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for processing drainage pipeline inspection data based on the fusion of temporal correlation and reliability, comprising the following steps: S100: Obtain characterless background images from drainage pipeline inspection records and perform data augmentation processing. Combined with a corpus containing business attribute characters, construct a dedicated detection dataset and a dedicated recognition dataset for graphic character data in drainage pipeline inspection records using an adaptive contrast rendering strategy. S200: Set training hyperparameters, train a character detection model based on the differentiable binarization algorithm using the dedicated detection dataset, and train a single-vision character recognition model using the dedicated recognition dataset, to obtain the optimal character detection model and the optimal character recognition model respectively. S300: Performs second-by-second segmentation processing on the input drainage pipe inspection video recording according to the preset timeline, and allocates multiple processing threads to perform interval sampling on the image data within each second in parallel, and obtains the associated set composed of the extracted graphic data frames and second-level time tags; S400: For each graphic data frame in the associated set, locate the character preset region and calculate the local contrast weight of the character preset region relative to the adjacent background region. Combine the character edge geometric features to evaluate character readability. Remove graphic data frames with character readability evaluation scores lower than a preset threshold to obtain an effective graphic data frame sequence. S500: Using the optimal character detection model and the optimal character recognition model, perform frame-by-frame detection and reading on the effective graphic data frame sequence to generate a second-level inspection candidate data set containing character region positions, character content and confidence levels; S600: Perform similar area determination aggregation based on spatial intersection-union ratio on the second-level inspection candidate data set, and combine the temporal logic characteristics of the drainage pipeline inspection record parameters to perform credibility optimization and temporal correction, and output the drainage pipeline inspection data processing results.

[0007] Preferably, the adaptive contrast rendering strategy in step S100 specifically includes: statistically analyzing the grayscale distribution feature value of the character preset writing area in the characterless background image; if the grayscale distribution feature value is higher than a preset brightness threshold, then assigning a first color to the character to be rendered and adding a high-contrast edge envelope to the character to be rendered; if the grayscale distribution feature value is lower than or equal to the preset brightness threshold, then assigning a second color to the character to be rendered that contrasts with the first color.

[0008] Further, the data augmentation processing in step S100 specifically includes: performing at least one operation on the characterless background image, including random horizontal flipping, random rotation, random scaling, random cropping, color space transformation, and adding Gaussian noise, to obtain a set of background images; randomly injecting gray-white water mist masking noise and lens splash masking of a simulated drainage pipe inspection environment into the set of background images to enhance the feature extraction capability of the character recognition model in scenes where the character and background colors are similar.

[0009] Preferably, step S200 includes: given hyperparameters including learning rate, batch size, and number of training rounds, inputting the dedicated detection dataset into the character detection model for training, and obtaining the optimal character detection model through forward propagation, error calculation, and backpropagation until the localization loss function converges; training the single-vision character recognition model using the dedicated recognition dataset, and obtaining the optimal character recognition model that can map character graphic data into character sequences and confidence scores by extracting local features and global topological structure features of character images.

[0010] Further, step S300 specifically involves: assigning a video second interval to each processing thread, with each processing thread extracting graphic data frames at a preset frequency within its assigned second interval, and storing the extracted graphic data in association with its corresponding second-level time stamp.

[0011] Preferably, the specific steps for character readability evaluation in step S400 include: converting the graphic data frame into a grayscale image; obtaining the original edge image using a dual-threshold edge detection operator; performing morphological closing operations on the original edge image using a transverse convolution kernel; performing connected component analysis on the processed edge image; calculating the area, width, height, and aspect ratio of each region; and selecting candidate character regions; calculating the average grayscale difference between the candidate character regions and adjacent background regions to obtain local contrast weights; and combining the number of candidate character regions and the sum of their widths to obtain a weighted calculation score for the character readability evaluation of the graphic data frame.

[0012] Further, step S500 specifically includes: sequentially extracting graphic data frames from the valid graphic data frame sequence, using the optimal character detection model to infer a set of character region positions, and cropping a set of character images from the original graphic data frame based on the set of character region positions; using the optimal character recognition model to perform inference reading on the set of character images to obtain the character content and confidence level corresponding to each character image, and combining them to generate an inspection data processing result composed of the character region positions, the character content, and the confidence level, until the processing of all valid graphic data frames within that second is completed.

[0013] Further, the specific steps of S600 for performing the same type of region determination aggregation and optimal result selection include: calculating the intersection-union ratio (IUGR) between character regions obtained from different valid graphic data frames within the same second; if the IUGR is greater than a preset IUGR threshold, the corresponding character regions are determined to be the same type of region and assigned to the same set of the same type of region; in each set of the same type of region, the set of associated data groups with the longest character length is first selected, and the result with the highest average confidence level is selected from the set of associated data groups as the set of associated data groups corresponding to that second.

[0014] Furthermore, the optimal result correction in step S600 specifically includes: extracting numerical character information from the associated data set corresponding to adjacent second-level time periods, and determining whether the numerical character information conforms to the monotonically increasing logic of running distance or time in the drainage pipeline inspection record; if the numerical character information of the current second violates the monotonically increasing logic, then logically correcting the associated data set of the current second according to the numerical change trend of the time sequence context.

[0015] Secondly, the present invention also provides a drainage pipeline inspection data processing system based on the fusion of temporal correlation and reliability, comprising: The dataset construction module is used to construct a dedicated detection dataset and a dedicated recognition dataset for graphic character data of drainage pipeline inspection records based on the background grayscale distribution characteristics of the character preset area and the noise of the simulated environment. The model training module is used to train and obtain the optimal character detection model and the optimal character recognition model using the dedicated dataset. The multi-threaded sampling module is used to perform parallel sampling of drainage pipeline inspection video recordings in segments of one second, in order to obtain an associated set consisting of graphic data frames and second-level time tags; The local weight evaluation module is used to evaluate the readability of characters by combining the local contrast weight of the preset region of the character with the edge geometric features, and to remove graphic data frames with evaluation scores below a preset threshold to obtain a valid graphic data frame sequence. The detection and recognition processing module is used to perform frame-by-frame detection, cropping and character reading on the effective graphic data frame sequence to generate a set of inspection candidate data; The time-series aggregation correction module is used to perform the same area judgment on a second-by-second basis, and output the final drainage pipeline inspection data processing result by combining the time-series monotonically increasing logic of the inspection record parameters.

[0016] This invention discloses a drainage pipeline inspection data processing method and system based on temporal correlation and credibility fusion. It constructs dedicated detection and recognition datasets for graphic character data in drainage pipeline inspection records, and adaptively processes the characters to be rendered by combining the background grayscale distribution characteristics of the preset character regions. This allows the training samples to better reflect the differences between character data and the background in the drainage pipeline inspection environment. Simultaneously, by adding interference factors corresponding to the drainage pipeline inspection environment to the background image, the adaptability of the character detection and recognition models to different inspection image conditions is improved, thereby enhancing the stability of graphic character data reading.

[0017] During the graphic character data reading process, this invention does not determine whether a graphic data frame is suitable for character reading solely based on its overall clarity. Instead, it assesses the readableness of character information within the graphic data frame by considering the local grayscale differences between the preset character area and adjacent background areas, as well as the geometric features of the character edges. Valid graphic data frames are then selected based on the assessment results. This reduces the impact of invalid graphic data frames caused by factors such as internal pipe lighting, water mist, pipe wall texture, and movement of the acquisition equipment on subsequent character detection and reading results, thereby improving the accuracy of character region positioning and character content reading.

[0018] Furthermore, this invention spatially correlates and aggregates the character reading results obtained from different valid graphic data frames within the same second-level time period, and selects corresponding associated data sets based on character length and confidence level. On this basis, it verifies and corrects the reading results by considering the temporal changes of parameters such as running distance or time in the drainage pipeline inspection records during continuous recording. Through the above processing, the impact of character blurring, partial occlusion, or instantaneous reading errors in a single graphic data frame on the final result can be reduced, ensuring better integrity and continuity of graphic character data in continuous drainage pipeline inspection records, and improving the reliability of automatic reading of graphic character data in drainage pipeline inspection records. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart of a drainage pipeline inspection data processing method based on temporal correlation and reliability fusion provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a drainage pipeline inspection data processing system based on the fusion of temporal correlation and reliability, provided as an embodiment of the present invention.

[0021] The following are the markings in the attached diagram: 100. Dataset Construction Module; 200. Model Training Module; 300. Multi-threaded Sampling Module; 400. Local Weight Evaluation Module; 500. Detection and Recognition Processing Module; 600. Temporal Aggregation Correction Module. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0023] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0024] During drainage pipeline inspection, the continuous image records generated by the inspection equipment contain graphic and character data such as pipeline number, task address, pipeline material, travel distance, and time. Affected by factors such as changes in lighting inside the pipeline, water mist, pipe wall texture, and the movement of the inspection equipment, the clarity of character areas varies in different graphic data frames. Some characters may also have low grayscale differences from adjacent background areas, thus affecting the stability of character area positioning and character content reading. Furthermore, the same character data in continuous inspection records often appears repeatedly in multiple adjacent graphic data frames, and the reading results of a single frame may differ due to local occlusion or changes in image quality. Therefore, it is necessary to select suitable graphic data frames for character reading from the continuous inspection records and perform correlation processing on the character reading results obtained from different graphic data frames.

[0025] To address this, the present invention proposes a method for processing drainage pipeline inspection data based on the fusion of temporal correlation and reliability, such as... Figure 1 As shown, it includes the following steps: S100: Obtain characterless background images from drainage pipeline inspection records and perform data augmentation processing. Combined with a corpus containing business attribute characters, construct a dedicated detection dataset and a dedicated recognition dataset for graphic character data in drainage pipeline inspection records using an adaptive contrast rendering strategy. S200: Set the training hyperparameters, train a character detection model based on the differentiable binarization algorithm using the dedicated detection dataset, and train a single-vision character recognition model using the dedicated recognition dataset, to obtain the optimal character detection model and the optimal character recognition model respectively. S300: Performs second-by-second segmentation processing on the input drainage pipe inspection video recording according to the preset timeline, and allocates multiple processing threads to perform interval sampling on the image data within each second in parallel, and obtains the associated set composed of the extracted graphic data frames and second-level time tags; S400: For each graphic data frame in the associated set, locate the character preset region and calculate the local contrast weight of the character preset region relative to the adjacent background region. Combine the character edge geometric features to evaluate character readability. Remove graphic data frames with character readability evaluation scores lower than a preset threshold to obtain a valid graphic data frame sequence. S500: Utilize the optimal character detection model and the optimal character recognition model to perform frame-by-frame detection and reading of the effective graphic data frame sequence, generating a second-level inspection candidate data set containing character region positions, character content, and confidence levels; S600: Performs aggregation of similar regions based on spatial intersection-union ratio on the second-level inspection candidate data set, and performs reliability optimization and time sequence correction in combination with the temporal logic characteristics of drainage pipeline inspection record parameters, and outputs the drainage pipeline inspection data processing results.

[0026] For ease of understanding, the following explains some key terms in this embodiment: The characterless background image in the drainage pipeline inspection record refers to the background image obtained from the image record during the drainage pipeline inspection process that does not contain the character data to be read, and is used as the basic material for constructing training data for character detection and character recognition.

[0027] Data augmentation is a technique that generates more diverse training samples by performing transformations such as rotation, scaling, cropping, and color adjustment on image data, aiming to improve the generalization ability of the model.

[0028] The corpus of business attribute characters refers to the collection of character data related to drainage pipeline inspection business, including character content such as task address, pipeline number, pipeline material, running distance, date and time, which is used to construct graphic character data with corresponding business attributes.

[0029] The adaptive contrast rendering strategy is a processing method that adjusts the color or edge envelope of the character to be written based on the local brightness distribution of the character's preset writing area in the background image, and is used to generate graphic character training data under different background conditions.

[0030] The dedicated detection dataset and the dedicated recognition dataset refer to training datasets constructed by combining background images, business attribute character corpora, and adaptive contrast rendering strategies for graphic character data in drainage pipeline inspection records. They are used for character region detection and character content reading, respectively.

[0031] The character detection model based on the differentiable binarization algorithm is a model for locating character regions in input graphic data. It forms the binary features of the character regions through differentiable binarization processing and obtains the position of the character regions accordingly.

[0032] A single-vision character recognition model is a model that reads detected character images, maps character graphic data to corresponding character sequences, and outputs confidence scores.

[0033] The character preset area refers to the area in the graphic data frame of the drainage pipeline inspection record that may contain character data, which is predetermined based on the display position of the characters in the inspection record.

[0034] Local contrast weight is a parameter used to represent the degree of grayscale difference between a character candidate region and its adjacent background region.

[0035] Character edge geometric features refer to the geometric attributes of the edge region corresponding to a character, such as area, width, height, aspect ratio, and connectivity. These features are used to filter candidate character regions and participate in character readability evaluation.

[0036] Character readability assessment is a process that quantitatively evaluates whether character data in a graphic data frame is suitable for subsequent reading based on the local contrast weights and edge geometric features of the character candidate region.

[0037] The temporal logic characteristics of drainage pipeline inspection record parameters refer to the temporal relationship of numerical character data such as running distance and time in the continuous inspection record of drainage pipeline as the recording time changes. In this embodiment, it includes the monotonically increasing relationship of running distance or time.

[0038] The present invention provides a method for processing drainage pipeline inspection data based on the fusion of temporal correlation and reliability, the specific implementation of which is as follows: In step S100, a characterless background image from the drainage pipeline inspection record is acquired and data augmentation processing is performed. Combined with a corpus containing characters with business attributes, an adaptive contrast rendering strategy is used to construct a dedicated detection dataset and a dedicated recognition dataset. Specifically, the grayscale feature value of the preset writing area for the character is calculated, such as the average grayscale. When the grayscale feature value is higher than a preset brightness threshold, a first color is assigned to the character to be written, and a high-contrast edge envelope with inverted colors is added. When the grayscale feature value is lower than or equal to the preset brightness threshold, a second color is assigned to the character to be written, contrasting with the first color. This yields graphic character training data suitable for different background brightness conditions.

[0039] In step S200, training hyperparameters are set, and a character detection model based on a differentiable binarization algorithm is trained using a dedicated detection dataset, while a single-vision character recognition model is trained using a dedicated recognition dataset. The training process iteratively updates the model parameters through forward propagation, error calculation, and backpropagation until the corresponding loss function converges, thereby obtaining the optimal character detection model and the optimal character recognition model.

[0040] In step S300, the input drainage pipe inspection video recording is segmented by second according to a preset timeline, and multiple processing threads are allocated to perform interval sampling in parallel. Each processing thread extracts graphic data frames at a preset frequency within the corresponding second interval, and associates and stores the extracted graphic data frames with their corresponding second-level timestamps.

[0041] In step S400, for the sampled graphic data frame, the preset region of the character is located and the local contrast weight is calculated. For example, the difference between the average gray level of the candidate character region and the average gray level of its adjacent background region is calculated. The character readability is evaluated by combining the geometric features of the character edge, and graphic data frames with evaluation scores lower than a preset threshold are eliminated to obtain a valid graphic data frame sequence.

[0042] In step S500, the optimal character detection model and the optimal character recognition model are used to process the effective graphic data frame sequence frame by frame. The character detection model locates the character regions, and the character recognition model reads the character images obtained by cropping based on the character regions. Finally, the two models are combined to generate the inspection data processing result, which includes the location of the character regions, the character content, and the confidence level.

[0043] In step S600, the candidate data set for second-level inspections is aggregated based on spatial intersection-union ratio (IUU) to determine similar regions. Furthermore, the reliability is optimized and the timing is corrected by combining the temporal logic characteristics of the drainage pipeline inspection record parameters. For example, character regions with highly overlapping locations within the same second are categorized, and reading results with longer character lengths and higher confidence are prioritized from each similar region set. It is further determined whether the numerical character information corresponding to adjacent seconds conforms to the monotonically increasing logic of running distance or time, and reading results that do not conform to this timing logic are corrected.

[0044] Through the above processing, corresponding character reading results can be obtained from the graphic data of continuous inspection records of drainage pipelines. Specifically, a dedicated dataset is used to enhance the adaptability of the character detection and recognition models to the drainage pipeline inspection environment; character readability assessment is used to reduce the impact of graphic data frames unsuitable for character reading on subsequent processing; and after spatial correlation, filtering, and subsequent temporal verification of character reading results from multiple graphic data frames within the same second, the impact of single-frame character blurring, occlusion, or instantaneous reading anomalies on the final reading result can be reduced.

[0045] Furthermore, this invention proposes a specific implementation method for the adaptive contrast rendering strategy in step S100. First, the grayscale distribution feature values ​​of the preset writing area for characters in the characterless background image are statistically analyzed to determine the brightness state of the background area corresponding to the character writing position. Specifically, the grayscale values ​​of each pixel within the preset writing area can be averaged, and the resulting average grayscale value can be used as the grayscale distribution feature value; alternatively, the mode, median, or corresponding percentile grayscale value can be extracted from a grayscale histogram as the grayscale distribution feature value. Furthermore, the variance or standard deviation of the grayscale values ​​can be calculated to characterize the dispersion of the grayscale distribution in the corresponding background area.

[0046] When the grayscale distribution characteristic value is higher than a preset brightness threshold, a first color is assigned to the character to be rendered, and a contrasting high-contrast edge envelope is added to the character. The first color can be a dark color that contrasts with the current background, such as black, dark blue, or dark gray. The contrasting high-contrast edge envelope can be achieved by generating a character outline based on the shape of the character to be rendered, expanding the character outline outwards or inwards to form an envelope area, and assigning a color that contrasts with the first color to the envelope area. For example, when the first color is black, the edge envelope can be white or a light color.

[0047] When the grayscale distribution characteristic value is lower than or equal to a preset brightness threshold, a second color that contrasts with the first color is assigned to the character to be rendered. The second color can be a light color such as white, light yellow, or light blue, so that the formed character graphic data maintains a corresponding grayscale difference with the darker background. The specific values ​​of the first and second colors can be set according to the character display format in the actual inspection record.

[0048] Through the above processing, character samples with different contrast relationships with the background can be generated in different grayscale background areas, so that the constructed dedicated detection dataset and dedicated recognition dataset contain different character display states that may appear in drainage pipeline inspection records, providing training data for subsequent character region detection and character content reading.

[0049] In some embodiments of the present invention, step S100 further constructs a dedicated detection dataset and a dedicated recognition dataset through data augmentation processing. Since drainage pipeline inspection records may be affected by factors such as changes in lighting, water mist obstruction, and lens splashes during the acquisition process, the brightness, clarity, and local occlusion states of different graphic data frames vary. Therefore, constructing graphic character samples using corresponding environmental changes is beneficial for improving the adaptability of the character detection model and character recognition model to the graphic data of the inspection records.

[0050] In this regard, the data augmentation process in step S100 includes: performing at least one of the following operations on the characterless background image: random horizontal flipping, random rotation, random scaling, random cropping, color space transformation, and adding Gaussian noise, to obtain a set of background images; and randomly injecting gray-white water mist masking noise and lens splash masking of a simulated drainage pipe inspection environment into the set of background images to form graphic data samples under different environmental conditions.

[0051] Specifically, by performing random horizontal flipping, rotation, scaling, cropping, color space transformation, and adding Gaussian noise on a characterless background image, background images with different viewpoints, directions, sizes, positions, brightness, and clarity can be formed, thus creating a set of background images with multiple graphic representation states, which can then be used for the subsequent synthesis of business attribute characters.

[0052] Building upon this, gray-white water mist masking noise and lens splash masking are randomly injected into the background image set. The gray-white water mist masking noise is generated by superimposing an image layer with a preset grayscale distribution and blurring characteristics onto the background image. The lens splash mask is formed by superimposing local masks with random shapes and transparency, used to simulate local occlusion and graphic deformation that may occur during the collection of inspection records. This ensures that the training data covers graphic states such as low character-background distinction, partial character occlusion, and blurred character edges.

[0053] Through the above data augmentation processing, the representation of background images and graphic character samples can be expanded without changing the character content of drainage pipeline inspection business. This allows the dedicated detection dataset and dedicated recognition dataset to cover the character data status under different inspection environments, improving the adaptability of subsequent character detection and character reading processes to changes in the graphic data of inspection records.

[0054] In the process of reading graphic character data in drainage pipeline inspection records, character detection models and character recognition models are used to convert input graphic data into character region positions and corresponding character content. The model training results directly affect the subsequent graphic character data reading results.

[0055] In response, in step S200, given hyperparameters including learning rate, batch size, and number of training rounds, a dedicated detection dataset is input into the character detection model for training. Through forward propagation, error calculation, and backpropagation until the localization loss function converges, the optimal character detection model is obtained. At the same time, a single-vision character recognition model is trained using a dedicated recognition dataset. By extracting local features and global topological structure features of character images, the optimal character recognition model that can map character graphic data into character sequences and confidence scores is obtained.

[0056] Specifically, when training the character detection model, the iterative update process of the model parameters is controlled by giving the learning rate, batch size, and number of training epochs. The character detection model performs forward propagation on a dedicated detection dataset to obtain predicted character regions and calculates the localization loss between the predicted character regions and the corresponding labeled regions; then, backpropagation and parameter updates are performed based on the localization loss until the localization loss function converges and tends to stabilize, thus obtaining the optimal character detection model.

[0057] For a single-vision character recognition model, the training objective is to establish a mapping relationship between character image data and corresponding character sequences. The character recognition model extracts local features such as character strokes and textures, as well as global topological features such as character arrangement and overall structure from the character image, and outputs the corresponding character sequence and confidence score based on these features. When the model's training on a dedicated recognition dataset reaches the preset conditions, the optimal character recognition model is obtained.

[0058] Through the above training process, the character detection model is used to determine the location of character regions in the graphic data of drainage pipeline inspection records, and the character recognition model is used to read the graphic data in the determined character regions into character sequences and output the corresponding confidence scores, providing basic data for the aggregation and correction of character reading results in the same second time period.

[0059] During the reading of graphic character data from drainage pipeline inspection records, continuous inspection video recordings consist of a large number of continuous graphic data frames. To read these continuous records in segments, this embodiment employs a multi-threaded parallel approach to sample graphic data frames according to the time axis. By pre-dividing the processing threads into second intervals, graphic data from different time periods can be processed separately.

[0060] Specifically, step S300 includes: assigning a video second interval to each processing thread, each processing thread extracting graphic data frames at a preset frequency within its assigned second interval, and storing the extracted graphic data in association with its corresponding second-level time stamp.

[0061] Specifically, the timeline of the drainage pipe inspection video recording is divided into corresponding second-level time intervals, and corresponding time intervals are configured for different processing threads, so that each processing thread can complete the extraction of graphic data frames within the corresponding time range.

[0062] During the sampling process, each processing thread extracts graphic data frames at a preset frequency to obtain multiple graphic data frames within the same second time period, which are then used for subsequent character readability evaluation and character data reading.

[0063] At the same time, each graphic data frame is associated with its corresponding second-level time tag for storage, so that the character reading results obtained later can establish a correspondence with the time position in the inspection video recording, and provide a time index for the aggregation of character reading results within the same second-level time period and the time sequence verification of inspection record parameters between adjacent seconds.

[0064] Through the above processing, a data set consisting of graphic data frames and corresponding time tags can be formed according to the second-level time relationship, so that subsequent character detection, character reading and result aggregation can be executed in the time sequence of continuous inspection records.

[0065] Further, the specific steps for character readability evaluation in step S400 include: converting the graphic data frame into a grayscale image, obtaining the original edge image using a dual-threshold edge detection operator, and performing morphological closing operations on the original edge image using a transverse convolution kernel; performing connected component analysis on the processed edge image, calculating the area, width, height, and aspect ratio of each region, and filtering out candidate character regions; calculating the average grayscale difference between the candidate character regions and the adjacent background regions to obtain local contrast weights, and combining the number of candidate character regions and the sum of their widths to obtain the character readability evaluation score of the graphic data frame through weighted calculation.

[0066] Specifically, the input color graphic data frame is converted into a grayscale image. This conversion can be achieved by weighting the pixel values ​​of the red, green, and blue channels, for example, using the luminance Y component as the grayscale value. After grayscale conversion, the brightness variations and edge structures corresponding to the character area can be processed while reducing the influence of color components.

[0067] Further edge detection is performed on the grayscale image using a dual-threshold edge detection operator, such as the Canny algorithm. The original edge map is obtained through smoothing and denoising, gradient calculation, non-maximum suppression, and dual-threshold concatenation. Then, a lateral morphological closing operation is performed on the original edge map, sequentially performing dilation and erosion through a lateral convolution kernel to form continuous regions from adjacent or partially broken character edges in the horizontal direction. Connectivity analysis is then performed on the edge map after the closing operation to obtain each connected region, and the area, bounding rectangle width, height, and aspect ratio of each region are calculated. Character candidate regions are then selected based on preset ranges for area, width, height, and aspect ratio. The average grayscale difference between the character candidate region and the adjacent background region is then calculated to obtain a local contrast weight. This weight is combined with parameters such as the number of character candidate regions and the total width to calculate a character readability evaluation score, characterizing the degree to which the character data in the current graphic data frame is suitable for reading.

[0068] Through the above steps, the character readability of the graphic data frame is determined by the geometric structure of the character candidate region and the grayscale difference between it and the adjacent background region, so that the subsequent character detection model and character recognition model can mainly process the valid graphic data frames with corresponding character structure information.

[0069] After completing dataset construction, model training, sampling of inspection video recordings, and selection of valid graphic data frames, it is necessary to convert the character graphic data in the valid graphic data frames into corresponding character content. To this end, this embodiment first uses a character detection model to determine the location of the character region, then crops the character image based on the determined location, and finally uses a character recognition model to read the character sequence corresponding to the character image, thus establishing a correlation between the character region location, character content, and confidence level.

[0070] In response, step S500 includes: sequentially extracting graphic data frames from the valid graphic data frame sequence, using the optimal character detection model to infer the character region location set, and cropping the character image set from the original graphic data frame based on the character region location set; using the optimal character recognition model to perform inference reading on the character image set to obtain the character content and confidence level corresponding to each character image, and combining them to generate the inspection data processing result consisting of character region location, character content, and confidence level, until the processing of all valid graphic data frames within that second is completed.

[0071] Specifically, valid graphic data frames, filtered by character readability assessment, are read one by one in a preset order. The optimal character detection model is used to perform inference on the current graphic data frame to obtain the set of positions corresponding to each character region. Character region position information can be represented by the coordinates of the character region bounding box. Based on the character region positions, corresponding character images are cropped from the original graphic data frame, allowing the subsequent character recognition model to independently read the corresponding character regions. For tilted character images, orientation correction can be performed through affine or perspective transformations. Subsequently, the optimal character recognition model is used to perform inference on each character image to obtain the corresponding character content and confidence level, and the character region positions, character content, and confidence levels are correlated to form the inspection data processing result.

[0072] Through the above processing, the character reading process in the continuous drainage pipeline inspection graphic data is divided into character region positioning, character region clipping, and character content reading. The structured inspection data processing results are formed by character region position, character content, and confidence level, providing a data foundation for the subsequent aggregation of the same character regions between different graphic data frames.

[0073] Since a single second-level time period typically includes multiple consecutive graphic data frames, the same business character data may appear repeatedly in multiple graphic data frames. Due to factors such as the clarity of the graphic data frames, partial occlusion, and changes in the state of the character region, the character content and confidence level obtained for the same character region from different graphic data frames may differ. Therefore, it is necessary to classify the character reading results in different frames according to the spatial relationship of the character regions, and determine the corresponding reading results from those within the same category.

[0074] In response, step S600 performs the same type of region determination aggregation and optimal result selection, specifically including: calculating the intersection-union ratio (IUGR) between character regions obtained from different valid graphic data frames within the same second; if the IUGR is greater than the preset IUGR threshold, the corresponding character regions are determined to be the same type of region and assigned to the same set of the same type of region; in each set of the same type of region, the set of associated data groups with the longest character length is first selected, and the result with the highest average confidence is selected from the set of associated data groups as the set of associated data groups corresponding to that second.

[0075] Specifically, the Intersection over Union (IoU) ratio represents the degree of spatial overlap between the bounding boxes of two character regions. It is the ratio of the intersection area to the union area of ​​the two character region bounding boxes. By calculating the IoU ratio between corresponding character regions in different valid graphic data frames within the same second-level time period, it can be determined whether character regions in different graphic data frames correspond to the same display area in the inspection record. When the IoU ratio exceeds a preset threshold, the corresponding character regions are grouped into the same set of regions of the same type. Within the same set of regions of the same type, the character reading results obtained from different graphic data frames are compared. First, the associated data group with the longest character length is selected to reduce content loss caused by missed character readings. When multiple associated data groups with the same character length exist, the reading result with the highest average confidence is selected as the set of associated data groups for the corresponding second-level time period.

[0076] Through the above processing, a correspondence is established between character regions located in the same or similar spatial positions in continuous graphic data frames, and the set of associated data groups is determined from multiple associated data groups by character length and confidence level, thereby reducing the impact of repeated reading of the same character data and single-frame reading differences on the final output.

[0077] For numerical character data such as running distance and time that change continuously during the drainage pipeline inspection process, after the second-level results are aggregated, the associated data set can be verified based on the temporal relationship between adjacent second-level time periods to reduce abnormal numerical changes caused by local character misreading.

[0078] In response, step S600 further refines the optimal result, including: extracting numerical character information from the associated data set corresponding to adjacent second-level time periods, and determining whether the numerical character information conforms to the monotonically increasing logic regarding running distance or time in the drainage pipeline inspection record; if the numerical character information of the current second violates the monotonically increasing logic, then the associated data set of the current second is logically corrected according to the numerical change trend of the time sequence context.

[0079] Specifically, after obtaining the associated data sets for each second-level time period, numerical character information such as running distance and time is extracted from the corresponding character sequences. This process can be performed using preset character formats, regular expressions, or character regions corresponding to business fields. Subsequently, the numerical character information of the current second is compared with the numerical character information of the adjacent second-level time periods to determine whether the read running distance or time satisfies the increasing relationship corresponding to the continuous inspection record of the drainage pipeline.

[0080] When the numerical character information corresponding to the current second does not conform to the aforementioned temporal relationship, the correction result corresponding to the current second is determined based on the changing trends of the numerical character information corresponding to multiple time nodes before and after the current second. Specifically, interpolation of adjacent second values, curve fitting, or estimation based on the changing trends of values ​​within a preset time window can be used to ensure that the corrected character reading result remains continuous with the inspection record parameters corresponding to adjacent time periods.

[0081] Through the above timing verification and correction, the numerical character information such as running distance and time obtained from the continuous inspection record graphic data is established to correspond with its time sequence, reducing the numerical jumps caused by abnormal character reading of a single graphic data frame or a single second-level time period.

[0082] As a specific implementation method, step S100 employs an adaptive contrast rendering strategy to construct a dedicated detection dataset and a dedicated recognition dataset for the graphic character data of drainage pipeline inspection records, mainly including the following steps: Step 1.1: Obtain the characterless background image from the drainage pipeline inspection record, and transform the characterless background image according to a preset data augmentation strategy to obtain a quantity of... Background image collection The characterless background image is an image selected from the graphic data of the drainage pipeline inspection record that does not contain the business attribute characters to be read. It retains the background graphic features formed by the internal pipe wall texture, light distribution and actual collection environment of the drainage pipeline.

[0083] Step 1.2: Define the set of character parameters used to construct the graphic character data, including the character size set. Font collection and character color change operations The character size set and font set are determined based on the actual size and font style of the characters displayed in the drainage pipe inspection record, and the character color transformation operation... Used to determine the display color and edge envelope of the character to be written based on the background grayscale characteristics of the character preset writing area.

[0084] Step 1.3: Collect character data corresponding to drainage pipeline inspection operations and generate a character dictionary. and corpus set , These represent text sets for task addresses, pipe numbers, pipe materials, distance and speed, and date and time, respectively. The character dictionary is used to limit the range of characters output by the character recognition model, and the corpus is used to generate graphic character training samples with drainage pipeline inspection business attributes.

[0085] Step 1.4: Construct a dedicated detection dataset and dedicated recognition datasets According to the background image set 、 Character size set Font collection Special transformation operations f and corpus information The business attribute characters are written into a preset character region in the background image, and corresponding detection and recognition samples are generated based on the position of the written character region. The specific steps include: Step 1.4.1: From the corpus A text set is formed by randomly selecting one piece of text information from each of the five categories of text. texts Randomly from Select a background image from the image, and randomly choose one from the following: font Select a font from the list, and randomly choose one. Select font size fs Specify the position of various text boxes box According to the operation f Select the color of a single character, and then write the text character by character into the various text boxes in the background image to obtain a collection of text boxes. boxes Image fusion pic .gather boxes Includes five types of text boxes box , box The possible values ​​are as follows: ,in It is the anchor point position of the upper left corner of the text box. Its position is determined according to the character category of the business attribute. For example: (1) Task address type: (2) Pipeline numbering category: (3) Pipe Material Category: (4) Distance and speed categories: (5) Date and Time Categories: in, ac For a pre-defined fixed value, offset 1. offset 2. offset 3. offset 4 is a random offset within a pre-defined range; The text length is calculated using formula (1): in Given the text length; select the color of a single character as shown in formula (2): in, thresh Determine the threshold for a pre-defined character color. brightness It is the average brightness of the background area of ​​a single character, calculated by formula (3): r i It is the pixel ratio of each grayscale value. i ∈[0,255]: P yes H The sum of the number of pixels corresponding to all grayscale values: H It is a histogram of the grayscale image of the background area containing a single character. H [ i ] indicates that the grayscale value is i The number of pixels; Step 1.4.2: Based on the text box position information, process the merged image. pic The text box areas in the image are cropped separately to obtain the image. patch and corresponding text information text ; Step 1.4.3: Repeat step 1.4.1 n det Secondary dedicated detection dataset , To check the number of samples in the dataset; repeat steps 1.4.1 and 1.4.2. n rec Secondary dedicated recognition dataset , To identify the number of samples in the dataset; In addition, as a specific implementation, the data augmentation processing in step S100 also includes environmental degradation simulation. To simulate real physical disturbances to the drainage pipes, the system adds background image sets... Randomly injected fuzzy noise to simulate water mist distribution and random masks to simulate water splashing from the lens The degradation model of image synthesis follows the following logic: in, For non-uniform occlusion masks generated based on Perlin noise, This is the original background image. This augmentation method forces the model to learn character structural features in low signal-to-noise ratio environments.

[0086] Through the specific steps described above, the adaptive contrast rendering strategy and diverse data augmentation operations are organically combined in the dataset construction process. This process not only generates a large number of realistic training samples, but more importantly, it ensures the character recognizability of the samples in complex backgrounds, laying a solid foundation for subsequent model training.

[0087] As a specific implementation method, the training of the character detection model and the character recognition model in step S200 specifically includes the following steps: Step 2.1: Train the character detection model. This embodiment selects a character detection model based on Differentiable Binarization, and the training dataset is... Hyperparameters are given in advance. thresh Hyperparameters Learning rate Batch size and number of training rounds The training process includes the following steps: Step 2.1.1: From a dedicated detection dataset Extract an image from f ∈ R C×H×W The text threshold map is obtained by forward propagation in the Differentiable Binarization detection model. Text probability graph For text threshold maps Text probability graph The text binarization image is calculated using the differentiable binarization method. , ; Step 2.1.2: Binarize the text image The Vatti clipping algorithm is used to expand the text box information. Step 2.1.3: Calculate the forward propagation error value based on the loss function of the Differentiable Binarization character detection model, and then perform backpropagation based on the error; Step 2.1.4: Repeat steps 2.1.1 to 2.1.3 until completion. Next iteration: Obtain the optimal detection model ; Step 2.2: Train the single-vision character recognition model. This embodiment selects the SVTR (Single-Vision Character Recognition Model) method, and the training dataset is... The character dictionary is Hyperparameters are given in advance. Learning rate Batch size and number of training rounds epoch rec The training steps are as follows: Step 2.2.1: From the dataset Extract an image from f ∈ R C×H×W The input is fed into the SVTR character recognition model for forward propagation to obtain the predicted character sequence. CC ; Step 2.2.2: Calculate the forward propagation error value based on the loss function of the SVTR character recognition model, and then perform back propagation based on the error; Step 2.2.3: Repeat steps 2.2.1 to 2.2.2 until completion. Next iteration: Obtain the optimal recognition model .

[0088] The number of iterations in the entire training process is determined by the batch size and the number of training rounds. The goal is to minimize the model's loss function on the training set, thereby learning an effective mapping from complex images of drainage pipes to character positions and content.

[0089] As a specific implementation method, the task allocation logic for multi-threaded parallel sampling in step S300 is as follows: Step 3.1: For the input drainage pipe inspection video recordings V The video frame rate is fps First, sampling is performed. The sampling strategy is to extract a fixed number of video frames every second. The sampling process is carried out in parallel by multiple threads, with each thread responsible for processing the frame sampling task within a certain second of the video. After multi-threaded parallel sampling, a set of clear video frames is obtained. L frame : in Indicates the total number of seconds in the video. For the first j The set of frame numbers in seconds: in This indicates the number of frames extracted per second. As a specific implementation method, the character readability evaluation process in step S400 is as follows: Step 3.1.1: The system first samples the video frame images from the associated set. Convert to grayscale To eliminate color noise interference, the grayscale image was then processed using a double-threshold Canny operator. Edge detection is performed to extract high-frequency structural information from the image, resulting in the original edge map. : (10) in, and The thresholds are preset to a low threshold and a high threshold. This dual-threshold mechanism initially preserves strong edges of character strokes while suppressing weak light noise in the background.

[0090] Step 3.1.2: Due to the scattering of water mist within the drainage pipes, which may cause micro-fractures at the character edges, the system updates the original edge map. Perform lateral morphological closing operations. This is done by using a pre-defined lateral convolution kernel. Perform a dilation-erosion operation on the edge graph to enhance horizontal feature connectivity and obtain a connected edge graph. : (11) in, This represents the expansion operation. This represents the erosion operation. The aspect ratio is set according to the flattening characteristics of the drain pipe characters.

[0091] Step 3.1.3: For the connected edge graph Perform connected component analysis to extract the geometric statistics of each region. , representing the area, height, and width of the region, respectively. To separate character features from the pipe wall texture, the system sets filtering criteria based on the geometric distribution of the horizontal text: (12) in, , , These are preset thresholds for width, height, and aspect ratio. The system retains only the set of regions that meet these constraints. The mask is calculated according to the following formula to generate the filtered character region image. : (13) Step 3.1.4: To further streamline non-character edges, [the following steps are taken]. A refined image is obtained by performing a transverse erosion operation. : (14) in, This is a preset horizontally reduced convolutional kernel. Then, Compared with the original edge map Perform a bitwise AND operation to ensure that only pixels belonging to the core skeleton of the character are retained from the original edge, resulting in the final character edge map. : (15) Step 3.1.5: Based on the final character edge map extracted above The system not only evaluates the geometric integrity of characters, but also introduces a local contrast weighting factor. This is used to determine the recognizability of characters in the current context. The final quality assessment score is then used. The following formulas are used for comprehensive calculation: (16) in, It is a preset proportional coefficient; The ratio of edge pixels. For set Size, For set The sum of the widths of all connected components in the equation. For edge map Total number of pixels: (17) in, for The number of non-zero pixels. The key lies in the local contrast weight. This reflects the grayscale difference between the character and its adjacent background area: in, The average grayscale value of the candidate character region. The average gray value of the background in the eight neighboring areas surrounding this region. Standard deviation of the background The smoothing factor. This evaluation mechanism ensures that even if the video experiences global background blur due to robot motion (at this time...) (Overall features are relatively weak), only the contrast weight of the character preset area is needed. If it is high enough, meaning the characters are still clearly distinguishable in the local background, then It can still exceed the preset threshold. The system only retains... The frames are then included in the valid video frame sequence. As an alternative embodiment, those skilled in the art may also use mutual information entropy or local contrast gain value as... Evaluation indicators.

[0092] As a specific implementation method, the process of performing character detection and character reading frame by frame and generating inspection data processing results in step S500 is as follows: Step 3.2.1: Take out the sets one by one L frame A frame of image is extracted and removed from the set, using the optimal character detection model. By performing inference, the set of character region locations in the current graphics data frame is obtained. , This represents the number of text boxes in a frame of an image. The corresponding character graphic regions are cropped based on their positions to obtain a set of character images. ; Step 3.2.2: For the text line image set Use the optimal recognition model Perform inference step by step to obtain the character content and the corresponding confidence set. , and Indicates the first p The set is obtained by combining the characters and confidence scores output by the recognition model from each text line image, and by comprehensively detecting and recognizing the results. ; Repeat steps 3.2.1 and 3.2.2, after each continuous process. After capturing a frame, the detection and recognition of the video content for a given second are completed, and then the data is combined to obtain the inspection candidate data set for that second. Among them, the triplet Indicates the number of seconds within a certain second q The first frame p The location of each character region, its content, and its confidence level; until... L frame If the set is empty, obtain the set of inspection candidate data for all seconds of the video. ; Step 3.3.1: The system first retrieves the inspection candidate data set for all seconds of the video. a subset (i.e., all results within a given second), perform the aggregation operation on similar boxes. Specifically, create a new empty collection. and from the set Extract the set of text boxes from the first frame. The set is initialized accordingly. and will from Remove from the middle. At this time... Classified as Subsets: .

[0093] Subsequently, the system from Extract the set of text boxes from the subsequent frames and delete them. For each text box in the set, sequentially... The first text box in each subset is compared to the first text box in the subset. If it meets the criteria for being a classmate, it is added to the corresponding subset; otherwise, it is added to the subset. Create a new subset and add the text box. Repeat this process until the subset is complete. If empty, the final result is a set of similar boxes. ,in Indicates the first A collection of similar boxes, express The number of subsets. The logic for determining whether they are of the same type is shown in formulas (18, (19), (20), and (21): in, This indicates the area of ​​the text box. This is the area of ​​the intersection of the two text boxes. For the area of ​​the union, For the intersection operation, For the union operation, This represents the preset intersection-union ratio threshold for determining whether two boxes belong to the same type.

[0094] Step 3.3.2: In the collection of similar boxes In the process, the system selects the optimal recognition result within that second through a multi-criteria filtering mechanism. To ensure the completeness and accuracy of the extracted information, the selection rules follow the principle of "length first, confidence second best", as shown in formula (23): (twenty three) in, Indicates the first The optimal text information result in each type of box. This represents retrieving the longest subset of text information from the set. This indicates the result with the highest average confidence level obtained from this subset.

[0095] Step 3.3.3: After completing the optimization of the spatial dimensions described above, in order to further eliminate numerical recognition errors caused by extreme interference, this embodiment introduces a temporal logic correction mechanism for drainage pipeline inspection operations. The system extracts the optimal results corresponding to adjacent second-level time intervals. and Numerical features (such as running distance) or time Determine whether it conforms to the monotonically increasing logic: (twenty four) If the judgment result is This indicates the recognition result for the current second. This violates the physical laws governing the movement of drainage inspection equipment (i.e., the mileage cannot be reversed). At this point, the system performs logical corrections to the result for the current second based on the numerical trend within the time context: (25) in, The average inspection speed within the preset window period. The time step for sampling video frames. Repeat steps 3.3.1 to 3.3.3 above until the set is complete. If empty, the final output will be a set of video information extraction results after logical smoothing. As an alternative embodiment, those skilled in the art can also use Kalman filtering or median smoothing algorithms to... The sequence undergoes further denoising processing.

[0096] This embodiment evaluates the character readability of graphic data frames in drainage pipeline inspection records, selects those that meet the requirements for character structure and local grayscale difference for character reading, and aggregates the character reading results corresponding to different graphic data frames within the same second time period by analyzing the spatial position relationship of character regions. Then, it verifies and corrects continuous numerical character data by combining the temporal order relationship of inspection record parameters, thereby obtaining graphic character data reading results corresponding to continuous drainage pipeline inspection records.

[0097] Furthermore, this embodiment of the invention provides a drainage pipeline inspection data processing system based on the fusion of temporal correlation and reliability, the structural schematic diagram of which is shown below. Figure 2 As shown, the system includes a dataset construction module 100, a model training module 200, a multi-threaded sampling module 300, a local weight evaluation module 400, a detection and recognition processing module 500, and a temporal aggregation correction module 600.

[0098] The dataset construction module 100 constructs a dedicated detection dataset and a dedicated recognition dataset for graphic character data in drainage pipeline inspection records based on the background grayscale distribution characteristics of the character preset area and the simulated environmental noise. The model training module 200 uses the dedicated dataset to train and obtain the optimal character detection model and the optimal character recognition model. The multi-threaded sampling module 300 performs parallel sampling on the drainage pipeline inspection video recordings in segments of one second to obtain an associated set consisting of graphic data frames and second-level time labels. The local weight evaluation module 400 evaluates the readability of characters by combining the local contrast weight of the character preset area and the edge geometric features, and removes graphic data frames with evaluation scores below a preset threshold to obtain a valid graphic data frame sequence. The detection and recognition processing module 500 performs frame-by-frame detection, cropping, and character reading on the valid graphic data frame sequence to generate an inspection candidate data set consisting of character area position, character content, and confidence level. The temporal aggregation and correction module 600 performs similar area judgment on a second-by-second basis and corrects the reading results by combining the temporal monotonically increasing logic of the inspection record parameters, outputting the final drainage pipeline inspection data processing result.

[0099] Through the above system, each module sequentially completes the construction of training data for graphic character data in drainage pipeline inspection records, training of character region detection models and character recognition models, sampling of continuous graphic data frames, character readability evaluation, character content reading, and aggregation and correction of reading results. This enables the graphic data in drainage pipeline inspection video records to be converted into reading results with character content, position, and time correspondence, improving the integrity, stability, and continuity of graphic character data reading in complex inspection environments.

[0100] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for processing drainage pipeline inspection data based on the fusion of temporal correlation and reliability, characterized in that, Includes the following steps: S100: Obtain characterless background images from drainage pipeline inspection records and perform data augmentation processing. Combined with a corpus containing business attribute characters, construct a dedicated detection dataset and a dedicated recognition dataset for graphic character data in drainage pipeline inspection records using an adaptive contrast rendering strategy. S200: Set training hyperparameters, train a character detection model based on the differentiable binarization algorithm using the dedicated detection dataset, and train a single-vision character recognition model using the dedicated recognition dataset, to obtain the optimal character detection model and the optimal character recognition model respectively. S300: Performs second-by-second segmentation processing on the input drainage pipe inspection video recording according to the preset timeline, and allocates multiple processing threads to perform interval sampling on the image data within each second in parallel, and obtains the associated set composed of the extracted graphic data frames and second-level time tags; S400: For each graphic data frame in the associated set, locate the character preset region and calculate the local contrast weight of the character preset region relative to the adjacent background region. Combine the character edge geometric features to evaluate character readability. Remove graphic data frames with character readability evaluation scores lower than a preset threshold to obtain an effective graphic data frame sequence. S5 00: Using the optimal character detection model and the optimal character recognition model, perform frame-by-frame detection and reading on the effective graphic data frame sequence to generate a second-level inspection candidate data set containing character region positions, character content, and confidence levels; S600: Perform similar area determination aggregation based on spatial intersection-union ratio on the second-level inspection candidate data set, and combine the temporal logic characteristics of the drainage pipeline inspection record parameters to perform credibility optimization and temporal correction, and output the drainage pipeline inspection data processing results.

2. The drainage pipeline inspection data processing method based on time-series correlation and reliability fusion according to claim 1, characterized in that, The adaptive contrast rendering strategy in step S100 specifically includes: Statistically analyze the grayscale distribution characteristics of the preset writing area of ​​characters in the characterless background image; If the grayscale distribution feature value is higher than the preset brightness threshold, then the character to be rendered is assigned a first color, and a high-contrast inverted edge envelope is added to the character to be rendered. If the grayscale distribution feature value is lower than or equal to the preset brightness threshold, then the character to be rendered is assigned a second color that contrasts with the first color.

3. The drainage pipeline inspection data processing method based on time-series correlation and reliability fusion according to claim 1, characterized in that, The data augmentation process performed in step S100 specifically includes: Perform at least one of the following operations on the characterless background image: random horizontal flipping, random rotation, random scaling, random cropping, color space transformation, and adding Gaussian noise, to obtain a set of background images; Randomly inject gray-white water mist and lens splash masking elements simulating a drainage pipe inspection environment into the background image set to enhance the feature extraction capability of the character recognition model in scenes where the character and background colors are similar.

4. The drainage pipeline inspection data processing method based on time-series correlation and reliability fusion according to claim 1, characterized in that, Step S200 includes: Given hyperparameters including learning rate, batch size and number of training rounds, the dedicated detection dataset is input into the character detection model for training. The optimal character detection model is obtained by forward propagation, error calculation and back-up update until the localization loss function converges. The single-vision character recognition model is trained using the dedicated recognition dataset. By extracting local features and global topological features of character images, the optimal character recognition model that can map character graphic data into character sequences and confidence scores is obtained.

5. The drainage pipeline inspection data processing method based on time-series correlation and reliability fusion according to claim 1, characterized in that, The specific steps of S300 are as follows: Each processing thread is assigned a video time interval to process. Within its assigned time interval, each processing thread extracts graphic data frames at a preset frequency and stores the extracted graphic data in association with its corresponding second-level timestamp.

6. The drainage pipeline inspection data processing method based on time-series correlation and reliability fusion according to claim 1, characterized in that, The specific steps for character readability assessment in step S400 include: The graphic data frame is converted into a grayscale image, the original edge image is obtained by using a double threshold edge detection operator, and the original edge image is processed by morphological closing operation using a transverse convolution kernel. Perform connected component analysis on the processed edge graph, calculate the area, width, height and aspect ratio of each region, and filter out candidate character regions; The average grayscale difference between the character candidate region and the adjacent background region is calculated to obtain the local contrast weight. Combined with the number and total width of the character candidate regions, the character readability evaluation score of the graphic data frame is obtained through weighted calculation.

7. The drainage pipeline inspection data processing method based on time-series correlation and reliability fusion according to claim 1, characterized in that, Step S500 specifically includes: The graphic data frames in the effective graphic data frame sequence are extracted sequentially, and the character region position set is obtained by reasoning using the optimal character detection model. The character image set is then cropped from the original graphic data frame based on the character region position set. The optimal character recognition model is used to perform inference reading on the character image set to obtain the character content and confidence level corresponding to each character image. The results are combined to generate the inspection data processing result consisting of the character region position, the character content and the confidence level, until the processing of all valid graphic data frames within the second is completed.

8. The drainage pipeline inspection data processing method based on time-series correlation and reliability fusion according to claim 1, characterized in that, The specific steps of performing similar region determination aggregation and optimal result selection in step S600 include: Calculate the intersection-union ratio (IUGR) between character regions obtained from different valid graphic data frames within the same second. If the IUGR is greater than a preset IUGR threshold, the corresponding character regions are determined to be of the same type and are assigned to the same set of regions of the same type. In each set of similar regions, the set of associated data groups with the longest character length is first selected, and the result with the highest average confidence level is selected from the set of associated data groups as the set of associated data groups corresponding to that second.

9. The drainage pipeline inspection data processing method based on time-series correlation and reliability fusion according to claim 8, characterized in that, The optimal result correction in step S600 specifically includes: Extract the numerical character information from the associated data set corresponding to adjacent second-level time intervals, and determine whether the numerical character information conforms to the monotonically increasing logic of the running distance or time in the drainage pipeline inspection record; If the numerical character information of the current second violates the monotonically increasing logic, then the associated data set of the current second is logically corrected according to the numerical change trend of the time sequence context.

10. A drainage pipeline inspection data processing system based on temporal correlation and reliability fusion, characterized in that, include: The dataset construction module is used to construct a dedicated detection dataset and a dedicated recognition dataset for graphic character data of drainage pipeline inspection records based on the background grayscale distribution characteristics of the character preset area and the noise of the simulated environment. The model training module is used to train and obtain the optimal character detection model and the optimal character recognition model using the dedicated dataset. The multi-threaded sampling module is used to perform parallel sampling of drainage pipeline inspection video recordings in segments of one second, in order to obtain an associated set consisting of graphic data frames and second-level time tags; The local weight evaluation module is used to evaluate the readability of characters by combining the local contrast weight of the preset region of the character with the edge geometric features, and to remove graphic data frames with evaluation scores below a preset threshold to obtain a valid graphic data frame sequence. The detection and recognition processing module is used to perform frame-by-frame detection, cropping and character reading on the effective graphic data frame sequence to generate a set of inspection candidate data; The time-series aggregation correction module is used to perform the same area judgment on a second-by-second basis, and output the final drainage pipeline inspection data processing result by combining the time-series monotonically increasing logic of the inspection record parameters.

Images