Line electronic standing book defect positioning method and device

Abstract

The invention discloses a line electronic standing book defect positioning method and device. The method comprises the steps that a multi-mode line defect recognition result and a line electronic standing book constructed through machine vision in advance are acquired; converting line defect identification results of different modes into corresponding feature vectors through a feature embedding representation network; the feature vector and facility data in the electronic standing book are input into a two-way mapping neural network together, mapping of the feature vector and standing book space is carried out through a multi-head attention mechanism and feature fusion in the two-way mapping neural network, and pixel coordinates corresponding to the defect position in an accumulated pixel coordinate system are output; and based on the pixel coordinates, adopting a nearest neighbor matching algorithm to position the defect position to the line facility with the matched distance in the line electronic standing book. The method is used for improving the efficiency and accuracy of railway line facility maintenance management.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of railway engineering and intelligent operation and maintenance technology, and particularly relates to a line electronic account defect positioning method and device. BACKGROUND

[0002] In recent years, railway infrastructure detection technology has developed rapidly, but traditional positioning technology is limited to meter-level precision and cannot meet the needs of facility-level fine maintenance. Although the existing technology such as the line facility account data processing method and device realizes machine vision-based account data acquisition, it only stays at the data construction stage and lacks a mechanism for intelligently associating detected defects with account facilities, which cannot solve the problem of accurate positioning in the field maintenance. The specific performance is that the application scene is limited, the positioning accuracy is insufficient, the data correlation is weak, and the intelligent level is limited, resulting in low maintenance efficiency and large consumption of human resources. SUMMARY

[0003] The line electronic account defect positioning method provided by the embodiments of the present application realizes accurate positioning of line defects from multi-modal recognition to facility level, improves the efficiency and accuracy of railway line facility maintenance management, and the method comprises the following steps:

[0004] Obtain multi-modal line defect recognition results and a line electronic account constructed in advance through machine vision; the line electronic account uses accumulated pixel coordinates of track inspection images as a unified base coordinate system;

[0005] Convert the line defect recognition results of different modalities into corresponding feature vectors through a feature embedding representation network;

[0006] Input the feature vectors and facility data in the electronic account into a double-path mapping neural network, and perform mapping of the feature vectors and the account space through a multi-head attention mechanism and feature fusion in the double-path mapping neural network, and output pixel coordinates of the defect position in the accumulated pixel coordinate system;

[0007] Based on the pixel coordinates, use a nearest neighbor matching algorithm to position the defect position to the line facility in the line electronic account that is closest to the matched line facility.

[0008] The line electronic account defect positioning device provided by the embodiments of the present application realizes accurate positioning of line defects from multi-modal recognition to facility level, improves the efficiency and accuracy of railway line facility maintenance management, and the device comprises the following steps:

[0009] A data acquisition module is configured to obtain multi-modal line defect recognition results and a line electronic account constructed in advance through machine vision; the line electronic account uses accumulated pixel coordinates of track inspection images as a unified base coordinate system;

[0010] The feature vector conversion module is configured to convert the line defect recognition results of different modalities into corresponding feature vectors through a feature embedding representation network.

[0011] The pixel coordinate output module is configured to input the feature vectors and facility data in the electronic account into a double-path mapping neural network, and perform mapping of the feature vectors and the account space through a multi-head attention mechanism and feature fusion in the double-path mapping neural network, and output pixel coordinates corresponding to the defect positions in a cumulative pixel coordinate system.

[0012] The defect position positioning module is configured to position the defect positions to the line facilities in the line electronic account that are closest to the matched line facilities based on the pixel coordinates and by using a nearest neighbor matching algorithm.

[0013] The embodiment of the present application further provides a computer device, which comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor, and the processor implements the line electronic account defect positioning method when executing the computer program.

[0014] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the line electronic account defect positioning method.

[0015] The embodiment of the present application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the line electronic account defect positioning method.

[0016] In the embodiment of the present application, the line defect recognition results of multiple modalities and the line electronic account constructed in advance through machine vision are obtained; the line electronic account adopts accumulated pixel coordinates of track inspection images as a unified base coordinate system; the line defect recognition results of different modalities are converted into corresponding feature vectors through a feature embedding representation network; the feature vectors and facility data in the electronic account are jointly input into a double-path mapping neural network, and the mapping of the feature vectors and the account space is performed through a multi-head attention mechanism and feature fusion in the double-path mapping neural network, and pixel coordinates corresponding to the defect position in the accumulated pixel coordinate system are output; based on the pixel coordinates, the nearest neighbor matching algorithm is used to locate the defect position to the line facility in the line electronic account that is closest to the matching position. In the embodiment of the present application, the multi-modal defect recognition results and the electronic account with accumulated pixel coordinates as the unified reference are obtained, the heterogeneous defect data is converted into unified feature vectors by using the feature embedding representation network, the accurate mapping of the defect features and the account space is realized through the multi-head attention mechanism and the feature fusion strategy in the double-path mapping neural network, the pixel coordinates of the defect in the accumulated pixel coordinate system are output, and the defect is located to the specific facility based on the nearest neighbor matching algorithm, thereby effectively breaking through the limitation of the traditional meter-level positioning, realizing the end-to-end accurate positioning from the multi-modal data to the facility level, significantly improving the precision and efficiency of the defect positioning, reducing the manpower and time cost of on-site investigation, and enhancing the intelligent and fine level of maintenance management. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor. In the drawings:

[0018] Figure 1 It is a flowchart of a line electronic account defect positioning method in the embodiment of the present application;

[0019] Figure 2 It is a specific example diagram of a line electronic account defect positioning method in the embodiment of the present application;

[0020] Figure 3 It is a specific example diagram of data processing of a multi-modal feature embedding representation network in the embodiment of the present application;

[0021] Figure 4 It is a specific example diagram of the specific architecture flow of a double-path mapping neural network in the embodiment of the present application;

[0022] Figure 5This is a schematic diagram of the structure of a line electronic ledger defect location device according to an embodiment of the present invention;

[0023] Figure 6 This is a schematic diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0025] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0026] In the description of this specification, the terms "comprising," "including," "having," and "containing" are open-ended terms, meaning that they include but are not limited to. The terms "an embodiment," "a specific embodiment," "some embodiments," and "for example," etc., refer to specific features, structures, or characteristics described in connection with that embodiment or example that are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. The order of steps involved in the various embodiments is used to illustrate the implementation of this application, and the order of steps is not limited and can be adjusted appropriately as needed.

[0027] The acquisition, storage, use, and processing of data in this application comply with relevant regulations. The information collected in this application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant standards, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding operation interfaces are provided for users to choose to authorize or refuse.

[0028] It should be noted that in the embodiments of this application, certain existing solutions in the industry, such as software, components, and models, may be mentioned. For example, some existing software tools, components, algorithm models, or solutions well-known in other technical fields may be cited. These should be considered exemplary, and their purpose is only to illustrate the feasibility of implementing the technical solution of this application. These mentions should be understood as typical examples, and their core purpose is to illustrate and verify the rationality and feasibility of implementing the technical solution proposed in this application. However, this does not mean that the applicant has already used or necessarily used the solution. Such citations do not imply that the applicant has actually adopted these existing solutions, or that it will necessarily adopt these methods in its technical implementation process in the future. In other words, these mentions are only illustrative in nature, helping to understand the connection and transcendence of the innovation points of this application with the prior art, and do not constitute an endorsement or reliance statement on a specific prior art product.

[0029] In recent years, with the rapid development and innovation of railway infrastructure inspection technology, traditional positioning technology, due to its inherent limitations of meter-level accuracy, has become insufficient to meet the refined requirements of modern railway maintenance and management. This lack of accuracy is particularly prominent in the investigation and verification of track facility defects, becoming a key bottleneck restricting efficient maintenance and precise management of railway infrastructure. Specifically, this manifests in the following aspects:

[0030] 1. In terms of maintenance efficiency: Traditional positioning methods force on-site maintenance teams to invest a lot of manpower in detailed investigations, which significantly increases operation time and labor costs;

[0031] 2. Response timeliness: The inability to quickly and accurately locate specific faulty facilities delays the fault response and repair process, affecting the timeliness of maintenance work;

[0032] 3. In terms of management precision, meter-level positioning accuracy is insufficient to meet the requirements of facility-level precise positioning, which restricts the upgrading of railway infrastructure maintenance and management towards refinement; even if the positioning accuracy meets the standards, it is still difficult to quickly locate the accurate position based on mileage during on-site verification and maintenance.

[0033] For example, current solutions for processing railway infrastructure ledger data mostly focus on the construction process of electronic ledgers. Although machine vision-based ledger data collection and processing have been achieved, they have the following main limitations:

[0034] 1. Limitations in application scenarios:

[0035] It only focuses on the collection and storage of ledger data, lacking a mechanism for in-depth application of ledger data;

[0036] The value of ledger data in line facility maintenance and management was not fully utilized;

[0037] There is a lack of application solutions that correlate with actual test data;

[0038] 2. Positioning accuracy limitations:

[0039] No mapping mechanism has been established between testing data and record data;

[0040] There is a lack of technical solutions to precisely pinpoint the defects detected by testing to specific facilities;

[0041] It cannot meet the urgent need for precise positioning in on-site maintenance;

[0042] 3. Insufficient data correlation:

[0043] Intelligent association between detection data from different modalities and ledger data has not been achieved;

[0044] The lack of a unified feature representation and coordinate mapping framework;

[0045] It is difficult to support the comprehensive application of multi-source detection data;

[0046] 4. Limited level of intelligence:

[0047] It only focuses on data collection and storage, lacking intelligent application scenarios;

[0048] Failed to fully utilize deep learning technology to achieve intelligent data matching and localization;

[0049] There is a lack of intelligent solutions tailored to actual maintenance needs.

[0050] In contrast, this invention, building upon previous technologies, focuses on addressing the application of ledger data. By innovatively proposing a unified multimodal defect location framework, it achieves intelligent correlation between detection data and ledger data, overcoming the accuracy limitations of traditional location technologies and providing a complete technical solution for the refined maintenance of line facilities. This technological extension and innovation from "establishment" to "application" significantly enhances the application value of electronic ledgers in practical maintenance management.

[0051] To overcome the aforementioned technical bottlenecks, this invention proposes a machine vision-based electronic railway track ledger defect facility-level positioning scheme. This scheme intelligently aligns the detected defect data with the electronic railway track ledger constructed using machine vision, achieving precise positioning of the defect location down to the specific facility. This innovative solution not only overcomes the accuracy limitations of traditional positioning technologies but also provides reliable technical support for the refined management of railway infrastructure. Its application value is mainly reflected in:

[0052] 1. Improved the accuracy of disease location, achieving a qualitative leap from "meter-level" to "facility-level";

[0053] 2. During the review and repair process, the area to be repaired can be quickly located on-site, which significantly improves the efficiency of testing and repair work and reduces the input of human resources;

[0054] 3. It provides strong support for the scientific maintenance and safe operation of railway infrastructure.

[0055] Specifically, Figure 1 This is a flowchart illustrating a method for locating defects in an electronic railway track ledger according to an embodiment of the present invention. The embodiment proposes a machine vision-based facility-level defect location scheme for electronic railway track ledgers. By constructing a machine vision-based electronic railway track ledger and establishing an intelligent alignment mechanism between detection data and facility information, precise location of defects to specific facilities is achieved. This effectively overcomes the limitations of traditional location technologies, significantly improves the intelligence and precision of railway track facility maintenance, and enables precise location of track defects from multimodal identification to facility-level accuracy, thereby improving the efficiency and accuracy of railway track facility maintenance management. See [link to relevant documentation]. Figure 1 The method may include:

[0056] Step 101: Obtain the multimodal track defect identification results and the electronic track ledger pre-constructed by machine vision; the electronic track ledger uses the cumulative pixel coordinates of the track inspection images as a unified basic coordinate system;

[0057] Step 102: Convert the line defect identification results of different modalities into corresponding feature vectors through a feature embedding representation network;

[0058] Step 103: Input the feature vector and the facility data in the electronic ledger into the dual-path mapping neural network. Through the multi-head attention mechanism and feature fusion in the dual-path mapping neural network, map the feature vector to the ledger space and output the pixel coordinates of the defect location in the cumulative pixel coordinate system.

[0059] Step 104: Based on the pixel coordinates, the nearest neighbor matching algorithm is used to locate the defect location to the line facility in the line electronic ledger that matches the distance.

[0060] In this embodiment of the invention, multimodal line defect identification results and a pre-constructed electronic ledger of the line using machine vision are obtained. The electronic ledger uses the cumulative pixel coordinates of track inspection images as a unified basic coordinate system. The line defect identification results of different modalities are converted into corresponding feature vectors through a feature embedding representation network. The feature vectors and facility data in the electronic ledger are input into a dual-path mapping neural network. Through the multi-head attention mechanism and feature fusion in the dual-path mapping neural network, the feature vectors are mapped to the ledger space, and the pixel coordinates of the defect location in the cumulative pixel coordinate system are output. Based on the pixel coordinates, a nearest neighbor matching algorithm is used to locate the defect location to a line facility in the electronic ledger that matches the distance. This invention, through acquiring multimodal defect identification results and an electronic ledger based on accumulated pixel coordinates, transforms heterogeneous defect data into a unified feature vector using a feature embedding representation network. Then, through a multi-head attention mechanism and feature fusion strategy in a dual-path mapping neural network, it achieves precise mapping between defect features and the ledger space, outputting the pixel coordinates of the defect in the accumulated pixel coordinate system. Based on a nearest neighbor matching algorithm, the defect is located to a specific facility, effectively overcoming the limitations of traditional meter-level positioning. This achieves end-to-end precise positioning from multimodal data to the facility level, significantly improving the accuracy and efficiency of defect location, reducing the manpower and time costs of on-site inspections, and enhancing the intelligence and refinement of maintenance management.

[0061] In specific implementation, the first step is to obtain multimodal line defect identification results and a pre-constructed electronic ledger of the line using machine vision; the electronic ledger of the line uses the cumulative pixel coordinates of the track inspection images as a unified basic coordinate system.

[0062] In one embodiment, the method further includes: constructing an electronic ledger of the route using machine vision in the following manner:

[0063] Acquire images of line facilities captured by a line scan camera along the target line;

[0064] The image is input into a target detection model for railway facility images based on an optimized YOLOv5 network structure to output the target detection area; OCR technology is used within the target detection area to extract text information and determine the ledger data of the railway facility, and the facility location in the ledger is associated with the cumulative pixel coordinates of the track inspection image; an electronic ledger of the railway is constructed with the cumulative pixel coordinates as a unified basic coordinate system.

[0065] In the above embodiments, the process of constructing an electronic ledger for railway lines using machine vision specifically includes the following steps: First, during the inspection of the target railway line, images of railway facilities are acquired using a line scan camera. Then, the acquired images are input into a target detection model for railway facility images based on an optimized YOLOv5 network structure. This model outputs the target detection area of ​​the railway facility. Within the target detection area, optical character recognition technology is used to extract text information from the image, and the ledger data of the railway facility is determined based on the text information. Next, the ledger data is associated with the cumulative pixel coordinates of the track inspection images to achieve pixel-level precise positioning and digital management of typical facilities such as sleepers and supports. Finally, the associated ledger data is uniformly stored in a pre-constructed database, thereby forming an electronic ledger for railway lines with cumulative pixel coordinates as a unified basic coordinate system, providing a high-precision spatial reference system for defect location.

[0066] In specific implementation, after step 101: obtaining multimodal line defect identification results and a pre-constructed electronic ledger of the line using machine vision; and after the electronic ledger of the line uses the cumulative pixel coordinates of the track inspection image as a unified basic coordinate system, step 102: converting the line defect identification results of different modes into corresponding feature vectors through a feature embedding representation network.

[0067] In one embodiment, the multimodal line defect identification result includes defect data in image mode, waveform mode, and point cloud mode;

[0068] The results of line defect identification in different modalities are converted into corresponding feature vectors through a feature embedding representation network, including:

[0069] For defect data in image modalities, an encoder containing a multi-layer convolutional structure is used for feature extraction;

[0070] For defective waveform modal data, an encoder containing long short-term memory units is used to extract timing features;

[0071] For defective point cloud modal data, an encoder based on a point cloud network structure is used to extract spatial geometric features;

[0072] The feature embedding representation network maps the defect data of the corresponding modality to a feature space of the same dimension. Then, the extracted features are fused and weighted by a module containing a multi-head attention mechanism, and the corresponding feature vector is output through the embedding transformation layer.

[0073] In the above embodiments, the multimodal line defect identification results include defect data in image mode, waveform mode, and point cloud mode. When these different modal defect identification results are converted into corresponding feature vectors through a feature embedding representation network, for the image mode defect data, an encoder containing a multi-layer convolutional structure is used for feature extraction. This encoder captures spatial features in the image through convolutional layers and pooling operations. For the waveform mode defect data, an encoder containing long short-term memory units is used to extract temporal features, and a recurrent neural network is used to capture the dynamic changes of the waveform data. For the point cloud mode defect data, an encoder based on a point cloud network structure is used to extract spatial geometric features, and point cloud processing technology is used to identify three-dimensional structural information.

[0074] Subsequently, the feature embedding representation network maps the defect data of each modality to a feature space of a unified dimension. Then, the extracted features are fused and weighted by a module containing a multi-head attention mechanism. This mechanism adaptively learns the importance of different features and realizes feature interaction. Finally, after processing by the embedding transformation layer, including a fully connected layer with activation functions and normalization operations, a standardized feature vector is output.

[0075] In specific implementation, after step 102: converting the line defect identification results of different modes into corresponding feature vectors through a feature embedding representation network, step 103: inputting the feature vectors and facility data in the electronic ledger into a dual-path mapping neural network, and mapping the feature vectors to the ledger space through the multi-head attention mechanism and feature fusion in the dual-path mapping neural network, and outputting the pixel coordinates of the defect location in the cumulative pixel coordinate system.

[0076] In one embodiment, the feature vector and facility data from the electronic ledger are jointly input into a dual-path mapping neural network. Through the multi-head attention mechanism and feature fusion in the dual-path mapping neural network, the feature vector is mapped to the ledger space, and the pixel coordinates corresponding to the defect location in the cumulative pixel coordinate system are output, including:

[0077] The standardized feature vector and the electronic ledger vector containing facility attributes are respectively input into multiple independent input branches of the neural network; wherein each branch performs feature transformation through a fully connected layer containing batch normalization and activation function, so that the two features are aligned in the same semantic space.

[0078] The feature is adaptively weighted by a multi-head attention mechanism, and the two feature paths are fused by feature concatenation, cross-attention mechanism and gating fusion strategy.

[0079] The pixel coordinates of the defect in the cumulative pixel coordinate system are obtained by mapping through the output layer.

[0080] In the above embodiment, the specific process of inputting the standardized feature vector and facility data from the electronic ledger into the dual-path mapping neural network is as follows. First, the standardized feature vector and the electronic ledger vector containing key facility attributes are respectively input into two independent input branches of the neural network. Each branch undergoes a nonlinear transformation through a fully connected layer containing a batch normalization layer and an activation function, making the two features more consistent in dimensionality and distribution, achieving feature alignment in the same semantic space. Subsequently, a multi-head attention mechanism is used to adaptively assign weights to the two features and calculate the importance scores of different feature dimensions. Next, a feature concatenation operation is used to preserve the original feature information, and a cross-attention mechanism is used to achieve deep interaction between the two features. Finally, a gating fusion strategy is used to dynamically adjust the contribution of different feature streams to complete the full fusion of features. The fused features are linearly transformed by the output layer and mapped to the precise pixel coordinates of defects in the cumulative pixel coordinate system, thereby completing the transformation from feature space to coordinate space.

[0081] In specific implementation, after step 103: inputting the feature vector and facility data in the electronic ledger into the dual-path mapping neural network, and mapping the feature vector to the ledger space through the multi-head attention mechanism and feature fusion in the dual-path mapping neural network, and outputting the pixel coordinates of the defect location in the cumulative pixel coordinate system, step 104: based on the pixel coordinates, using the nearest neighbor matching algorithm, locating the defect location to the line facility in the line electronic ledger that matches the distance.

[0082] In one embodiment, based on the pixel coordinates, a nearest neighbor matching algorithm is used to locate the defect location to a matching line facility in the electronic line ledger, including:

[0083] The area of ​​each facility is pre-marked in the electronic ledger of machine vision lines and represented as a set of area coordinates based on cumulative pixel coordinates;

[0084] Calculate the minimum distance from the pixel coordinates of the defect location to the set of coordinates of each facility area in the electronic ledger;

[0085] Based on multiple minimum distances, the defect location is located to the designated line facility with the minimum distance, establishing a precise correspondence between the defect location and the specific facility.

[0086] In the above embodiments, the process of locating the defect location to a matching line facility in the electronic line ledger based on the pixel coordinates using a nearest neighbor matching algorithm specifically includes the following steps. First, the area of ​​each facility is pre-marked in the machine vision electronic line ledger, represented as a set of area coordinates based on accumulated pixel coordinates. Then, the minimum distance from the pixel coordinates of the defect location to the coordinate sets of each facility area in the electronic ledger is calculated, using Euclidean distance as a metric. By comparing the minimum distances from the defect location to all facility areas, the defect location is located to the specific line facility with which it has the minimum distance, thereby establishing a precise correspondence between the defect location and the specific facility. This matching strategy not only considers the spatial distribution characteristics of facilities but also effectively handles the relationship between the defect location and facility boundaries, ensuring the accuracy and reliability of the location results, while providing reliable data support for subsequent maintenance decisions and management.

[0087] The following is a specific embodiment to illustrate the specific application of the method of the present invention.

[0088] This invention proposes a defect facility-level localization method based on a machine vision-based electronic track ledger, using a pre-constructed machine vision-based electronic track ledger as its foundation platform. This electronic ledger innovatively employs the "cumulative pixel coordinates" of track inspection images as a unified basic coordinate system, achieving pixel-level precise localization and digital management of typical track facilities such as sleepers and supports, providing a high-precision spatial reference system for subsequent defect localization. Based on this, this embodiment mainly includes the following key technical steps:

[0089] 1. Feature embedding representation:

[0090] For defect identification results of different modalities (such as image-based and waveform-based defects), feature embedding technology is used to transform them into standardized feature vector representations. This transformation ensures consistent representation of data from different modalities in the feature space, laying the foundation for subsequent accurate localization.

[0091] 2. Ledger pixel coordinate mapping:

[0092] A dedicated pixel coordinate mapping method for electronic ledgers was designed. This network uses standardized defect feature vectors and electronic ledger data as dual inputs. Through the coordinate mapping method, it can adaptively learn the mapping relationship between defect features of different modalities and the ledger space, and finally output the precise pixel coordinates corresponding to the defect location.

[0093] 3. Facility-level positioning:

[0094] Based on the acquired pixel coordinates and combined with the pre-divided facility area information, the nearest neighbor matching algorithm is used to accurately locate the defect location to the facility closest to the machine vision line, thus achieving a precise mapping from pixel coordinates to specific facilities.

[0095] This embodiment cleverly solves the technical challenge of multimodal data localization through a unified feature representation and mapping framework, achieving precise location of specific facilities from the identification results of different types of defects. The solution is simple and efficient, with strong versatility and scalability, providing innovative technical support for the refined management of railway line facilities.

[0096] Figure 2 This is a specific example diagram of a method for locating defects in an electronic ledger of a circuit according to an embodiment of the present invention. Figure 2 First, the multimodal defect data is processed through a feature embedding representation step. This step includes a multimodal feature embedding representation network, which uses a convolutional neural network to process image modal data, a long short-term memory network to process waveform modal data, and a point cloud network to process point cloud modal data. A multi-head attention mechanism is then used to achieve feature fusion and embedding transformation. Next, the system proceeds to the ledger pixel coordinate mapping step. A dual-path mapping neural network processes the feature vectors and electronic ledger data separately. Feature fusion is achieved through feature transformation, multi-head attention, feature concatenation, cross-attention, and gating fusion, ultimately outputting the pixel coordinates. Finally, a facility-level localization step uses a nearest neighbor matching algorithm to match the pixel coordinates with the facility areas in the electronic ledger, achieving precise location of the defect from the specific facility.

[0097] Specifically, this embodiment may include the following:

[0098] 1. Feature embedding representation network design:

[0099] Multimodal feature embedding representation networks are primarily used to process multi-source heterogeneous data in railway line inspection, including image modalities, waveform modalities, and point cloud modalities. The network adopts a modular design, with dedicated encoder structures designed for the characteristics of different modalities. Image modalities are extracted using a three-layer CNN structure, containing convolutional layers with 64, 128, and 256 channels, combined with max pooling and global average pooling for feature dimensionality reduction. Waveform modalities employ a two-layer LSTM structure, capturing temporal features through 128-dimensional and 64-dimensional recurrent units. Point cloud modalities utilize the PointNet architecture, extracting spatial geometric features through PointConv and global max pooling. Although these three encoders have different structures, they all map the input data to a unified 256-dimensional feature space, laying the foundation for subsequent processing.

[0100] In the feature fusion stage, the network introduces an 8-head attention mechanism, which dynamically fuses features by adaptively learning the importance weights of features from different modalities. The fused features are further processed through an embedding transform layer, which includes a fully connected layer with Tanh activation and L2 normalization to ensure that the final output feature vector has a uniform scale and good geometric properties. It is worth noting that although the network supports multimodal input, a single-modal input strategy is adopted in practical applications, that is, only one modality of data is processed at a time. This design ensures both processing efficiency and maintains the consistency of output features.

[0101] This network architecture design enables a unified representation of multimodal data. The network's modular structure gives it excellent scalability, allowing for the flexible addition of new modality encoders as needed, while the unified feature output format facilitates subsequent tasks.

[0102] In one example, the line defect identification results of different modalities are converted into corresponding feature vectors through a feature embedding representation network. The feature embedding representation network adopts a modular design and includes a three-layer convolutional neural network encoder for image modality, a two-layer long short-term memory network encoder for waveform modality, and a point cloud network encoder for point cloud modality. Each encoder maps the input data of the corresponding modality to a unified 256-dimensional feature space, performs feature fusion through an eight-head attention mechanism, and then completes the embedding transformation through a fully connected layer containing a hyperbolic tangent activation function and L2 normalization operation, outputting a standardized feature vector. The feature vector and the facility data in the electronic ledger are input together into a dual-path mapping neural network. Through the multi-head attention mechanism and feature fusion strategy in this network, the feature vector is mapped to the ledger space, and the pixel coordinates of the defect location in the cumulative pixel coordinate system are output.

[0103] The specific operation process of the multimodal feature embedding representation network is as follows:

[0104] First, the input data of different modalities are processed separately. For image modal data, a three-layer convolutional neural network is used for feature extraction, sequentially passing through convolutional layers with 64, 128, and 256 channels, and combining max pooling and global average pooling operations to complete feature dimensionality reduction. For waveform modal data, a two-layer long short-term memory network structure is adopted, sequentially extracting temporal features through 128-dimensional and 64-dimensional recurrent units. For point cloud modal data, spatial geometric features are extracted through point convolution operations and global max pooling in the point cloud network architecture. Subsequently, the features output by each modal encoder are mapped to a unified 256-dimensional feature space.

[0105] Building upon this foundation, an eight-head attention mechanism is employed for adaptive weight allocation and fusion of features, dynamically weighting them by calculating importance scores for different feature dimensions. The fused features are then processed through an embedding transform layer, which includes a fully connected layer using a hyperbolic tangent activation function. The output is then normalized using L2, ultimately generating a standardized feature vector. The entire process employs a single-modal input strategy, processing one modality of data independently each time, ensuring consistent dimensionality and good geometric properties in the output features, thus providing standardized feature representations for subsequent tasks.

[0106] Figure 3 This is a specific example diagram illustrating data processing in a multimodal feature embedding representation network according to an embodiment of the present invention, such as... Figure 3 As shown, the multimodal feature embedding representation network in this embodiment is as follows: Figure 3 As shown:

[0107] First, three inputs are received. The image branch starts from the input image modality with dimensions [B, H, W, C]. It then passes through Conv2D(64)+ReLU→MaxPool2D→Conv2D(128)+ReLU→MaxPool2D→Conv2D(256)+ReLU→GlobalAvgPool in sequence, and finally outputs the image feature vector of [B, 256].

[0108] The waveform branch starts from the input waveform mode with dimensions [B, T, F]; it passes through LSTM(128)→Dropout(0.2)→LSTM(64)→Dropout(0.2)→Dense(256)→ReLU in sequence, and finally outputs a waveform feature vector of [B, 256].

[0109] The point cloud branch starts from the input point cloud modality with dimensions [B, N, 3]; it passes through PointConv(64)+ReLU→MaxPool1D→PointConv(128)+ReLU→MaxPool1D→Dense(256)+ReLU→GlobalMaxPool in sequence, and finally outputs a point cloud feature vector of [B, 256].

[0110] The fusion branch starts with three feature vectors, which are fed into a multi-head self-attention mechanism through f1, f2, and f3 respectively. The Query / Key / Value are all set to [B, 256] and the number of Heads is 8. The weights are calculated by scaling dot product and weighted fusion is completed to finally obtain a fused feature vector of [B, 256].

[0111] The embedding transformation branch starts with the fused feature vector. First, a nonlinear transformation is performed by Dense(256)+Tanh, followed by L2 normalization, and finally a unified embedding vector [B, 256] is output.

[0112] 2. Ledger pixel coordinate mapping:

[0113] The Dual-Path Mapping Neural Network is a deep learning architecture specifically designed for fusing feature vectors and electronic ledger data. This network employs a dual-input design, receiving a 256-dimensional feature vector (from the output of a feature embedding network) and a 20-dimensional electronic ledger vector (containing key facility attribute information). Through multi-level feature transformation and fusion, the network ultimately outputs precise pixel coordinates, achieving accurate mapping from a high-dimensional feature space to specific locations.

[0114] In terms of network architecture design, the two input branches employ a symmetrical but unique processing flow. The feature vector branch first increases the feature capacity through a 256→512 dimensional expansion layer, then enhances feature representation and training stability through BatchNorm and ReLU activation functions, and finally extracts refined features through a 512→256 dimensional compression layer. The ledger vector branch adopts a similar structure, but maintains the same dimensionality (512→512) in the first layer. Through multiple nonlinear transformations and regularization, it also outputs a 256-dimensional feature representation, ensuring that the features of the two branches are aligned in the same semantic space.

[0115] The feature fusion module is the core innovation of the network, employing a multi-layered fusion strategy. First, a multi-head attention mechanism (Q, K, and V three-way attention calculation) is used to adaptively assign weights to features. Then, feature concatenation preserves the original information. Next, a cross-attention mechanism enables deep interaction between features from two branches. Finally, a gating fusion mechanism dynamically adjusts the importance of different features. This multi-layered fusion design ensures full interaction of features and maximizes the utilization of information.

[0116] During training, the network employs an end-to-end joint optimization strategy. The loss function chosen is MSE Loss, as shown in the following formula:

[0117]

[0118] in, For the sample size, These are the actual coordinate values. For predicting coordinate values, i represents the index of the i-th data point.

[0119] This optimization optimizes the error between predicted and true coordinates. The optimizer uses Adam with an initial learning rate of 1e-3 and a batch size of 64. To improve the model's generalization ability, multiple regularization measures are implemented, including a BatchNorm layer to prevent internal covariate shift, Dropout to prevent overfitting, and L2 regularization to control weight growth.

[0120] The advantages of this network architecture are: it can effectively fuse information from different modalities (visual features and structured data), extract effective features through multi-level feature transformation and fusion strategies, and achieve accurate coordinate mapping through end-to-end training. Its modular design also gives the network good scalability and maintainability, allowing for flexible adjustment and optimization according to actual needs.

[0121] The specific operation process of a dual-path mapping neural network includes the following four stages:

[0122] First, dual-path feature input and preprocessing are performed, with the 256-dimensional feature vector and the 20-dimensional electronic ledger vector input into two independent branches respectively.

[0123] Next, feature transformation and alignment are performed. The feature vector branch is expanded and compressed in dimension through a fully connected layer, while the ledger vector branch maintains dimensionality consistency through multi-layer nonlinear transformation. Both branches are processed using batch normalization layers and modified linear unit activation functions, and finally output feature representations of the same dimension to achieve semantic space alignment.

[0124] Then, multi-level feature fusion is performed. Adaptive weights are calculated sequentially using a multi-head attention mechanism. Original information is preserved by feature concatenation. Deep interaction between branches is achieved using a cross-attention mechanism. Finally, the feature contribution is dynamically adjusted by a gated fusion unit.

[0125] Finally, coordinate mapping and output are performed, mapping the fused features to two-dimensional pixel coordinate values ​​through a fully connected layer. The network training employs an end-to-end optimization strategy, using the mean squared error loss function to measure the difference between predicted and true coordinates, updating network parameters through an adaptive moment estimation algorithm, and combining dropout and weight decay regularization techniques to improve the model's generalization ability.

[0126] Specifically, Figure 4 This is a specific example diagram illustrating the architecture of a dual-path mapping neural network in an embodiment of the present invention, such as... Figure 4The diagram illustrates the specific architecture of the dual-path mapping neural network. The feature vector branch starts with the input feature vector, which has a dimension of batch_size × 256. First, it undergoes a dimension transformation through an FC1 layer, expanding it from 256 to 512. Then, BatchNorm and ReLU activation functions are applied. Next, an FC2 layer compresses the dimension back from 512 to 256, and BatchNorm and ReLU are applied again, finally outputting a 256-dimensional feature vector. The ledger vector branch starts with the input electronic ledger vector, which has a dimension of batch_size × 20. First, it undergoes a dimension transformation through an FC1 layer, expanding it from 20 to 512. Then, BatchNorm and ReLU are applied. Next, an FC2 layer maintains the dimension from 512 to 256, and BatchNorm and ReLU are applied again, finally outputting a 256-dimensional ledger vector.

[0127] The outputs of the two branches enter the feature fusion module, which sequentially performs Attention (involving Q, K, and V calculations), then a Concat operation, followed by Cross-Attention for feature interaction, and finally dynamic weighting using the GatedFusion strategy to generate fused features. The fused features then pass through an input-output mapping layer. First, a Projection layer reduces the dimension from 512 to 128, then a Regression layer maps the dimension from 128 to 1, ultimately outputting Pixel Coordinates. The entire process achieves an end-to-end mapping from feature vectors and ledger vectors to pixel coordinates.

[0128] 3. Facility-level positioning:

[0129] After obtaining the cumulative pixel coordinates output by the model, a nearest neighbor matching algorithm based on distance metric is used to accurately locate the defect from the specific facility. Specifically, the area of ​​each facility is first pre-marked in the machine vision electronic ledger, and represented as a set of area coordinates based on the cumulative pixel coordinates.

[0130] Let the pixel coordinates of the defect location be... The first in the electronic ledger The set of regional coordinates of each facility is Then the index of the facility to which the defect belongs. It can be represented as:

[0131]

[0132] Where d(p, Fi) represents the minimum distance from point p to facility area Fi:

[0133]

[0134] p represents the Euclidean distance. This indicates the pixel coordinates of the defect location in the cumulative pixel coordinate system of the machine vision electronic ledger. Indicates facility area Any coordinate point within the range.

[0135] This nearest-neighbor-based matching strategy not only considers the spatial distribution characteristics of facilities but also effectively handles the relationship between defect locations and facility boundaries, ensuring the accuracy and reliability of the location results. Furthermore, by establishing a precise correspondence between defect locations and facilities, it provides reliable data support for subsequent maintenance decisions and management.

[0136] The facility-level positioning operation procedure is as follows:

[0137] First, the area of ​​each facility is pre-marked in the machine vision-based electronic ledger, forming a set of area coordinates based on accumulated pixel coordinates. After obtaining the pixel coordinates of the defect location, the minimum Euclidean distance from these coordinates to the set of area coordinates of each facility in the electronic ledger is calculated.

[0138] The specific calculation process is as follows: The straight-line distance between the defect pixel coordinates and each coordinate point within the facility area is calculated, and the minimum value is taken as the minimum distance to that facility. By comparing the minimum distance values ​​from the defect location to all facility areas, the defect is located to the specific facility with the smallest distance, thus establishing a precise correspondence between the defect and the specific facility. This matching strategy achieves precise facility-level location of the defect through spatial distance measurement, considering both facility distribution characteristics and effectively handling boundary relationships, providing a reliable basis for maintenance decisions.

[0139] To verify the practical application effect of this embodiment, a missing fastener spring clip defect discovered during on-site inspection was selected for location testing. After inputting the inspection results into this embodiment, the location information of the defect can be accurately output, as shown in Table 1 below: A missing fastener spring clip defect was detected, located at track mileage K6+436. Through the facility-level positioning of this method, the defect was accurately located at 27 sleepers downstream of support D03, and 39 sleepers upstream of support D05.

[0140] Table 1

[0141]

[0142] This bidirectional reference method of location description not only provides precise spatial location information of defects, but also provides clear location navigation references for on-site maintenance personnel.

[0143] The defect facility-level location method based on machine vision-based electronic ledgers for circuits proposed in this embodiment significantly improves on-site maintenance efficiency. By achieving precise positioning from a "general area" to a "specific facility," on-site inspection time is significantly reduced. Clear location descriptions provide maintenance personnel with clear location navigation, greatly reducing manual inspection costs and effectively improving maintenance response speed. Simultaneously, this method optimizes the maintenance workflow; detection data can directly locate specific facilities, eliminating the need for manual secondary positioning. The standardized location description method facilitates work order handover and on-site construction, reducing repetitive work by on-site personnel and significantly improving operational efficiency.

[0144] In terms of intelligence, this embodiment innovatively proposes a unified multimodal defect localization framework, which realizes intelligent processing of different types of detection data; it uses deep learning methods to automatically complete feature extraction and coordinate mapping, effectively reducing manual intervention; at the same time, it establishes a standardized feature representation mechanism, which significantly improves the system's versatility and scalability.

[0145] The application of this embodiment significantly improves maintenance management. By achieving precise mapping between defect locations and facilities, it facilitates facility status tracking and assessment; the system supports accurate recording and analysis of facility maintenance history, which helps in developing preventative maintenance strategies; and it provides reliable data support for the entire lifecycle management of facilities, promoting the refined development of maintenance management. In terms of economic benefits, the application of this invention significantly reduces maintenance and operating costs. By reducing on-site inspection manpower and improving maintenance efficiency, it effectively shortens equipment downtime; at the same time, it optimizes maintenance resource allocation, comprehensively improves maintenance quality, and provides strong protection for railway operational safety.

[0146] The key technical point of this invention lies in constructing a complete facility-level defect localization system, which comprises four core layers: First, a unified multimodal defect localization framework is proposed, using feature embedding representation technology to convert heterogeneous detection data such as image and waveform data into a consistent representation in the feature space, achieving standardized processing of different data types; second, a coordinate mapping mechanism based on machine vision-based electronic railway ledgers is established, designing a dedicated dual-path mapping neural network and utilizing mechanisms such as multi-head attention to achieve accurate mapping of defect features to the ledger coordinate space; third, a facility-level precise localization algorithm is adopted, establishing a precise correspondence between defect locations and specific facilities based on a nearest neighbor matching strategy, and providing a standardized location description method; finally, through a system integration scheme, modules such as multimodal data processing, coordinate mapping, and facility localization are integrated with the machine vision-based electronic railway ledgers to form an end-to-end processing flow, and the output and application methods of the localization results are standardized. This technical solution can be applied to high-speed rail, conventional railway, and subway sections.

[0147] Of course, it is understood that there may be other variations of the above detailed process, and all such variations should fall within the protection scope of this invention.

[0148] In this embodiment of the invention, multimodal line defect identification results and a pre-constructed electronic ledger of the line using machine vision are obtained. The electronic ledger uses the cumulative pixel coordinates of track inspection images as a unified basic coordinate system. The line defect identification results of different modalities are converted into corresponding feature vectors through a feature embedding representation network. The feature vectors and facility data in the electronic ledger are input into a dual-path mapping neural network. Through the multi-head attention mechanism and feature fusion in the dual-path mapping neural network, the feature vectors are mapped to the ledger space, and the pixel coordinates of the defect location in the cumulative pixel coordinate system are output. Based on the pixel coordinates, a nearest neighbor matching algorithm is used to locate the defect location to a line facility in the electronic ledger that matches the distance. This invention, through acquiring multimodal defect identification results and an electronic ledger based on accumulated pixel coordinates, transforms heterogeneous defect data into a unified feature vector using a feature embedding representation network. Then, through a multi-head attention mechanism and feature fusion strategy in a dual-path mapping neural network, it achieves precise mapping between defect features and the ledger space, outputting the pixel coordinates of the defect in the accumulated pixel coordinate system. Based on a nearest neighbor matching algorithm, the defect is located to a specific facility, effectively overcoming the limitations of traditional meter-level positioning. This achieves end-to-end precise positioning from multimodal data to the facility level, significantly improving the accuracy and efficiency of defect location, reducing the manpower and time costs of on-site inspections, and enhancing the intelligence and refinement of maintenance management.

[0149] This invention also provides a defect location device for electronic railway line ledgers, as described in the following embodiments. Since the principle by which this device solves the problem is similar to the defect location method for electronic railway line ledgers, the implementation of this device can refer to the implementation of the defect location method for electronic railway line ledgers; repeated details will not be elaborated further.

[0150] This invention also provides a railway line electronic ledger defect location device, which enables precise location of railway line defects from multimodal identification to facility-level accuracy, thereby improving the efficiency and accuracy of railway line facility maintenance and management. Figure 5 This is a schematic diagram of the structure of a line electronic ledger defect location device according to an embodiment of the present invention, as shown below. Figure 5 As shown, the device includes:

[0151] The data acquisition module 501 is used to acquire multimodal line defect identification results and a pre-constructed electronic ledger of the line using machine vision; the electronic ledger of the line uses the cumulative pixel coordinates of the track inspection images as a unified basic coordinate system.

[0152] The feature vector conversion module 502 is used to convert the identification results of line defects of different modes into corresponding feature vectors through the feature embedding representation network;

[0153] The pixel coordinate output module 503 is used to input the feature vector and the facility data in the electronic ledger into the dual-path mapping neural network. Through the multi-head attention mechanism and feature fusion in the dual-path mapping neural network, the feature vector is mapped to the ledger space, and the pixel coordinates corresponding to the defect location in the cumulative pixel coordinate system are output.

[0154] The defect location module 504 is used to locate the defect location to a line facility in the electronic ledger of the line that matches the distance based on the pixel coordinates and using the nearest neighbor matching algorithm.

[0155] In one embodiment, it further includes: a route electronic ledger construction module, used for:

[0156] Construct an electronic ledger for the route using machine vision as follows:

[0157] Acquire images of line facilities captured by a line scan camera along the target line;

[0158] The image is input into a target detection model for railway facility images based on an optimized YOLOv5 network structure to output the target detection area; OCR technology is used within the target detection area to extract text information and determine the ledger data of the railway facility, and the facility location in the ledger is associated with the cumulative pixel coordinates of the track inspection image; an electronic ledger of the railway is constructed with the cumulative pixel coordinates as a unified basic coordinate system.

[0159] In one embodiment, the multimodal line defect identification result includes defect data in image mode, waveform mode, and point cloud mode;

[0160] The results of line defect identification in different modalities are converted into corresponding feature vectors through a feature embedding representation network, including:

[0161] For defect data in image modalities, an encoder containing a multi-layer convolutional structure is used for feature extraction;

[0162] For defective waveform modal data, an encoder containing long short-term memory units is used to extract timing features;

[0163] For defective point cloud modal data, an encoder based on a point cloud network structure is used to extract spatial geometric features;

[0164] The feature embedding representation network maps the defect data of the corresponding modality to a feature space of the same dimension. Then, the extracted features are fused and weighted by a module containing a multi-head attention mechanism, and the corresponding feature vector is output through the embedding transformation layer.

[0165] In one embodiment, the feature vector and facility data from the electronic ledger are jointly input into a dual-path mapping neural network. Through the multi-head attention mechanism and feature fusion in the dual-path mapping neural network, the feature vector is mapped to the ledger space, and the pixel coordinates corresponding to the defect location in the cumulative pixel coordinate system are output, including:

[0166] The standardized feature vector and the electronic ledger vector containing facility attributes are respectively input into multiple independent input branches of the neural network; wherein each branch performs feature transformation through a fully connected layer containing batch normalization and activation function, so that the two features are aligned in the same semantic space.

[0167] The feature is adaptively weighted by a multi-head attention mechanism, and the two feature paths are fused by feature concatenation, cross-attention mechanism and gating fusion strategy.

[0168] The pixel coordinates of the defect in the cumulative pixel coordinate system are obtained by mapping through the output layer.

[0169] In one embodiment, based on the pixel coordinates, a nearest neighbor matching algorithm is used to locate the defect location to a matching line facility in the electronic line ledger, including:

[0170] The area of ​​each facility is pre-marked in the electronic ledger of machine vision lines and represented as a set of area coordinates based on cumulative pixel coordinates;

[0171] Calculate the minimum distance from the pixel coordinates of the defect location to the set of coordinates of each facility area in the electronic ledger;

[0172] Based on multiple minimum distances, the defect location is located to the designated line facility with the minimum distance, establishing a precise correspondence between the defect location and the specific facility.

[0173] This invention provides an embodiment of a computer device for implementing all or part of the above-described method for locating defects in electronic ledgers for railway lines. The computer device specifically includes the following components:

[0174] The computer device comprises a processor, memory, a communications interface, and a bus; wherein the processor, memory, and communications interface communicate with each other via the bus; the communications interface is used to realize information transmission between related devices; the computer device can be a desktop computer, tablet computer, or mobile terminal, etc., and this embodiment is not limited to these. In this embodiment, the computer device can be implemented with reference to the embodiments for implementing the method for locating defects in electronic ledgers for railway lines and the embodiments for implementing the device for locating defects in electronic ledgers for railway lines, the contents of which are incorporated herein by reference, and repeated details will not be described again.

[0175] Figure 6 This is a schematic diagram of a computer device provided in an embodiment of the present invention, which discloses a schematic block diagram of the system configuration of a computer device 1000 according to an embodiment of this application. Figure 6 As shown, the computer device 1000 may include a central processing unit 1001 and a memory 1002; the memory 1002 is coupled to the central processing unit 1001. It is worth noting that... Figure 6 This is an example; other types of structures can also be used to supplement or replace this structure to achieve telecommunications functions or other functions.

[0176] In one embodiment, the defect location function of the electronic ledger for the circuit can be integrated into the central processing unit 1001. The central processing unit 1001 can be configured to perform the following control:

[0177] The system acquires multimodal track defect identification results and a pre-constructed electronic track ledger using machine vision; the electronic track ledger uses the cumulative pixel coordinates of track inspection images as a unified basic coordinate system.

[0178] The results of line defect identification in different modes are converted into corresponding feature vectors through a feature embedding representation network;

[0179] The feature vector and facility data in the electronic ledger are input into a dual-path mapping neural network. Through the multi-head attention mechanism and feature fusion in the dual-path mapping neural network, the feature vector is mapped to the ledger space, and the pixel coordinates corresponding to the defect location in the cumulative pixel coordinate system are output.

[0180] Based on the pixel coordinates, the nearest neighbor matching algorithm is used to locate the defect location to a line facility in the electronic ledger that matches the distance.

[0181] In another embodiment, the line electronic ledger defect location device can be configured separately from the central processing unit 1001. For example, the line electronic ledger defect location device can be configured as a chip connected to the central processing unit 1001, and the line electronic ledger defect location function can be realized through the control of the central processing unit.

[0182] like Figure 6 As shown, the computer device 1000 may further include: a communication module 1003, an input unit 1004, an audio processor 1005, a display 1006, and a power supply 1007. It is worth noting that the computer device 1000 does not necessarily need to include... Figure 6 All components shown; in addition, the computer device 1000 may also include Figure 6 For components not shown, please refer to existing technologies.

[0183] like Figure 6 As shown, the central processing unit 1001, sometimes also referred to as a controller or operation control, may include a microprocessor or other processor device and / or logic device. The central processing unit 1001 receives input and controls the operation of various components of the computer device 1000.

[0184] The memory 1002 may be, for example, one or more of a cache, flash memory, hard drive, removable medium, volatile memory, non-volatile memory, or other suitable device. It can store the aforementioned device-related information, and may also store programs for executing that information. The central processing unit 1001 can execute the program stored in the memory 1002 to perform information storage or processing, etc.

[0185] Input unit 1004 provides input to central processing unit 1001. This input unit 1004 may be, for example, a keypad or touch input device. Power supply 1007 provides power to computer device 1000. Display 1006 displays images, text, and other display objects. This display may be, for example, an LCD display, but is not limited to this.

[0186] The memory 1002 can be a solid-state memory, such as a read-only memory (ROM), random access memory (RAM), a SIM card, etc. It can also be a memory that retains information even when power is off, can be selectively erased, and contains more data; examples of this type of memory are sometimes referred to as EPROMs. The memory 1002 can also be some other type of device. The memory 1002 includes a buffer memory 1021 (sometimes referred to as a buffer). The memory 1002 may include an application / function storage unit 1022 for storing application programs and function programs or processes for executing operations of the computer device 1000 via the central processing unit 1001.

[0187] The memory 1002 may also include a data storage unit 1023 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the computer device. The driver storage unit 1024 of the memory 1002 may include various drivers for the computer device for communication functions and / or for performing other functions of the computer device (such as messaging applications, address book applications, etc.).

[0188] The communication module 1003 is a transmitter / receiver that transmits and receives signals via the antenna 1008. The communication module (transmitter / receiver) 1003 is coupled to the central processing unit 1001 to provide input signals and receive output signals, which is the same as in a conventional mobile communication terminal.

[0189] Based on different communication technologies, multiple communication modules 1003 can be configured in the same computer device, such as cellular network modules, Bluetooth modules, and / or wireless LAN modules. The communication module (transmitter / receiver) 1003 is also coupled to a speaker 1009 and a microphone 1010 via an audio processor 1005 to provide audio output via the speaker 1009 and receive audio input from the microphone 1010, thereby realizing typical telecommunications functions. The audio processor 1005 may include any suitable buffer, decoder, amplifier, etc. Additionally, the audio processor 1005 is also coupled to a central processing unit 1001, enabling on-device recording via the microphone 1010 and on-device playback of stored sound via the speaker 1009.

[0190] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for locating defects in electronic ledgers for circuits.

[0191] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described method for locating defects in electronic ledgers for circuits.

[0192] In this embodiment of the invention, multimodal line defect identification results and a pre-constructed electronic ledger of the line using machine vision are obtained. The electronic ledger uses the cumulative pixel coordinates of track inspection images as a unified basic coordinate system. The line defect identification results of different modalities are converted into corresponding feature vectors through a feature embedding representation network. The feature vectors and facility data in the electronic ledger are input into a dual-path mapping neural network. Through the multi-head attention mechanism and feature fusion in the dual-path mapping neural network, the feature vectors are mapped to the ledger space, and the pixel coordinates of the defect location in the cumulative pixel coordinate system are output. Based on the pixel coordinates, a nearest neighbor matching algorithm is used to locate the defect location to a line facility in the electronic ledger that matches the distance. This invention, through acquiring multimodal defect identification results and an electronic ledger based on accumulated pixel coordinates, transforms heterogeneous defect data into a unified feature vector using a feature embedding representation network. Then, through a multi-head attention mechanism and feature fusion strategy in a dual-path mapping neural network, it achieves precise mapping between defect features and the ledger space, outputting the pixel coordinates of the defect in the accumulated pixel coordinate system. Based on a nearest neighbor matching algorithm, the defect is located to a specific facility, effectively overcoming the limitations of traditional meter-level positioning. This achieves end-to-end precise positioning from multimodal data to the facility level, significantly improving the accuracy and efficiency of defect location, reducing the manpower and time costs of on-site inspections, and enhancing the intelligence and refinement of maintenance management.

[0193] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0194] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0195] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0196] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0197] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for line electronic office account defect positioning, characterized in that, The method comprises the following steps: acquiring multi-modal track defect recognition results and a track electronic account constructed in advance through machine vision; the track electronic account adopts accumulated pixel coordinates of track inspection images as a unified base coordinate system; the multi-modal track defect recognition results are converted into corresponding feature vectors through a feature embedding representation network; the feature vectors and facility data in the electronic account are jointly input into a two-way mapping neural network, and the feature vectors and the account space are mapped through multi-head attention mechanism and feature fusion in the two-way mapping neural network, and pixel coordinates of the defect position in the accumulated pixel coordinate system are output; based on the pixel coordinates, the nearest neighbor matching algorithm is used to locate the defect position to the track facility in the electronic account that is closest to the matched track facility.

2. The method of claim 1, wherein, The method further comprises the following steps: constructing a track electronic account through machine vision in the following manner: acquiring track facility images collected by a line array camera in a target track; inputting the images into a track facility image target detection model based on an optimized YOLOv5 network structure to output a target detection region; extracting text information and determining account data of the track facility in the target detection region by using an OCR technology, and associating facility positions in the account with accumulated pixel coordinates of track inspection images; and constructing a track electronic account with the accumulated pixel coordinates as a unified base coordinate system.

3. The method of claim 1, wherein, The multi-modal track defect recognition results include defect data in image modalities, waveform modalities and point cloud modalities; the multi-modal track defect recognition results are converted into corresponding feature vectors through a feature embedding representation network, which comprises the following steps: for defect data in image modalities, an encoder containing a multi-layer convolution structure is used for feature extraction; for defect data in waveform modalities, an encoder containing a long short-term memory unit is used to extract time sequence features; for defect data in point cloud modalities, an encoder based on a point cloud network structure is used to extract spatial geometric features; the defect data in corresponding modalities is mapped to a unified dimensional feature space through the feature embedding representation network, and the extracted features are fused and weighted through a module containing a multi-head attention mechanism, and a corresponding feature vector is output through an embedding transformation layer.

4. The method of claim 1, wherein, the feature vectors and facility data in the electronic account are jointly input into a two-way mapping neural network, and the feature vectors and the account space are mapped through multi-head attention mechanism and feature fusion in the two-way mapping neural network, and pixel coordinates of the defect position in the accumulated pixel coordinate system are output, which comprises the following steps: the standardized feature vectors and the electronic account vectors containing facility attributes are respectively input into multiple independent input branches of the neural network; wherein each branch performs feature transformation through a fully connected layer containing batch normalization and an activation function, so that the two features are aligned in the same semantic space; the features are adaptively weighted through a multi-head attention mechanism, and the fusion of the two features is realized through feature splicing, cross-attention mechanism and gating fusion strategy; the pixel coordinates of the defect in the accumulated pixel coordinate system are obtained through the output layer mapping.

5. The method of claim 1, wherein, Based on the pixel coordinates, the defect position is located to the line facility with the closest distance in the line electronic account by using a nearest neighbor matching algorithm, comprising: The area range of each facility is pre-calibrated in the machine vision line electronic account and is represented as a set of area coordinates based on accumulated pixel coordinates; The minimum distance from the pixel coordinates of the defect position to the set of area coordinates of each facility in the electronic account is calculated; Based on the multiple minimum distances, the defect position is located to the specified line facility with the minimum distance, establishing an accurate correspondence between the defect position and the specific facility.

6. A line electronic office account defect location apparatus, characterized by, Comprise: A data acquisition module for acquiring multi-modal line defect recognition results and a line electronic account previously constructed by machine vision; the line electronic account uses accumulated pixel coordinates of track inspection images as a unified base coordinate system; A feature vector conversion module for converting line defect recognition results of different modalities into corresponding feature vectors through a feature embedding representation network; A pixel coordinate output module for inputting the feature vectors and facility data in the electronic account into a double-path mapping neural network, performing mapping of the feature vectors and the account space through a multi-head attention mechanism and feature fusion in the double-path mapping neural network, and outputting pixel coordinates of the defect position in the accumulated pixel coordinate system; A defect position locating module for locating the defect position to the line facility with the closest distance in the line electronic account by using a nearest neighbor matching algorithm based on the pixel coordinates.

7. The apparatus of claim 6, wherein, Also comprise: A line electronic account construction module for: Constructing a line electronic account by machine vision in the following way: Obtain line facility images collected by a line array camera in a target line; Input the images into a line facility image target detection model based on an optimized YOLOv5 network structure to output target detection regions; extract text information and determine account data of line facilities by using OCR technology in the target detection regions, associate facility positions in the account with accumulated pixel coordinates of track inspection images; construct a line electronic account with the accumulated pixel coordinates as a unified base coordinate system.

8. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the method of any one of claims 1-5 when executing the computer program.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method of any one of claims 1-5.

10. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program is executed by the processor to implement the method of any one of claims 1-5. The computer program product comprises a computer program, and the computer program is executed by the processor to implement the method of any one of claims 1-5.

Images