A method and system for weighing a swing bridge based on a point cloud model algorithm

Abstract

The present application relates to the field of image processing, and relates to a kind of based on point cloud model algorithm's rotary bridge weighing method and system;Through the point cloud video set of preset training having been weighed key point cloud video collected, all point cloud video set is cut after using cutting algorithm, based on the frequency of having been weighed gray picture transformation in point cloud video set, having been weighed gray picture is sorted, the mapping of having been weighed gray picture and the repair bridge sample to be weighed is generated Association table, and the point cloud video frame vector of each point cloud video frame corresponding in association table is pre-trained, and the point cloud video frame vector of each point cloud video frame corresponding in association table is pre-trained, using strategy gradient algorithm to train detection completion, input is the weighed gray picture sequence of bridge q to be weighed, and output is the weighed gray picture of detection completion;The present application has the beneficial effects as follows: the present application proposes an efficient and practical interactive detection method, and promotes the exploration in this aspect in the field of picture detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, in particular to a method and system for weighing a rotating bridge based on a point cloud model algorithm using reinforcement learning. BACKGROUND

[0002] Point cloud detection is a technology for realizing point cloud geometric calibration, which obtains the mapping relationship between the reference point cloud and the to-be-detected point cloud by calculating the correlation matrix. In point cloud detection, the correlation matrix plays an important role. By finding corresponding feature points in multiple point clouds and calculating the correlation matrix of these points, the detection and reconstruction of point clouds can be realized.

[0003] The traditional point cloud detection method mainly includes a pixel-based method and a feature-based method. In the feature-based point cloud detection method, when using a feature point detection operator to detect point clouds, a large number of trivial and clustered feature points often appear. The correlation matrix calculated according to these feature points will cause excessive description of the local area of the point cloud, which will further lead to low detection quality. However, this method is not suitable for low-resolution scenes. At present, there are few studies on point cloud detection methods for low-resolution and single-resolution scenes. In the pixel-based point cloud detection method, the transformation relationship between point clouds is directly estimated by using the video frames of the video in the point cloud. First, the correlation matrix between point clouds is initialized. Then, the point cloud transformation is performed on each point cloud by using the correlation matrix, and the video frame error of the transformed point cloud is calculated. Finally, the error function is minimized by using optimization technology to realize point cloud detection. This method has good effect on low-resolution scenes, but it is not robust to point clouds with scale and brightness changes. SUMMARY

[0004] In order to overcome the shortcomings of the prior art, the purpose of the present application is to provide a method and system for weighing a rotating bridge based on a point cloud model algorithm, so as to obtain a reference detection that is not comprehensive or does not meet the requirements, and to provide a more reasonable suggestion for the bridge to be weighed.

[0005] In order to achieve the above purpose, the present application provides a method for weighing a rotating bridge based on a point cloud model algorithm, which comprises the following steps:

[0006] Step S1, by collecting the point cloud video set of the preset training weighed point cloud video, using a cutting algorithm to cut all the point cloud video sets, based on the frequency of the weighed gray picture transformation in the point cloud video set, the weighed gray picture is sorted, the correlation table of the weighed gray picture and the repaired sample of the bridge to be weighed is generated, and the point cloud video frame vector of each corresponding point cloud video frame in the correlation table is pre-trained;

[0007] Step S2, after cutting the point cloud video frame of the user's to-be-weighed bridge q in the generative model, the repair to-be-weighed bridge sample sequence is obtained by looking up the association table, and each repair to-be-weighed bridge sample in the repair to-be-weighed bridge sample sequence is converted into a corresponding point cloud video frame vector by using the point cloud video frame vector pre-trained in step S1, thereby obtaining a corresponding to-be-weighed bridge feature vector Hq;

[0008] Step S3, based on the retrieval algorithm, the point cloud video frame weighed gray picture sequence obtained in step S1 is retrieved from the point cloud video set to obtain the most similar repair to-be-weighed bridge q' to the to-be-weighed bridge q, and the corresponding weighed gray picture P of the most similar repair to-be-weighed bridge q' in the point cloud video set in ;

[0009] Step S4, according to the to-be-weighed bridge feature vector obtained in step S2, a transformer convolution head is used to generate a weighed gray picture vector related to the point cloud video frame

[0010] Step S5, the weighed gray picture retrieved in step S3 is cut into a weighed gray picture sequence by using Tokens, and the weighed gray picture sequence is converted into a repair to-be-weighed bridge sample sequence by using the association table in step S1, the Tokens point cloud video frame vector corresponding to each repair to-be-weighed bridge sample in the repair to-be-weighed bridge sample sequence is searched, a weighed gray picture feature vector Ha1 is obtained, and the to-be-weighed bridge feature vector Hq in step S2 is sequentially passed through a mask automatic update model, and then a dimension correction grid is used to obtain a reference detection vector after point cloud video frame information

[0011] Step S6, the weighed gray picture vector generated in step S4 and the to-be-weighed bridge feature vector Hq in step S2 are sequentially processed by downsampling, and then the two are combined and passed through a dimension correction grid to obtain a feature vector of the reference detection vector a2 before and after frame information

[0012] Step S7, the detection is completed by using a policy gradient algorithm, the input is the weighed gray picture sequence of the to-be-weighed bridge q, and the output is the weighed gray picture F of the detection out .

[0013] Further, step S1 further comprises:

[0014] Step S100, the historical to-be-weighed bridge of the training weighed point cloud video is collected by using a timer, and filtering is completed to obtain a point cloud video set pair of the historical to-be-weighed bridge

[0015] Step S101, in the pre-training point cloud video frame vector stage, the cutting algorithm in the point cloud video image analysis is used to cut the to-be-weighed bridge and the weighed gray picture in each sample in the point cloud video set, all the point cloud video frames in the point cloud video set are merged after weighing, an association table is established, and the point cloud video frame vector of the commonly used point cloud video frame is obtained by pre-training using the Tokens algorithm;

[0016] Step S4 further comprises:

[0017] Step S401, the encoder end of the Transformer network is used to input the to-be-weighed bridge feature vector Hq, the full-text information of the to-be-weighed bridge is fused through the global attention mechanism, and a to-be-weighed bridge feature vector H'q fused with the full-text information of the point cloud video set is further obtained;

[0018] Step S402, the decoder end of the Transformer network is used to input the to-be-weighed bridge feature vector H'q fused with the full-text information of the point cloud video set, and all the point cloud video frame vectors in the corresponding weighed gray picture vector are sequentially output.

[0019] Further, in step S3, the L1-LOSS and EDGE-LOSS evaluation values are combined as the final similarity discrimination, and the most relevant weighed gray picture sequence is searched from the point cloud video set.

[0020] Further, in step S3,

[0021] In the training stage, the weighed gray picture is first sent to the convolution head module for dimension correction and downsampling operation to obtain

[0022] Where B is the sample size, C1=3 is the P in channel dimension, H1=512, W1=512 are the height and width of P in respectively, C2=512 is the F out channel dimension, H2=16, W2=16 are the height and width of F out respectively.

[0023] The calculation formula of the L1-LOSS and EDGE-LOSS evaluation values is as follows:

[0024] L1-Loss is used to calculate the absolute difference value of the RGB value of F out and GT, GT refers to Groundtruth, i.e. the real weighed picture.

[0025] The formula is as follows:

[0026]

[0027] wherein, n represents the total number of videos on a picture; y i represents the RGB value of the i-th video on the GT picture; f(x i ) represents the F out RGB value of the i-th video;

[0028] Edge-Loss is to ensure the clarity of the edges of each object in the detection area when color diffusion, and the formula is as follows:

[0029]

[0030] wherein, X and Y are the F out and the real GT picture respectively, E is the edge feature obtained, and H and W are the height and width of the P in .

[0031] Further, in step S3, if the EDGE-LOSS evaluation value calculated by the retrieved bridge to be weighed and the original bridge to be weighed is not 0, the EDGE-LOSS evaluation value is used; otherwise, the L1-LOSS evaluation value is used.

[0032] According to the second aspect of the present application, the present application claims to protect a rotary bridge weighing system based on a point cloud model algorithm, comprising:

[0033] A training module, by collecting a point cloud video set of a preset training weighed key point cloud video, cutting all point cloud video sets by using a cutting algorithm, based on the frequency of the weighed gray picture transformation in the point cloud video set, completing the sorting of the weighed gray picture, generating a mapping association table of the weighed gray picture and the repaired bridge-to-be-weighed sample, and pre-training the point cloud video frame vector of each point cloud video frame corresponding in the association table;

[0034] A convolution head processing module, for completing point cloud video frame cutting on the user's to-be-processed bridge to be weighed q, obtaining a repaired bridge-to-be-weighed sample sequence by searching the association table, and converting each repaired bridge-to-be-weighed sample in the repaired bridge-to-be-weighed sample sequence into a corresponding point cloud video frame vector by using the pre-trained point cloud video frame vector of step S1, to obtain a corresponding bridge-to-be-weighed feature vector Hq;

[0035] A global residual connection module, for retrieving the closest repaired bridge-to-be-weighed q' to the bridge to be weighed q from the point cloud video set based on a retrieval algorithm, and the corresponding weighed gray picture P in of the closest repaired bridge-to-be-weighed q' in the point cloud video set from the point cloud video frame weighed gray picture sequence obtained by the training module;

[0036] The style container and module are used to generate the weighed gray picture vector related to the point cloud video frame by using the transformer convolution head according to the bridge feature vector to be weighed obtained by the convolution head processing module

[0037] The first reference detection vector acquisition module is used to convert the weighed gray picture sequence into the repaired bridge sample sequence to be weighed by using the correlation table in the training module, search the Tokens point cloud video frame vector corresponding to each repaired bridge sample to be weighed in the repaired bridge sample sequence to be weighed, obtain the weighed gray picture feature vector Ha1, and sequentially pass the weighed gray picture feature vector Ha1 and the bridge feature vector Hq in the convolution head processing module through a mask automatic update model, and then pass the weighed gray picture feature vector Ha1 and the bridge feature vector Hq through a dimension correction grid to obtain the reference detection vector after the point cloud video frame information

[0038] The second reference detection vector acquisition module is used to generate the weighed gray picture vector in the style container and module and the bridge feature vector Hq in the convolution head processing module are sequentially passed through down-sampling processing, and then the two are combined and passed through a dimension correction grid to obtain the feature vector of the reference detection vector a2 before and after the frame information

[0039] The test module is used to test the detection completion by using the policy gradient algorithm, the input of which is the weighed gray picture sequence of the bridge to be weighed, and the output of which is the weighed bridge picture F after the detection is completed out .

[0040] Further, the working process of the training module includes the following steps:

[0041] Step S100, the timer is used to collect the historical bridge to be weighed of the training weighed point cloud video, and filtering is completed to obtain the point cloud video set pair of the historical bridge to be weighed

[0042] Step S101, in the pre-training point cloud video frame vector stage, the cutting algorithm in the point cloud video image analysis is used to complete cutting of the bridge to be weighed and the weighed gray picture in each sample in the point cloud video set, all point cloud video frames in the point cloud video set are merged after being weighed, an association table is established, and the point cloud video frame vector of the commonly used point cloud video frame is obtained by using the Tokens algorithm for pre-training

[0043] The working process of the style container and module includes the following steps:

[0044] Step S401, the encoder end of the Transformer network is used to input the to-be-weighed bridge feature vector Hq, the full-text information of the to-be-weighed bridge is fused through a global attention mechanism, and a to-be-weighed bridge feature vector H'q fused with the full-text information of the point cloud video set is further obtained.

[0045] Step S402, the decoder end of the Transformer network is used to input the to-be-weighed bridge feature vector H'q fused with the full-text information of the point cloud video set, and corresponding weighed gray picture vectors of all point cloud video frame vectors are sequentially output.

[0046] Further, the L1-LOSS and EDGE-LOSS evaluation values need to be combined as the final similarity discrimination, and the most relevant weighed gray picture sequence is searched out from the point cloud video set.

[0047] Further, in the training stage, the weighed gray picture is first sent to the convolution head module for dimension correction and down-sampling operation to obtain

[0048] Where B is the sample size, C1=3 is the P in channel dimension, H1=512 and W1=512 are the height and width of P in respectively, and C2=512 is the F out channel dimension, H2=16 and W2=16 are the height and width of F out respectively.

[0049] The calculation formula of the L1-LOSS and EDGE-LOSS evaluation values is as follows:

[0050] L1-Loss is used to calculate the absolute difference value of the RGB value of F out and GT, GT refers to Groundtruth, i.e., the real weighed picture.

[0051] The formula is as follows:

[0052]

[0053] Where n represents the total number of videos on a picture; y i represents the RGB value of the i-th video on the GT picture; f(x i ) represents the RBG value of the i-th video on F out .

[0054] Edge-Loss is used to ensure the clarity of the edges of each object in the detection area when the color is diffused, and the formula is as follows:

[0055]

[0056] Where X and Y are F out And the real GT image, E is the obtained edge feature, H and W are P in Height and width.

[0057] Furthermore, if the calculated EDGE-LOSS evaluation value of the retrieved bridge to be weighed is not 0 compared to that of the original bridge to be weighed, then the EDGE-LOSS evaluation value is used; otherwise, the L1-LOSS evaluation value is used.

[0058] This invention relates to the field of image processing, specifically to a method and system for weighing rotating bridges based on a point cloud model algorithm. The method involves collecting a set of pre-trained point cloud videos of weighed bridges, segmenting the set using a cutting algorithm, and then sorting the weighed grayscale images based on their frequency of change within the set. This generates a mapping table between the weighed grayscale images and the bridge samples to be weighed. The method also pre-trains the point cloud video frame vectors for each corresponding point cloud video frame in the mapping table and uses a policy gradient algorithm to train the detection process. The input is a sequence of weighed grayscale images of the bridge q to be weighed, and the output is the detected weighed bridge image. The beneficial effects of this invention are as follows: It proposes an efficient and practical interactive detection method, advancing exploration in the field of image detection; and it significantly improves the coherence and boundary characteristics of color diffusion by introducing L1-Loss and Edge-Loss. Attached Figure Description

[0059] Figure 1 This is a flowchart illustrating the workflow of a rotating bridge weighing method based on a point cloud model algorithm according to the present invention.

[0060] Figure 2 This is a second flowchart of a method for weighing rotating bridges based on point cloud model algorithms according to the present invention.

[0061] Figure 3 This is the third workflow diagram of a rotating bridge weighing method based on point cloud model algorithm according to the present invention;

[0062] Figure 4 This is a structural block diagram of a rotating bridge weighing system based on a point cloud model algorithm according to the present invention. Detailed Implementation

[0063] According to a first embodiment of the present invention, referring to the appendix Figure 1 This invention claims protection for a method for weighing rotating bridges based on a point cloud model algorithm, comprising the following steps:

[0064] Step S1, by collecting the preset training point cloud video set of the key cloud video, using the cutting algorithm to cut all the point cloud video sets, based on the frequency of the weighted gray picture transformation in the point cloud video set, the weighted gray picture is sorted, the mapping association table of the weighted gray picture and the repair bridge sample to be weighed is generated, and the point cloud video frame vector of each point cloud video frame corresponding to the association table in the pre-training is pre-trained;

[0065] Step S2, in the generative model, after cutting the point cloud video frame of the user to be processed, the repair bridge sample sequence to be weighed is obtained by looking up the association table, and the point cloud video frame vector corresponding to each repair bridge sample to be weighed in the repair bridge sample sequence to be weighed is obtained by using the point cloud video frame vector pre-trained in step S1, so as to obtain the corresponding bridge feature vector Hq to be weighed;

[0066] Step S3, based on the retrieval algorithm, the point cloud video frame weighted gray picture sequence obtained in step S1 is retrieved from the point cloud video set to obtain the most similar repair bridge q' to be weighed and the corresponding weighted gray picture P of the most similar repair bridge q' to be weighed in the point cloud video set in ;

[0067] Step S4, according to the bridge feature vector to be weighed obtained in step S2, the transformer convolution head is used to generate the weighted gray picture vector related to the point cloud video frame

[0068] Step S5, the weighted gray picture retrieved in step S3 is cut by Tokens to obtain the weighted gray picture sequence, and then the weighted gray picture sequence is converted into the repair bridge sample sequence to be weighed by using the association table in step S1, the Tokens point cloud video frame vector corresponding to each repair bridge sample to be weighed in the repair bridge sample sequence to be weighed is searched, the weighted gray picture feature vector Ha1 is obtained, and the bridge feature vector to be weighed Hq in step S2 is sequentially passed through a mask automatic update model, and then a dimension correction grid is passed through to obtain the reference detection vector after the point cloud video frame information

[0069] Step S6, the weighted gray picture vector generated in step S4 and the bridge feature vector to be weighed Hq in step S2 are sequentially processed by downsampling, and then the two are combined and passed through a dimension correction grid to obtain the feature vector of the reference detection vector a2 before and after the frame information

[0070] Step S7: Use the policy gradient algorithm to train the detection completion. The input is the sequence of weighed grayscale images of the bridge q to be weighed, and the output is the weighed bridge image F that has been detected as completed. out .

[0071] Further, refer to the appendix. Figure 2 Step S1 further includes:

[0072] Step S100: Use a timer to collect historical point cloud video of the bridge to be weighed that has been trained, and filter the data to obtain a set of point cloud video of the historical bridge to be weighed.

[0073] Step S101: In the pre-training point cloud video frame vector stage, the cutting algorithm in point cloud video image parsing is used to cut the bridge to be weighed and the grayscale image that has been weighed in each sample of the point cloud video set. After deduplication of all point cloud video frames in the point cloud video set, an association table is established, and the point cloud video frame vector of commonly used point cloud video frames is obtained by pre-training using the Tokens algorithm.

[0074] See attached document Figure 3 Step S4 further includes:

[0075] Step S401: Using the encoder end of the Transformer network, the feature vector Hq of the bridge to be weighed is input, and the full text information of the bridge to be weighed is fused through the global attention mechanism to further obtain the feature vector H′q of the bridge to be weighed that is fused with the full text information of the point cloud video set.

[0076] Step S402: Using the decoder of the Transformer network, the feature vector H′q of the bridge to be weighed, which integrates the full-text information of the point cloud video set, is input, and the corresponding weighed grayscale image vectors are output sequentially. All point cloud video frame vectors;

[0077] Step S5 further includes:

[0078] Step S501: The feature vector Hq of the bridge to be weighed q obtained in step S2 is used to obtain the feature vectors of the preceding and following frames of the point cloud video frame through a loss model. The reference detection a1 obtained through the retrieval algorithm is then downsampled to obtain the feature vectors of the frames before and after the reference detection a1.

[0079] Step S502, the feature vector of the bridge q to be weighed is... And the retrieved weighed grayscale image P in eigenvectors First, the attention matrix is ​​obtained through matrix multiplication. Each element mi,j in the attention matrix represents the similarity evaluation value of the i-th point cloud video frame in the point cloud video frame q and the j-th point cloud video frame in the reference detection a1, and the matrix M1 is weighted and summed in the column dimension and the row dimension in turn, and the result after the summation is and The mq is multiplied by the eigenvector of the point cloud video frame q to obtain the point cloud video frame vector H'q of the reference detection information, and the is multiplied by the eigenvector of the reference detection a1 to obtain the reference detection vector of the point cloud video frame information

[0080] In step S503, the point cloud video frame vector H'q obtained in step S502 and the reference detection vector are concatenated and a feedforward neural network including two linear layers is used to obtain the detected completed weighed bridge picture F of the to-be-weighed bridge q and the reference detection a1 out ;

[0081] Step S6 further includes:

[0082] In step S601, the weighed gray picture vector is processed by downsampling to obtain the eigenvector of the reference detection vector a2 front and back frame information

[0083] In step S602, the to-be-weighed bridge eigenvector Hq of the to-be-weighed bridge q obtained in step S2 is used to obtain the eigenvector of the point cloud video frame front and back frame information of the to-be-weighed bridge q through a loss model, and then the obtained eigenvector of the point cloud video frame front and back frame information of the to-be-weighed bridge q is multiplied by the eigenvector of the weighed gray picture vector First, the attention matrix is obtained by matrix multiplication The matrix M2 is weighted and summed in the column dimension and the row dimension in turn, and the result after the summation is and The mq is multiplied by the eigenvector of the point cloud video frame q to obtain the point cloud video frame vector H"q of the reference detection information, and the is multiplied by the eigenvector of the weighed gray picture vector to obtain the reference detection vector of the point cloud video frame information

[0084] In step S603, the point cloud video frame vector H"q of the reference detection information obtained in step S602 and the reference detection vector of the point cloud video frame information are concatenated And through a feedforward neural network including two linear layers, a weighed gray picture F of the weighed bridge q to be weighed and a detection vector a2 of a reference detection are obtained out ;

[0085] Step S7 further includes:

[0086] Step S701, the L1-LOSS and EDGE-LOSS evaluation values of the weighed gray picture sequence of the reference detection a1 obtained by the retrieval model and the weighed gray picture sequence of the standard reference detection a* corresponding to the bridge q to be weighed are calculated, and the L1-LOSS and EDGE-LOSS evaluation values are sequentially f(Yr, Y*) and g(Yr, Y*), and the L1-LOSS and EDGE-LOSS evaluation values are weighted and summed to obtain the confidence weight corresponding to the reference detection a1. Meanwhile, the L1-LOSS and EDGE-LOSS evaluation values of the weighed gray picture sequence of the reference detection vector a2 obtained by the generation model and the weighed gray picture sequence of the standard reference detection a* corresponding to the point cloud video frame q are calculated, and the L1-LOSS and EDGE-LOSS evaluation values are sequentially f(Yg, Y*) and g(Yg, Y*), and the L1-LOSS and EDGE-LOSS evaluation values are weighted and summed to obtain the confidence weight corresponding to the reference detection vector a2.

[0087] Step S702, the confidence weights obtained by the two reference detections are compared to obtain the final confidence weight.

[0088] Further, in step S3, the L1-LOSS and EDGE-LOSS evaluation values are combined as the final similarity discrimination, and the most relevant weighed gray picture sequence is searched from the point cloud video set.

[0089] Further, in step S3, in the training stage, the weighed gray picture is first sent to the convolution head module for dimension correction and downsampling operation to obtain

[0090] Where B is the sample size, C1=3 is the P in channel dimension, H1=512, W1=512 are the height and width of P in , and C2=512 is the F out channel dimension, H2=16, W2=16 are the height and width of F out ;

[0091] The calculation formula of the L1-LOSS and EDGE-LOSS evaluation values is as follows:

[0092] L1-Loss is used to calculate F out and GT, GT refers to Groundtruth, i.e. the real weighed picture, the absolute difference value of RGB value on each video;

[0093] The formula is as follows:

[0094]

[0095] Wherein, n represents the total number of videos on a picture; y i represents the RGB value of the i-th video on the GT picture; f(x i ) represents the F out RGB value of the i-th video;

[0096] Edge-Loss is to ensure the clarity of the edges of each object in the detection area during color diffusion, and the formula is as follows:

[0097]

[0098] Wherein, X and Y are F out and the real GT picture respectively, E is the obtained edge feature, and H and W are the height and width of P in .

[0099] Further, in step S3, if the EDGE-LOSS evaluation value calculated by the retrieved bridge to be weighed and the original bridge to be weighed is not 0, the EDGE-LOSS evaluation value is used; otherwise, the L1-LOSS evaluation value is used.

[0100] According to the second embodiment of the present application, the present application claims to protect a swivel bridge weighing system based on a point cloud model algorithm, and the purpose of the present application is to provide a reliable and efficient interactive detection method for a to-be-weighed video, which uses an interactive detection model based on a point cloud model to manually detect a to-be-detected area; wherein, as shown in the figure, Figure 4 the swivel bridge weighing system based on the point cloud model algorithm comprises:

[0101] The training module uses a cutting algorithm to cut all the point cloud video sets after collecting the preset training weighed point cloud video set, sorts the weighed gray pictures based on the frequency of the weighed gray pictures in the point cloud video set, generates a mapping association table of the weighed gray pictures and the repaired to-be-weighed bridge samples, and pre-trains the point cloud video frame vector of each corresponding point cloud video frame in the association table.

[0102] Specifically, the gray information of the to-be-detected area needs to be reserved, the weighed information of the non-to-be-detected area needs to be reserved, the weighed information and the gray information coexist, and the color can be positively affected by the surrounding weighed information during color diffusion in the to-be-detected area.

[0103] The input model input picture is quickly down-sampled to 1 / 8 of the original size and output tokens for feeding into the transformer body. It contains 5 transformer blocks that can communicate remotely to process the input tokens, the first layer is used to change the input dimension, and the other layers are used for down-sampling;

[0104] In the training stage, the weighted information and the gray information of the picture are obtained from the COCO dataset. The COCO dataset is all full-weighted pictures, and the model obtains the gray information by calling the grayscale function in OpenCV to grayscale a part of the full-color picture. In this process, a corresponding mask matrix must be generated for each weighted picture, which is composed of only 0 or 1 and has a size of 512x512, and is used to record the position information of the corresponding color picture. 0 represents that the position is grayscaled, and 1 represents that the position retains the weighted information. As a result, a picture that retains both the weighted information and the gray information (hereinafter referred to as 'color and gray picture') and a mask matrix containing position information (hereinafter referred to as'mask matrix') are obtained. Then the color and gray picture and the mask matrix are sent into the model for calculation.

[0105] In the training stage, the F out output by step 2 needs to be input into the transformer body module. This transformer body module is composed of 5 adjusted SwinTransformer blocks in series, and the position embedding module is discarded, and a 3x3 convolution kernel is used for global residual connection to promote the long-range interaction ability of the network. In order to process a large number of tokens generated, a multi-head attention mechanism is introduced in the transformer body block, which greatly reduces the computational complexity. Therefore, after multi-layer calculation of the transformer body block, the

[0106] In the training stage, in order to increase the anti-interference ability of the model, a style fusion module is introduced in the transformer body block. This module designs a reconstruction process with additional noise, in which it mainly changes the weight normalization of the convolution layer to achieve the effect of disturbance.

[0107] The convolution head processing module is used to obtain the repair to-be-weighted bridge sample sequence by looking up the association table after the point cloud video frame cutting of the to-be-processed to-be-weighted bridge q of the user, and convert each repair to-be-weighted bridge sample in the repair to-be-weighted bridge sample sequence into a corresponding point cloud video frame vector by using the point cloud video frame vector pre-trained in step S1, to obtain a corresponding to-be-weighted bridge feature vector Hq;

[0108] The global residual connection module is used to retrieve, based on a retrieval algorithm, the weighed grayscale image sequence of point cloud video frames obtained from the training module, the closest bridge to be weighed (q′) to be repaired (q′), and the corresponding weighed grayscale image P of the closest bridge to be weighed (q′) in the point cloud video set. in ;

[0109] The style-capacity module is used to generate a weighed grayscale image vector associated with the point cloud video frame based on the feature vector of the bridge to be weighed obtained from the convolution head processing module and using a transformer convolution head.

[0110] The first reference detection vector acquisition module is used to obtain a sequence of weighed grayscale images retrieved from the global residual connection module, and then convert this sequence into a sequence of bridge samples to be weighed for repair using the association table in the training module. It then searches for the Tokens point cloud video frame vector corresponding to each bridge sample in the sequence, obtaining the feature vector Ha1 of the weighed grayscale image. This feature vector, along with the bridge feature vector Hq from the convolution head processing module, is then automatically updated using a mask. Finally, the model is corrected using a one-dimensional grid to obtain the reference detection vector after obtaining the point cloud video frame information.

[0111] The second reference detection vector acquisition module is used to obtain the weighted grayscale image vector generated in the style integration module. The bridge feature vector Hq to be weighed in the convolution head processing module is downsampled sequentially, and then the two are combined and passed through a one-dimensional corrected grid to obtain the feature vector of the reference detection vector a2 and the information of the frames before and after it.

[0112] The testing module is used to test the completion of detection using the policy gradient algorithm. Its input is a sequence of weighed grayscale images of the bridge q to be weighed, and its output is the weighed image F of the bridge that has been detected. out .

[0113] During the testing phase, a color-grayscale image is input into the model, but loss function calculations and gradient regression are not performed. The model outputs P... out This refers to the final image after the detection is completed.

[0114] Furthermore, the training module's operation includes the following steps:

[0115] Step S100: Use a timer to collect historical point cloud video of the bridge to be weighed that has been trained, and filter the data to obtain a set of point cloud video of the historical bridge to be weighed.

[0116] Step S101, in the pre-training point cloud video frame vector stage, the cutting algorithm in the point cloud video image analysis is used to cut the to-be-weighed bridge in each sample in the point cloud video set and the weighed gray picture, all the point cloud video frames in the point cloud video set are merged after weighing, an association table is established, and the point cloud video frame vector of the commonly used point cloud video frame is obtained by pre-training using the Tokens algorithm;

[0117] The working process of the style container and module includes the following steps:

[0118] Step S401, the encoder end of the Transformer network is used to input the to-be-weighed bridge feature vector Hq, the full-text information of the to-be-weighed bridge is fused through the global attention mechanism, and the to-be-weighed bridge feature vector H'q fused with the full-text information of the point cloud video set is further obtained;

[0119] Step S402, the decoder end of the Transformer network is used to input the to-be-weighed bridge feature vector H'q fused with the full-text information of the point cloud video set, and all the point cloud video frame vectors in the corresponding weighed gray picture vector are sequentially outputted;

[0120] The working process of the first reference detection vector acquisition module includes the following steps:

[0121] Step S501, the to-be-weighed bridge feature vector Hq of the to-be-weighed bridge q obtained in step S2 is inputted into a loss model to obtain the feature vector of the front and rear frame information of the point cloud video frame, and the reference detection a1 obtained through the retrieval algorithm is subjected to down-sampling processing to obtain the feature vector of the front and rear frame information of the reference detection a1;

[0122] Step S502, the feature vector of the to-be-weighed bridge q and the feature vector in of the retrieved weighed gray picture P are first subjected to matrix multiplication to obtain an attention matrix M1 Each element m i,j in the attention matrix M1 represents the similarity evaluation value of the i th point cloud video frame in the point cloud video frame q and the j th point cloud video frame in the reference detection a1, and the matrix M1 is sequentially weighted and summed in the column dimension and the row dimension, and the result after the summation is and The feature vector of the to-be-weighed bridge q is multiplied by mq to obtain the point cloud video frame vector H'q of the reference detection information, and the feature vector is multiplied by the feature vector of the reference detection a1 to obtain the reference detection vector H'q of the point cloud video frame information.

[0123] Step S503: Combine the point cloud video frame vector H′q obtained in step S502 with the reference detection vector. Two types of vector concatenation Then, using a feedforward neural network consisting of two linear layers, the image F of the weighed bridge (q) to be weighed and the reference detector (a1) are obtained. out ;

[0124] The working process of the second reference detection vector acquisition module includes the following steps:

[0125] Step S601: Convert the weighed grayscale image vector... After downsampling, feature vectors of the frame information before and after the reference detection vector a2 are obtained.

[0126] Step S602: The feature vector Hq of the bridge to be weighed q obtained in step S2 is used to obtain the feature vectors of the preceding and following frames of the point cloud video frame through a loss model. Then, the feature vectors of the point cloud video frames of the bridge q to be weighed are obtained. and weighed grayscale image vector First, the attention matrix is ​​obtained through matrix multiplication. After weighted summation along both column and row dimensions, the result of matrix M2 is: and Combine the feature vectors of mq and point cloud video frame q. Multiplying them yields the point cloud video frame vector H″q of the reference detection information. Compared with the weighed grayscale image vector Multiplying them yields the reference detection vector for the point cloud video frame information.

[0127] Step S603: Combine the point cloud video frame vector H″q obtained in step S602 with the reference detection vector of the point cloud video frame information. Two types of vector concatenation Then, using a feedforward neural network consisting of two linear layers, the image F of the weighed bridge (q) to be weighed and the reference detection vector a2 are obtained. out ;

[0128] The training module operates by including the following steps:

[0129] Step S701, the L1-LOSS and EDGE-LOSS evaluation values of the weighed gray picture sequence of the reference detection a1 obtained by the retrieval model and the standard reference detection a* corresponding to the bridge q to be weighed are calculated, and the L1-LOSS and EDGE-LOSS evaluation values are sequentially f(Yr, Y*) and g(Yr, Y*), and the L1-LOSS evaluation value and the EDGE-LOSS evaluation value are weighted and summed to obtain the confidence weight corresponding to the reference detection a1, and the L1-LOSS and EDGE-LOSS evaluation values of the weighed gray picture sequence of the reference detection vector a2 obtained by the generation model and the standard reference detection a* corresponding to the point cloud video frame q are calculated, and the L1-LOSS and EDGE-LOSS evaluation values are sequentially f(Yg, Y*) and g(Yg, Y*), and the L1-LOSS evaluation value and the EDGE-LOSS evaluation value are weighted and summed to obtain the confidence weight corresponding to the reference detection vector a2.

[0130] Step S702, the confidence weights obtained by the two reference detections are compared to obtain the final confidence weight.

[0131] Further, the L1-LOSS and EDGE-LOSS evaluation values are combined as the final similarity discrimination, and the most relevant weighed gray picture sequence is searched from the point cloud video set.

[0132] Further, in the training stage, the weighed gray picture is sent into the convolution head module for dimension correction and downsampling operation to obtain

[0133] Where B is the sample size, C1=3 is the P in channel dimension, H1=512, W1=512 are the height and width of P in , and C2=512 is the F out channel dimension, H2=16, W2=16 are the height and width of F out .

[0134] The calculation formula of the L1-LOSS and EDGE-LOSS evaluation values is as follows:

[0135] L1-Loss is used to calculate F out and GT, GT refers to Groundtruth, i.e. the absolute difference value of the RGB value on each video of the real weighed picture;

[0136] The formula is as follows:

[0137]

[0138] Where n represents the total number of videos on a picture; y irepresents the RGB value of the i-th video on the GT image; f(x i ) represents the RGB value of the i-th video on the GT image; f(x out

[0139] Edge-Loss is to ensure the clarity of the edges of each object in the detection area when color diffusion, and the formula is as follows:

[0140]

[0141] wherein X and Y are the F out and the real GT image, E is the edge feature obtained, and H and W are the height and width of the P in .

[0142] Further, if the EDGE-LOSS evaluation value of the retrieved bridge to be weighed and the original bridge to be weighed is not 0, the EDGE-LOSS evaluation value is used; otherwise, the L1-LOSS evaluation value is used.

[0143] Flowcharts are used in the present disclosure to illustrate the steps of the method according to the embodiments of the present disclosure. It should be understood that the preceding or subsequent steps are not necessarily completed in sequence. On the contrary, various steps can be processed in reverse order or simultaneously. Meanwhile, other operations can also be added to these processes.

[0144] A person of ordinary skill in the art can understand that all or part of the steps in the above method can be instructed by a computer program to complete relevant hardware, and the program can be stored in a computer readable storage medium, such as a read-only memory, a magnetic disk or an optical disk, etc. Alternatively, all or part of the steps of the above embodiments can also be implemented using one or more integrated circuits. Accordingly, each module in the above embodiments can be implemented in the form of hardware or in the form of a software functional module. The present disclosure is not limited to any specific form of combination of hardware and software.

[0145] Unless otherwise defined, all terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It will be further understood that terms, such as those defined in commonly associated tables, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and should not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0146] ​The foregoing is a summary of the disclosure, and shall not be considered a limitation as to the scope thereof. While several exemplary embodiments of the disclosure have been described, it will be apparent to those of ordinary skill in the art that many modifications, substitutions, and alterations to the examples described are possible without departing from the novel teachings and advantages of the disclosure. Thus, all such modifications, substitutions, and alterations are intended to be encompassed by the scope of the disclosure as defined by the appended claims. It should be understood that the description herein is intended to be illustrative, and is not considered limiting, and that modifications to the disclosed embodiments, as well as other embodiments, are intended to be included within the scope of the claims. The disclosure is defined by the claims and their equivalents.

[0147] In the description of the specification, reference to "one embodiment", "some embodiments", "an exemplary embodiment", "an example", "a specific example", or "some examples" means that a particular feature, structure, material, or characteristic being described is included in at least one embodiment or example of the application. The appearances of the phrases "in one embodiment", "in some embodiments", "in an exemplary embodiment", "an example", "a specific example", or "some examples" in various places in the specification are not necessarily referring to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics being described can be combined in any suitable manner in one or more embodiments or examples.

[0148] While embodiments of the application have been shown and described, it is to be understood that the application is not limited to the details of the embodiments described, since the scope of the application is defined with respect to the appended claims.

Claims

1. A method for weighing rotating bridges based on point cloud model algorithms, characterized in that, Includes the following steps: Step S1: Using the collected point cloud video set of pre-trained weighed point cloud videos, the point cloud video set is segmented using a segmentation algorithm. Based on the frequency of transformation of the weighed grayscale images in the point cloud video set, the weighed grayscale images are sorted to generate an association table mapping the weighed grayscale images to the bridge samples to be weighed for repair. The point cloud video frame vector of each point cloud video frame in the association table is pre-trained. Step S2: In the generative model, after segmenting the point cloud video frames of the bridge q to be weighed by the user, the sample sequence of the bridge to be weighed is obtained by looking up the association table. Then, using the point cloud video frame vector pre-trained in step S1, each bridge sample to be weighed in the sample sequence is converted into the corresponding point cloud video frame vector, thus obtaining the corresponding feature vector H of the bridge to be weighed. q ; Step S3: Using the weighed grayscale image sequence of point cloud video frames obtained in Step S1, a retrieval algorithm is used to retrieve the closest bridge to be weighed (q′) to be repaired (q′) from the point cloud video set, and the corresponding weighed grayscale image of the closest bridge to be weighed (q′) in the point cloud video set. ; Step S4: Based on the feature vector of the bridge to be weighed obtained in step S2, use a transformer convolution head to generate a weighed grayscale image vector related to the point cloud video frame. Step S5: The weighed grayscale images retrieved in Step S3 are segmented using Tokens to obtain a sequence of weighed grayscale images. Then, using the association table from Step S1, the sequence of weighed grayscale images is converted into a sequence of bridge samples to be weighed for repair. The Tokens point cloud video frame vector corresponding to each bridge sample to be weighed in the sequence of bridge samples to be weighed for repair is searched to obtain the feature vector H of the weighed grayscale images. a1 And the feature vector H of the bridge to be weighed in step S2. q The model is automatically updated sequentially using a mask, and then the mesh is corrected using one dimension to obtain the reference detection vector after obtaining the point cloud video frame information. Step S6: The weighed grayscale image vector generated in step S4 is... and the feature vector H of the bridge to be weighed in step S2 q The two samples are then processed sequentially through downsampling, and then combined and passed through a one-dimensional grid to obtain the feature vectors of the frame information before and after the reference detection vector a2. Step S7: Use the policy gradient algorithm to train the detection completion. The input is the sequence of weighed grayscale images of the bridge q to be weighed, and the output is the weighed images of the bridge that have been detected. ; In step S3, the L1-LOSS and EDGE-LOSS evaluation values ​​are combined as the final similarity criteria to search for the most relevant weighed grayscale image sequence from the point cloud video set. In step S3, During the training phase, grayscale images have been weighed. The data will first be fed into the convolution head module for dimension correction and downsampling operations. ; Where B is the sample size, and C1=3 is... Channel dimensions, H1=512, W1=512 are respectively The height and width, C2=512 is Channel dimensions, H2=16, W2=16 are respectively Height and width; The formulas for calculating the L1-LOSS and EDGE-LOSS evaluation values ​​are as follows: L1-Loss is used to calculate GT stands for Groundtruth, which is the absolute difference of the RGB values ​​of the actual weighed image across each video. The formula is as follows: in, This indicates the total number of videos on a single image; This represents the RGB value of the i-th video in the GT graph; express The RGB value of the i-th video; Edge-Loss ensures the sharpness of object edges within the detection area during color diffusion, and the formula is as follows: Where X and Y are respectively And the real GT image, E is the obtained edge feature, H and W are Height and width; In step S3, if the EDGE-LOSS evaluation value calculated by the retrieved bridge to be weighed is not 0 compared with that of the original bridge to be weighed, then the EDGE-LOSS evaluation value is used; otherwise, the L1-LOSS evaluation value is used.

2. The method for weighing rotating bridges based on point cloud model algorithm as described in claim 1, characterized in that: Step S1 further includes: Step S100: Use a timer to collect historical point cloud video of the bridge to be weighed that has been trained, and filter the data to obtain a set of point cloud video of the historical bridge to be weighed. Step S101: In the pre-training point cloud video frame vector stage, the cutting algorithm in point cloud video image parsing is used to cut the bridge to be weighed and the grayscale image that has been weighed in each sample of the point cloud video set. After deduplication of all point cloud video frames in the point cloud video set, an association table is established, and the point cloud video frame vector of commonly used point cloud video frames is obtained by pre-training using the Tokens algorithm. Step S4 further includes: Step S401: Input the feature vector H of the bridge to be weighed into the encoder end of the Transformer network. q By fusing the full-text information of the bridge to be weighed through a global attention mechanism, a feature vector H′ of the bridge to be weighed, which integrates the full-text information of the point cloud video set, is further obtained. q ; Step S402: Using the decoder of the Transformer network, input the feature vector H′ of the bridge to be weighed, which integrates the full-text information of the point cloud video set. q Output the corresponding weighed grayscale image vectors sequentially. All point cloud video frame vectors; 3. A weighing system for rotating bridges based on point cloud model algorithms, comprising: The training module collects a set of point cloud videos of pre-weighed grayscale images and uses a segmentation algorithm to segment all point cloud video sets. Based on the frequency of transformation of the weighed grayscale images in the point cloud video set, the weighed grayscale images are sorted, and an association table mapping the weighed grayscale images to the bridge samples to be weighed for repair is generated. The module also pre-trains the point cloud video frame vector for each point cloud video frame in the association table. The convolutional head processing module is used to segment the point cloud video frames of the bridge to be weighed q to be processed by the user, obtain the sample sequence of the bridge to be weighed to be repaired by looking up the association table, and use the point cloud video frame vectors pre-trained in step S1 to convert each bridge sample in the sample sequence of the bridge to be weighed to be repaired into the corresponding point cloud video frame vector, thereby obtaining the corresponding feature vector H of the bridge to be weighed. q ; The global residual connection module is used to retrieve, based on a retrieval algorithm, the weighed grayscale image sequence of point cloud video frames obtained from the training module, the closest bridge to be weighed (q′) to be repaired (q′), and the corresponding weighed grayscale image of the closest bridge to be weighed (q′) in the point cloud video set. ; The style-capacity module is used to generate a weighed grayscale image vector associated with the point cloud video frame based on the feature vector of the bridge to be weighed obtained from the convolution head processing module and using a transformer convolution head. The first reference detection vector acquisition module is used to obtain a sequence of weighed grayscale images from the global residual connection module, after segmentation, and then convert the sequence of weighed grayscale images into a sequence of bridge samples to be weighed for repair using the association table in the training module. It then searches for the Tokens point cloud video frame vector corresponding to each bridge sample in the sequence of bridge samples to be weighed for repair, thus obtaining the feature vector H of the weighed grayscale images. a1 And together with the feature vector H of the bridge to be weighed in the convolution head processing module. q The model is automatically updated sequentially using a mask, and then the mesh is corrected using one dimension to obtain the reference detection vector after obtaining the point cloud video frame information. The second reference detection vector acquisition module is used to obtain the weighted grayscale image vector generated in the style integration module. and the feature vector H of the bridge to be weighed in the convolution head processing module q The two samples are then processed sequentially through downsampling, and then combined and passed through a one-dimensional grid to obtain the feature vectors of the frame information before and after the reference detection vector a2. The testing module is used to test the completion of detection using the policy gradient algorithm. Its input is a sequence of weighed grayscale images of the bridge q to be weighed, and its output is an image of the weighed bridge that has been detected. .

4. The rotating bridge weighing system based on point cloud model algorithm as described in claim 3, characterized in that: The training module operates by including the following steps: Step S100: Use a timer to collect historical point cloud video of the bridge to be weighed that has been trained, and filter the data to obtain a set of point cloud video of the historical bridge to be weighed. Step S101: In the pre-training point cloud video frame vector stage, the cutting algorithm in point cloud video image parsing is used to cut the bridge to be weighed and the grayscale image that has been weighed in each sample of the point cloud video set. After deduplication of all point cloud video frames in the point cloud video set, an association table is established, and the point cloud video frame vector of commonly used point cloud video frames is obtained by pre-training using the Tokens algorithm. The working process of the style compatibility module includes the following steps: Step S401: Input the feature vector H of the bridge to be weighed into the encoder end of the Transformer network. q By fusing the full-text information of the bridge to be weighed through a global attention mechanism, a feature vector H′ of the bridge to be weighed, which integrates the full-text information of the point cloud video set, is further obtained. q ; Step S402: Using the decoder of the Transformer network, input the feature vector H′ of the bridge to be weighed, which integrates the full-text information of the point cloud video set. q Output the corresponding weighed grayscale image vectors sequentially. All point cloud video frame vectors.

5. The rotating bridge weighing system based on point cloud model algorithm as described in claim 4, characterized in that: By combining L1-LOSS and EDGE-LOSS evaluation values ​​as the final similarity criterion, the most relevant weighed grayscale image sequence is searched from the point cloud video set.

6. The rotating bridge weighing system based on point cloud model algorithm as described in claim 5, characterized in that, During the training phase, grayscale images have been weighed. The data will first be fed into the convolution head module for dimension correction and downsampling operations. ; Where B is the sample size, and C1=3 is... Channel dimensions, H1=512, W1=512 are respectively The height and width, C2=512 is Channel dimensions, H2=16, W2=16 are respectively Height and width; The formulas for calculating the L1-LOSS and EDGE-LOSS evaluation values ​​are as follows: L1-Loss is used to calculate GT stands for Groundtruth, which is the absolute difference of the RGB values ​​of the actual weighed image across each video. The formula is as follows: in, This indicates the total number of videos on a single image; This represents the RGB value of the i-th video in the GT graph; express The RGB value of the i-th video; Edge-Loss ensures the sharpness of object edges within the detection area during color diffusion, and the formula is as follows: Where X and Y are respectively And the real GT image, E is the obtained edge feature, H and W are Height and width.

7. A bridge rotation weighing system based on a point cloud model algorithm as described in claim 6, characterized in that: If the EDGE-LOSS evaluation value calculated from the retrieved bridge to be weighed is not 0 compared with that of the original bridge to be weighed, then the EDGE-LOSS evaluation value is used; otherwise, the L1-LOSS evaluation value is used.

Images