Real-time imaging robot putting method, system, equipment and medium

Abstract

The invention relates to the technical field of image processing, and discloses a real-time imaging robot putting method, system, equipment and medium, the method comprises the following steps: acquiring an arc vertical line image sequence, environment wind power data and line parameters of a target putting line on a time sequence, determining the current swing amplitude of an arc vertical line, and predicting the future swing amplitude of the arc vertical line; performing swing trend analysis on the sagittaria sagitta, predicting future position distribution of the sagittaria sagitta according to the swing trend data and future swing amplitude, thereby dynamically adjusting a putting arrangement scheme of the imaging robot, ensuring that the imaging robot can accurately capture a real-time state of a target line, and determining the real-time state of the target line according to the future position distribution of the sagittaria sagitta. According to the embodiment of the invention, a plurality of candidate delivery arrangement schemes of the to-be-delivered imaging robot are generated, and the optimal delivery arrangement scheme for imaging the target delivery line is screened out from the plurality of candidate delivery arrangement schemes, so that the consumption of excessive time is avoided, and meanwhile, the delivery accuracy can be ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a real-time imaging robot launching method, system, device and medium. BACKGROUND

[0002] High-voltage transmission lines bear the important task of transporting electric energy from power stations to user terminals. In high-voltage transmission lines, strain clamps are one of the key components connecting cables and power towers on the line, and the quality of strain clamps directly affects the safe and stable operation of the power system. Therefore, strain clamps need to be checked regularly to find problems in time.

[0003] Traditional manual inspection methods not only have low inspection efficiency and accuracy, but also have high-altitude operation risks, electric shock risks and other problems, which can easily cause injuries or accidents to workers, so imaging robots are introduced for inspection.

[0004] Among them, in order to accurately launch the imaging robot to be connected with the high-voltage line, the launch pose of the imaging robot needs to be analyzed. At present, a simulation experiment method is usually used to plan the launch pose, but this method takes too long and relies too much on the parameter and rule settings of related personnel, which may miss analysis conditions or parameters, resulting in insufficient guarantee of the accuracy of the launch pose analysis and affecting the reliability of the imaging robot launch. SUMMARY

[0005] Therefore, in order to solve the above technical problems, the present application provides a real-time imaging robot launching method, system, device and medium.

[0006] The first aspect of the present application provides a real-time imaging robot launching method, comprising:

[0007] Collecting an image sequence of sagging lines, environmental wind data and line parameters of a target launch line in time sequence;

[0008] According to the sagging line image sequence, the environmental wind data and the line parameters, determining a current swing amplitude of the sagging line, and predicting a future swing amplitude of the sagging line based on the current swing amplitude;

[0009] According to the sagging line image sequence, performing swing trend analysis on the sagging line to obtain swing trend data of the sagging line;

[0010] According to the swing trend data and the future swing amplitude, predicting a future position distribution of the sagging line;

[0011] generate a plurality of candidate deployment arrangement schemes of the imaging robot to be deployed according to the future position distribution of the catenary line, and screen a candidate deployment arrangement scheme best for imaging the target deployment line from the plurality of candidate deployment arrangement schemes as the deployment arrangement scheme of the imaging robot to be deployed;

[0012] deploy the imaging robot to be deployed according to the deployment arrangement scheme.

[0013] Preferably, the method further comprises: image preprocessing the catenary line image sequence; the image preprocessing the catenary line image sequence comprises:

[0014] color channel compensation is performed on the catenary line image sequence to obtain a color channel compensated catenary line image sequence;

[0015] color balance is performed on the color channel compensated catenary line image sequence to obtain a color balanced catenary line image sequence;

[0016] filter processing is performed on the color balanced catenary line image sequence to obtain a filter processed catenary line image sequence as the catenary line image sequence after image preprocessing.

[0017] Preferably, the environmental wind force data comprises horizontal wind force and vertical wind force; and the line parameters comprise span data;

[0018] the current swing amplitude of the catenary line is determined according to the catenary line image sequence, the environmental wind force data and the line parameters, and the future swing amplitude of the catenary line is predicted based on the current swing amplitude, which comprises:

[0019] a synthetic wind force is determined according to the vector square sum between the horizontal wind force and the vertical wind force;

[0020] the drag and lift of the target deployment line are determined according to the synthetic wind force, the span data, preset fluid drag and preset fluid density;

[0021] a first swing amplitude of the catenary line is determined according to the drag, the lift and preset aerodynamic force;

[0022] target features and coordinate positions of the target features in the catenary line image sequence are extracted, distances between coordinate positions of the target features in a plurality of adjacent frame images in the catenary line image sequence are determined, and a distance between coordinate positions of adjacent frame images with the largest distance is screened as a second swing amplitude;

[0023] the first swing amplitude and the second swing amplitude are weighted and fused to obtain the current swing amplitude;

[0024] Based on the current swing amplitude, in combination with a pre-trained swing amplitude prediction model, a future swing amplitude of the sagging line is predicted; wherein the pre-trained swing amplitude prediction model is obtained by training a long short-term memory network based on historical sagging line swing amplitude data.

[0025] Preferably, the swing trend analysis of the sagging line based on the sequence of sagging line images to obtain the swing trend data of the sagging line comprises:

[0026] Difference operation is performed on each adjacent frame image in the sequence of sagging line images to obtain a difference image;

[0027] Edge extraction is performed on each difference image according to an edge extraction operator to obtain a plurality of edge feature images;

[0028] Connected domain analysis is performed on each edge feature image to obtain a swing edge contour of the sagging line;

[0029] Based on the swing edge contour, the swing center position of the sagging line in each frame image is calculated;

[0030] According to the position change of the swing center position in each adjacent frame image and the time step of adjacent frames, the position change direction, position change speed and position change acceleration of the swing center position are determined;

[0031] The position change direction, position change speed and position change acceleration of the swing center position are taken as the swing trend data of the sagging line.

[0032] Preferably, the prediction of the future position distribution of the sagging line based on the swing trend data and the future swing amplitude comprises:

[0033] The swing trend data and the future swing amplitude are input into a pre-trained position prediction model, so that the pre-trained position prediction model outputs a future position heat map of the sagging line; wherein the pre-trained position prediction model is obtained by training a convolutional neural network based on historical sagging line position data, swing trend data samples and future swing amplitude samples;

[0034] According to the future position heat map of the sagging line, the future position distribution of the sagging line is determined.

[0035] Preferably, the generation of a plurality of candidate deployment arrangement schemes of the imaging robot to be deployed based on the future position distribution of the sagging line, and the screening of a candidate deployment arrangement scheme best for imaging the target deployment line from the plurality of candidate deployment arrangement schemes as the deployment arrangement scheme of the imaging robot to be deployed, comprises:

[0036] determine a plurality of candidate delivery arrangement schemes based on the future position distribution of the sag line; wherein each of the candidate delivery arrangement schemes comprises a candidate delivery point and a candidate delivery pose corresponding to the candidate delivery point; the candidate delivery pose comprises a candidate delivery angle;

[0037] for each of the candidate delivery arrangement schemes, determine an imaging quality of the imaging robot to be delivered to the target delivery route under the candidate delivery arrangement scheme according to a relative position relationship between the candidate delivery point and the target delivery route, and a coverage range of the target delivery route by the candidate delivery angle;

[0038] select a candidate delivery arrangement scheme with the best imaging quality from the plurality of candidate delivery arrangement schemes as the delivery arrangement scheme of the imaging robot to be delivered.

[0039] Preferably, the determination of the imaging quality of the imaging robot to be delivered to the target delivery route under the candidate delivery arrangement scheme according to the relative position relationship between the candidate delivery point and the target delivery route, and the coverage range of the target delivery route by the candidate delivery angle comprises:

[0040] determine a distance between the imaging robot to be delivered and the target delivery route according to the relative position relationship between the candidate delivery point and the target delivery route;

[0041] calculate a theoretical coverage range of the target delivery route by the imaging robot to be delivered according to the candidate delivery angle and the distance;

[0042] determine an actual imaging range of the target delivery route by the imaging robot to be delivered under the candidate delivery arrangement scheme in combination with the theoretical coverage range and imaging parameters of the imaging robot to be delivered;

[0043] determine the imaging quality of the imaging robot to be delivered to the target delivery route under the candidate delivery arrangement scheme according to a difference between the actual imaging range and a preset expected imaging range; wherein the smaller the difference is, the better the imaging quality is.

[0044] In a second aspect, the present application further provides a delivery system of a real-time imaging robot, comprising:

[0045] a data acquisition module configured to acquire a sequence of sag line images, environmental wind data and line parameters of a target delivery route over time;

[0046] a swing amplitude prediction module, configured to determine a current swing amplitude of the sagging line according to the sagging line image sequence, the environmental wind force data and the line parameter, and predict a future swing amplitude of the sagging line based on the current swing amplitude;

[0047] a swing trend analysis module, configured to perform swing trend analysis on the sagging line according to the sagging line image sequence, and obtain swing trend data of the sagging line;

[0048] a position distribution prediction module, configured to predict a future position distribution of the sagging line according to the swing trend data and the future swing amplitude;

[0049] a placement arrangement screening module, configured to generate a plurality of candidate placement arrangement schemes of the imaging robot to be placed according to the future position distribution of the sagging line, and screen a candidate placement arrangement scheme best for imaging the target placement line from the plurality of candidate placement arrangement schemes as the placement arrangement scheme of the imaging robot to be placed;

[0050] a placement execution module, configured to place the imaging robot to be placed according to the placement arrangement scheme.

[0051] In a third aspect, the present application further provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the real-time imaging robot placement method according to the first aspect.

[0052] In a fourth aspect, the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed to implement the steps of the real-time imaging robot placement method according to the first aspect.

[0053] As can be seen from the above technical solutions, the present application collects the sagging line image sequence of the target placement line in time sequence, environmental wind force data and line parameter, determines the current swing amplitude of the sagging line, and predicts the future swing amplitude of the sagging line, further performs swing trend analysis on the sagging line, predicts the future position distribution of the sagging line according to the swing trend data and the future swing amplitude, dynamically adjusts the placement arrangement scheme of the imaging robot, ensures that the imaging robot can accurately capture the real-time state of the target line, generates a plurality of candidate placement arrangement schemes of the imaging robot to be placed according to the future position distribution of the sagging line, and screens a placement arrangement scheme best for imaging the target placement line from the plurality of candidate placement arrangement schemes, thereby avoiding wasting too much time, ensuring the accuracy of placement, making the imaging quality of the real-time imaging robot higher, and improving the reliability of the imaging robot placement. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to make the technical solutions of the embodiments of the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description only aim to some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor.

[0055] Figure 1 An application environment diagram of the real-time imaging robot delivery method provided by the embodiments of the present application is shown in the figure;

[0056] Figure 2 A flowchart of the real-time imaging robot delivery method provided by the embodiments of the present application is shown in the figure;

[0057] Figure 3 A structural schematic diagram of the real-time imaging robot delivery system provided by the embodiments of the present application is shown in the figure;

[0058] Figure 4 A structural schematic diagram of the electronic device provided by the embodiments of the present application is shown in the figure. DETAILED DESCRIPTION

[0059] In order to make the technical solutions of the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely by combining the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0060] The real-time imaging robot delivery method provided by the embodiments of the present application can be applied to, for example Figure 1The application environment shown. Among them, the terminal 101 communicates with the server 102 through the network. The data storage system can store the data required by the server 102 to process. The data storage system can be integrated on the server 102, or placed on the cloud or other network servers. The terminal 101 or the server 102 collects the sag line image sequence of the target laying line in time sequence, environmental wind data and line parameters; according to the sag line image sequence, the environmental wind data and the line parameters, the current swing amplitude of the sag line is determined, and the future swing amplitude of the sag line is predicted based on the current swing amplitude; according to the sag line image sequence, the swing trend of the sag line is analyzed, and the swing trend data of the sag line is obtained; according to the swing trend data and the future swing amplitude, the future position distribution of the sag line is predicted; according to the future position distribution of the sag line, a plurality of candidate laying arrangement schemes of the imaging robot to be laid are generated, and the candidate laying arrangement scheme best for imaging the target laying line is selected from the plurality of candidate laying arrangement schemes as the laying arrangement scheme of the imaging robot to be laid; according to the laying arrangement scheme, the imaging robot to be laid is laid.

[0061] The terminal 101 can be, but is not limited to, various personal computers, notebook computers, smart phones and tablet computers, etc.

[0062] The server 102 can be a stand-alone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0063] As Figure 2 shown, the embodiment of the application provides a real-time imaging robot laying method. Taking the terminal 101 or the server 102 in the Figure 1 as an example, the method comprises the following steps S1 to S6. Among them:

[0064] Step S1, collecting the sag line image sequence of the target laying line in time sequence, environmental wind data and line parameters.

[0065] Among them, the target laying line is a specific line that needs to be laid and monitored by the real-time imaging robot. These lines may be in complex environments, such as mountainous areas, river-crossing areas, etc., and the swing of the sag line is directly related to the safe operation of the line.

[0066] The sag line image sequence can be obtained by continuously shooting with cameras installed in appropriate positions. These cameras should have the ability of high-definition and stable shooting to ensure the clarity and accuracy of the images.

[0067] The environmental wind force data can be collected by wind speed sensors, wind direction sensors and the like, which should be able to measure the size and direction of horizontal and vertical wind forces in real time and accurately. The line parameters, such as span data, can be obtained through line design documents or field measurements.

[0068] Step S2, according to the sagitta image sequence, the environmental wind force data and the line parameters, the current swing amplitude of the sagitta is determined, and the future swing amplitude of the sagitta is predicted based on the current swing amplitude.

[0069] Wherein, the future swing amplitude is predicted by the current swing amplitude of the sagitta, which can predict the swing of the sagitta in advance, and provide a key basis for the subsequent deployment of the imaging robot.

[0070] Step S3, the swing trend of the sagitta is analyzed according to the sagitta image sequence, and the swing trend data of the sagitta is obtained.

[0071] Wherein, the swing trend analysis of the sagitta is to accurately grasp the dynamic change rule of the sagitta, and the swing trend data of the sagitta can be obtained by in-depth analysis of the sagitta image sequence. These data cover key information such as swing direction, speed and acceleration, which helps to fully understand the swing characteristics of the sagitta. Through detailed analysis of these data, the future position distribution of the sagitta can be further predicted.

[0072] Step S4, according to the swing trend data and the future swing amplitude, the future position distribution of the sagitta is predicted.

[0073] Wherein, through the swing trend data and the future swing amplitude, the possible position distribution model of the sagitta at future time can be predicted. This model is presented in the form of a heat map, which intuitively shows the probability of each position point of the sagitta.

[0074] Step S5, according to the future position distribution of the sagitta, a plurality of candidate deployment arrangement schemes of the imaging robot to be deployed are generated, and the candidate deployment arrangement scheme best for imaging the target line is selected from the plurality of candidate deployment arrangement schemes as the deployment arrangement scheme of the imaging robot to be deployed.

[0075] Wherein, based on the future position distribution of the sagitta, a plurality of candidate deployment arrangement schemes are generated. These schemes not only consider the possible position of the sagitta, but also comprehensively evaluate the imaging quality of each candidate scheme for the target line in the screening process, including the coverage of imaging. Finally, the candidate scheme with the best imaging quality is selected as the actual deployment arrangement scheme to ensure that the imaging robot can accurately and comprehensively capture the real-time state of the target line.

[0076] Step S6, according to the deployment arrangement scheme, the imaging robot to be deployed is deployed.

[0077] According to the optimal placement scheme screened out, the placement position and posture of the imaging robot are accurately controlled. During the placement process, it is necessary to ensure that the robot can stably and accurately reach the specified position and be adjusted to the optimal imaging angle. After the placement is completed, the imaging robot will start working and collect image data of the target placement line in real time.

[0078] It should be noted that, by collecting the sag curve image sequence of the target placement line in time sequence, environmental wind force data and line parameters, the current swing amplitude of the sag curve is determined, and the future swing amplitude of the sag curve is predicted. In addition, the swing trend of the sag curve is analyzed, and according to the swing trend data and the future swing amplitude, the future position distribution of the sag curve is predicted, so as to dynamically adjust the placement scheme of the imaging robot, ensure that the imaging robot can accurately capture the real-time state of the target line, and generate multiple candidate placement schemes of the imaging robot to be placed according to the future position distribution of the sag curve, and screen out the optimal placement scheme for imaging the target line from the multiple candidate placement schemes, so as to avoid wasting too much time, while ensuring the accuracy of the placement, making the imaging quality of the real-time imaging robot higher, and improving the reliability of the imaging robot placement.

[0079] In some embodiments, in order to ensure the clarity and accuracy of the image and avoid affecting the subsequent image analysis, the method further includes: image preprocessing of the sag curve image sequence; the image preprocessing of the sag curve image sequence includes:

[0080] Step S11, color channel compensation is performed on the sag curve image sequence to obtain a sag curve image sequence after color channel compensation.

[0081] In the color channel compensation, the sag curve image sequence is decomposed into an R channel image component, a G channel image component and a B channel image component, the average values of the R channel, the G channel and the B channel are calculated, the R channel image component, the G channel image component and the B channel image component are normalized according to the average values, the other two channel image components are compensated according to the channel image component corresponding to the maximum value in the normalization, and the compensated R channel image component, G channel image component and B channel image component are combined to obtain the sag curve image sequence after color channel compensation.

[0082] Step S12, color balance is performed on the sag curve image sequence after color channel compensation to obtain a sag curve image sequence after color balance.

[0083] The gray world white balance algorithm is used to balance the color of the catenary image sequence after color channel compensation, so that the gray mean value of the RGB channel of the catenary image sequence can be adjusted, color deviation can be eliminated, and the catenary image sequence after color adjustment can be obtained.

[0084] In step S13, the catenary image sequence after color balance is filtered to obtain a catenary image sequence after filtering as the catenary image sequence after image preprocessing.

[0085] The Gaussian Laplacian operator can be used for filtering.

[0086] In some embodiments, the environmental wind force data includes horizontal wind force and vertical wind force, and the line parameter includes span data. In this case, the current swing amplitude of the catenary is determined according to the catenary image sequence, the environmental wind force data and the line parameter, and the future swing amplitude of the catenary is predicted based on the current swing amplitude, including:

[0087] In step S201, the synthetic wind force is determined according to the vector square sum between the horizontal wind force and the vertical wind force.

[0088] The synthetic wind force is the result of considering the combined action of the horizontal wind force and the vertical wind force. By calculating the vector square sum of the two, the size of the synthetic wind force can be accurately obtained, and more comprehensive wind force data can be provided for subsequent analysis of catenary swing.

[0089] In step S202, the resistance and lift of the target delivery line are determined according to the synthetic wind force, the span data, the preset fluid resistance and the preset fluid density.

[0090] The preset fluid resistance is a parameter preset based on the principle of fluid dynamics, which is used to describe the resistance of the target delivery line in the fluid. The value generally depends on the Reynolds number of the airflow. For example, when the Reynolds number is 10 3 ~10 4 , the preset fluid resistance is 1~1.2, and when the Reynolds number is 10 4 ~10 5 , the preset fluid resistance is 0.8~1. The preset fluid density is a parameter describing the density of the fluid, which is generally the density of air.

[0091] The drag of the target deployment path is the force that hinders its movement when the combined wind force interacts with the target deployment path, while the lift is the force perpendicular to the direction of the combined wind force, causing the target deployment path to rise. By comprehensively considering the magnitude of the combined wind force, the span data (i.e., the distance between the two ends of the path), the preset fluid resistance, and the preset fluid density, the drag and lift of the target deployment path under the current wind conditions can be accurately calculated. The specific calculation process (all parameters involved in the calculation below are dimensionless parameters) is as follows:

[0092] Calculate the product of the span data and the combined wind force, and then calculate the ratio of the preset fluid resistance to this product to obtain the resistance. Based on the span data, combined wind force, preset fluid density, and torsional angular velocity per unit length, calculate the Magnus force. The Magnus force is the lateral force generated by the velocity difference between the two sides of a rotating object in a fluid. The Magnus force is:

[0093]

[0094] In the formula, For Magnus force, To preset fluid density, To synthesize wind power, The torsional angular velocity per unit length, For gear distance, This is the Magnus force correction factor, typically taken as 0.8 to 1.2.

[0095] Lift is calculated based on Magnus force, combined wind force, preset fluid density, and preset maximum cross-sectional area of ​​particles:

[0096]

[0097] In the formula, For lift, S is the preset maximum cross-sectional area of ​​the particle, that is, the maximum cross-sectional area per unit length.

[0098] Step S203: Determine the first swing amplitude of the sag line based on the drag, lift, and preset aerodynamic force.

[0099] Among them, the aerodynamic force is the force acting on the high-voltage line when it moves relative to the air. The aerodynamic force is obtained by simulating the force on the high-voltage line through a preset simulated air flow field. The preset aerodynamic force is based on the aerodynamic force of the high-voltage line in the scene analyzed by the preset simulated air flow field. The three-dimensional model of the line is input into the simulation software for fluid simulation analysis to obtain the simulated air flow field. Based on the simulated air flow field, the force on the high-voltage line in the air is analyzed and the aerodynamic force is output.

[0100] The first oscillation amplitude is obtained by dividing the partial derivative of the aerodynamic force by the partial derivative of the sum of the drag and lift data:

[0101]

[0102] In the formula, This is the first swing amplitude. It is aerodynamic.

[0103] Step S204: Extract the target features and coordinate positions of the target features from the sag line image sequence, determine the distance between the coordinate positions of the target features in multiple adjacent frame images in the sag line image sequence, and select the distance between the coordinate positions of the adjacent frame images with the largest distance as the second swing amplitude.

[0104] The target feature is a point or region on the sag line that is easily identifiable and has distinct characteristics, such as the lowest point on the sag line. Using image recognition technology, the target features are extracted from the sag line image sequence, and their coordinate positions in each frame are recorded. Next, the movement trajectory of these target features in multiple adjacent frames is tracked within the sag line image sequence, and the distance between their coordinate positions in different frames is calculated. By comparing these distances, the distance between the coordinate positions in the adjacent frames with the largest distance is selected; this distance represents the second swing amplitude.

[0105] Step S205: Weighted fusion of the first swing amplitude and the second swing amplitude to obtain the current swing amplitude.

[0106] The first sway amplitude is calculated considering the environmental dimension, while the second sway amplitude is calculated considering the image dimension. A weighted fusion of the first and second sway amplitudes is performed. The weights of the first and second sway amplitudes can be set according to the actual application scenario and requirements. For example, the weight of the first sway amplitude can be set to 0.6, and the weight of the second sway amplitude to 0.4. The two are then weighted and summed to obtain the current sway amplitude that comprehensively considers both environmental and image dimensions. This weighted fusion method can more comprehensively and accurately reflect the actual sway of the sloping line. After obtaining the current sway amplitude, it can be used to predict the sway amplitude of the sloping line in the future based on this current sway amplitude, combined with historical data and empirical models.

[0107] Step S206: Based on the current swing amplitude, and combined with the pre-trained swing amplitude prediction model, predict the future swing amplitude of the sag line; wherein, the pre-trained swing amplitude prediction model is obtained by training a long short-term memory network based on historical sag line swing amplitude data.

[0108] The sag amplitude prediction model is based on a Long Short-Term Memory (LSTM) network, which possesses powerful sequential data processing capabilities, making it particularly suitable for handling time-dependent data. During training, historical sag amplitude data of the sag line is used as input, and by continuously adjusting the network parameters, the model learns the changing patterns of the sag amplitude. After sufficient training, the model can accurately predict the sag amplitude of the sag line over a future period based on the current amplitude. This prediction method not only considers the regularity of historical data but also incorporates current real-time data, thereby improving the accuracy and reliability of the prediction. In practical applications, the prediction time range can be adjusted as needed to meet the requirements of different scenarios.

[0109] In some embodiments, the sway trend analysis of the sloping line is performed based on the sloping line image sequence to obtain the sloping trend data of the sloping line, including:

[0110] Step S301: Perform a difference operation on each adjacent frame image in the vertical line image sequence to obtain a difference image.

[0111] This process involves extracting consecutive frames from a sequence of sloping line images, performing a difference operation between the preceding and following frames, and calculating the grayscale differences of corresponding pixels to generate a series of difference images. These difference images highlight the changes in the sloping line at different times, especially the positional shift caused by its swaying.

[0112] Step S302: Perform edge extraction on each difference image according to the edge extraction operator to obtain multiple edge feature images.

[0113] Edge extraction operators can identify regions in the difference image where grayscale values ​​change drastically; these regions often correspond to the edges of sloping lines. By applying edge extraction operators such as Sobel and Canny, multiple edge feature images can be extracted from the difference image.

[0114] Step S303: Perform connected component analysis on each edge feature image to obtain the swaying edge contour of the sag line.

[0115] Connected component analysis is an image processing technique used to identify interconnected pixel regions in an image. Performing connected component analysis on each edge feature image allows us to determine the swaying edge contours of the sag line in each difference image. These contours reflect the swaying of the sag line at different times, providing foundational data for subsequent trend analysis.

[0116] Step S304: Based on the swaying edge contour, calculate the swaying center position of the sag line in each frame image.

[0117] Specifically, by calculating the geometric center of the swaying edge contour or by calculating the average coordinates of the contour points, the swaying center position of the sag line in each frame of the image can be determined. These center positions represent the swaying center of the sag line at different times.

[0118] Step S305: Based on the position change of the swing center position in each adjacent frame image and the time step of the adjacent frames, determine the position change direction, position change speed and position change acceleration of the swing center position.

[0119] By comparing the coordinate differences of the swing center position in adjacent frames and combining the time interval between adjacent frames, the direction of position change of the swing center position (i.e., the direction of the swing of the sag), the velocity of position change (i.e., the distance the swing center moves per unit time), and the acceleration of position change (i.e., the rate of change of the velocity of position change) can be accurately calculated. These parameters together constitute the swing trend data of the sag.

[0120] Specifically, the direction of position change can be determined by calculating the coordinate difference vector between the swing centers of adjacent frames; the velocity of position change can be obtained by dividing the position change distance by the time step; and the acceleration of position change can be obtained by calculating the ratio of the difference in position change velocity to the time step. These calculations are all based on the fundamental principles of image processing and mathematical analysis, ensuring the accuracy and reliability of the results.

[0121] Step S306: Use the direction of position change, speed of position change, and acceleration of position change of the swing center position as the swing trend data of the vertical arc.

[0122] In some embodiments, predicting the future positional distribution of the plumb line based on oscillation trend data and future oscillation amplitude includes:

[0123] Step S401: Input the swing trend data and future swing amplitude into the pre-trained position prediction model, so that the pre-trained position prediction model outputs a heat map of the future position of the sag line; wherein, the pre-trained position prediction model is obtained by training a convolutional neural network based on historical sag line position data, swing trend data samples and future swing amplitude samples.

[0124] The training process of the location prediction model is as follows: First, a large amount of historical sag line location data, swing trend data samples, and future swing amplitude samples are collected. These sample data should cover different environmental conditions, line parameters, and time spans to ensure the model's generalization ability. The collected sample data is divided into training and test sets. The training set is used for model parameter learning, while the test set is used to evaluate the model's prediction performance through K-fold cross-validation. Dynamically dividing the dataset avoids overfitting and underfitting and improves the reliability of the evaluation results. K-fold cross-validation is used to determine whether the model passes the validation. If it passes the validation, the model training can be stopped, and the location prediction model is obtained. During the training process, a convolutional neural network is used as the basic architecture. This network uses a combination of convolutional layers, pooling layers, and fully connected layers to fully extract features from the input data and make effective predictions. Convolutional layers are responsible for extracting local features from the input data, pooling layers are used to reduce the dimensionality of the feature map and enhance the robustness of the model, and fully connected layers integrate the extracted features and output the final prediction result. During training, the training set data is input into the convolutional neural network, and the network parameters are continuously adjusted through the backpropagation algorithm to minimize the error between the model's predicted results and the true values.

[0125] After training, the model's final performance is evaluated using test set data to ensure high predictive accuracy and reliability. After sufficient training, this location prediction model can accurately predict the future location heatmap of a sag line based on input sway trend data and future sway amplitude. This heatmap uses different colors or grayscale values ​​to represent the probability or density of the sag line appearing at different future locations, providing important reference data for the subsequent deployment of imaging robots. In practical applications, the input parameters and output format of the location prediction model can be adjusted as needed to meet the requirements of different scenarios.

[0126] Step S402: Determine the future position distribution of the sag line based on the heat map of the future position of the sag line.

[0127] The future location heatmap, presented visually, shows the possible future locations of the sag line and their corresponding probabilities or densities. By analyzing the color distribution or grayscale value changes in the heatmap, the main activity areas of the sag line and its potential extreme locations can be determined over a future period. Specifically, the areas with the darkest color or highest grayscale value in the heatmap can be selected as high-probability areas for the sag line's future location; these areas represent the most likely locations where the sag line will appear. Furthermore, the shape and distribution characteristics of the heatmap can be combined to further analyze the future movement trends and possible change patterns of the sag line. For example, if the heatmap shows a clear direction or trend, it can be inferred that the sag line may swing or move along a specific direction in the future. This analytical approach provides more accurate and reliable location information for the subsequent deployment of imaging robots, ensuring that the robots are deployed at the optimal time and location, improving imaging quality and efficiency.

[0128] In some embodiments, based on the future position distribution of the sag line, multiple candidate deployment arrangements for the imaging robot to be deployed are generated, and the candidate deployment arrangement that best images the target deployment line is selected from the multiple candidate deployment arrangements as the deployment arrangement for the imaging robot to be deployed, including:

[0129] Step S501: Based on the future position distribution of the vertical line, determine multiple alternative deployment schemes; wherein, each alternative deployment scheme includes a alternative deployment point and the alternative deployment pose corresponding to the alternative deployment point; the alternative deployment pose includes the alternative deployment angle.

[0130] Based on the future location distribution of the plumb line, the high-probability areas and potential activity ranges of the plumb line are identified, and multiple candidate delivery points are planned within these key areas. For each candidate delivery point, its corresponding delivery pose, including the angle at which it is delivered, is further determined.

[0131] For example, if the plumb line is expected to oscillate primarily on one side, then the deployment angle of the backup deployment points planned on that side should be tilted towards the area where the plumb line is likely to appear, in order to better capture its dynamics. In this way, a suitable deployment position is determined for each backup deployment point, thus forming multiple backup deployment schemes.

[0132] Step S502: For each alternative deployment scheme, based on the relative positional relationship between the alternative deployment point and the target deployment route, and the coverage of the target deployment route by the alternative deployment angle, determine the imaging quality of the imaging robot to be deployed on the target deployment route under the alternative deployment scheme.

[0133] This method allows for simulated deployment of each candidate deployment scheme. During the simulation, 3D modeling technology is used to construct virtual models of the target deployment route and the imaging robot. The imaging robot is placed in the virtual environment according to the candidate deployment points and poses in the candidate deployment schemes, and then the imaging process is simulated. Based on the imaging principles, the relative positional relationship between the candidate deployment points and the target deployment route, as well as the coverage area of ​​the target deployment route by the candidate deployment angle, are analyzed. Through this simulation analysis method, the imaging quality of the imaging robot on the target deployment route under each candidate deployment scheme can be quantitatively evaluated.

[0134] Specifically, based on the relative positions of the candidate deployment points and the target deployment route, and the coverage area of ​​the candidate deployment angle on the target deployment route, the imaging quality of the imaging robot to be deployed on the target deployment route under the candidate deployment arrangement is determined, including:

[0135] Step S5021: Determine the distance between the imaging robot to be deployed and the target deployment line based on the relative positional relationship between the candidate deployment point and the target deployment line.

[0136] By measuring the coordinates of the candidate delivery point in three-dimensional space and the coordinates of the target feature (such as the lowest point of the sag line) on the target delivery line, and using the spatial distance calculation formula, the distance between the imaging robot and the target delivery line can be accurately determined.

[0137] Step S5022: Calculate the theoretical coverage of the target deployment route by the imaging robot to be deployed, based on the candidate deployment angle and distance.

[0138] Based on the imaging robot's lens parameters (such as field of view), known candidate deployment angles, and the distance between the imaging robot and the target deployment path, the theoretical coverage area of ​​the imaging robot over the target deployment path can be accurately calculated using geometric relationships and trigonometric functions. Specifically, the theoretical coverage area is calculated as follows:

[0139]

[0140] In the formula, For effective imaging length, Here, is the distortion correction factor (taken as 0.9-1.0), and d is the distance. As a backup deployment angle, This is the robot's field of view (typically 30-60°).

[0141] The theoretical coverage is [- , ].

[0142] Step S5023: Combining the theoretical coverage area and the imaging parameters of the imaging robot to be deployed, determine the actual imaging range of the imaging robot to be deployed on the target deployment route under the alternative deployment scheme.

[0143] Among the imaging parameters, imaging resolution is crucial, as it affects the percentage of effective pixels captured by the imaging robot on the target delivery route. Combining this with the previously calculated theoretical coverage area, and through correction and adjustment of the imaging parameters, the actual imaging range of the imaging robot on the target delivery route under this alternative deployment scheme can be determined. This actual imaging range more closely approximates reality, and the actual imaging range of the target delivery route can be obtained as follows:

[0144]

[0145] In the formula, The effective imaging length within the actual imaging range. For valid pixel values, This refers to the imaging resolution. A higher effective pixel count results in a longer effective imaging length within the actual imaging range, allowing for a more detailed representation of the target delivery route. This method enables accurate evaluation of the imaging robot's actual imaging capability of the target delivery route under each alternative deployment scheme.

[0146] The actual imaging range is [- , ].

[0147] The process of obtaining the effective pixel value involves using the imaging robot's sensor to count the number of pixels within the effective pixel area; this number is the effective pixel value. The specific value of the imaging resolution needs to be determined based on the performance of the lens and sensor equipped on the imaging robot; for example, it can be set to different levels such as 1080P (1920x1080 pixels) or 4K (3840x2160 pixels).

[0148] Step S5024: Based on the difference between the actual imaging range and the preset expected imaging range, determine the imaging quality of the imaging robot to be deployed on the target deployment route under the alternative deployment scheme; wherein, the smaller the difference, the better the imaging quality.

[0149] The desired imaging range is an ideal imaging area pre-defined based on actual application requirements and imaging mission objectives. By comparing the actual imaging range with the desired imaging range and calculating the difference between the two (such as area difference), the imaging quality can be quantitatively evaluated. The smaller the difference, the closer the actual imaging is to the desired effect, and the better the imaging quality; conversely, the imaging quality needs improvement. This evaluation method provides an objective and accurate basis for selecting the optimal candidate deployment scheme.

[0150] Step S503: Select the candidate deployment scheme with the best imaging quality from multiple candidate deployment schemes, and use it as the deployment scheme for the imaging robot to be deployed.

[0151] Among them, the candidate deployment scheme with the smallest difference between the actual imaging range and the preset expected imaging range is the deployment scheme of the imaging robot to be deployed.

[0152] Based on the same inventive concept, this application also provides a real-time imaging robot deployment system for implementing the real-time imaging robot deployment method described above.

[0153] The solution provided by this system is similar to the solution described in the above method. Therefore, the specific limitations of one or more real-time imaging robot deployment system embodiments provided below can be found in the limitations of the real-time imaging robot deployment method described above, and will not be repeated here.

[0154] like Figure 3 As shown, this application embodiment provides a real-time imaging robot deployment system, including:

[0155] The data acquisition module 100 is used to acquire the time-series image sequence of the target delivery line's sag line, environmental wind data, and line parameters;

[0156] The sway amplitude prediction module 200 is used to determine the current sway amplitude of the sloping line based on the sloping line image sequence, environmental wind data and line parameters, and to predict the future sway amplitude of the sloping line based on the current sway amplitude.

[0157] The swing trend analysis module 300 is used to perform swing trend analysis on the sag line based on the sag line image sequence to obtain the swing trend data of the sag line.

[0158] The position distribution prediction module 400 is used to predict the future position distribution of the vertical line based on the swing trend data and the future swing amplitude.

[0159] The deployment and screening module 500 is used to generate multiple candidate deployment and screening schemes for the imaging robot to be deployed based on the future position distribution of the vertical line, and to select the candidate deployment and screening scheme that provides the best imaging of the target deployment line from the multiple candidate deployment and screening schemes as the deployment and screening scheme for the imaging robot to be deployed.

[0160] The deployment execution module 600 is used to deploy the imaging robot to be deployed according to the deployment plan.

[0161] like Figure 4As shown, this application provides an electronic device. The electronic device 10 includes a memory 20 and a processor 30. The memory 20 stores a computer program. When the computer program is executed by the processor 30, the processor 30 performs the steps of the real-time imaging robot deployment method as described in the above embodiment.

[0162] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed, implements the steps of the real-time imaging robot deployment method as described in the above embodiments.

[0163] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, electronic devices, and computer storage media described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0164] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0165] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0166] In the several embodiments provided by this invention, it should be understood that the disclosed systems, electronic devices, computer storage media, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0167] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0168] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0169] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for executing all or part of the steps of the methods described in the various embodiments of the present invention through a computer device (which may be a personal computer, a server, or a network device, etc.). The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0170] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method of dispensing a real-time imaging robot, characterized by, The method comprises the following steps: collecting a sagging line image sequence of a target laying line in time sequence, environmental wind data and line parameters; determining a current swing amplitude of the sagging line according to the sagging line image sequence, the environmental wind data and the line parameters, and predicting a future swing amplitude of the sagging line based on the current swing amplitude; analyzing a swing trend of the sagging line according to the sagging line image sequence to obtain swing trend data of the sagging line; predicting a future position distribution of the sagging line according to the swing trend data and the future swing amplitude; generating a plurality of candidate laying arrangement schemes of an imaging robot to be laid according to the future position distribution of the sagging line, and screening a candidate laying arrangement scheme best imaging the target laying line from the plurality of candidate laying arrangement schemes as a laying arrangement scheme of the imaging robot to be laid; laying the imaging robot to be laid according to the laying arrangement scheme.

2. The drop-off method of a real-time imaging robot according to claim 1, characterized by, The method further comprises the following steps: image preprocessing of the sagging line image sequence; the image preprocessing of the sagging line image sequence comprises the following steps: color channel compensation is performed on the sagging line image sequence to obtain a sagging line image sequence after color channel compensation; color balance is performed on the sagging line image sequence after color channel compensation to obtain a sagging line image sequence after color balance; filtering processing is performed on the sagging line image sequence after color balance to obtain a sagging line image sequence after filtering processing as a sagging line image sequence after image preprocessing.

3. The drop-off method of a real-time imaging robot according to claim 1, characterized in that, The environmental wind data comprises horizontal wind and vertical wind; and the line parameters comprise span data; the determination of the current swing amplitude of the sagging line according to the sagging line image sequence, the environmental wind data and the line parameters, and the prediction of the future swing amplitude of the sagging line based on the current swing amplitude, comprises the following steps: determining a synthetic wind force according to a vector square sum between the horizontal wind and the vertical wind; determining a resistance and a lift of the target laying line according to the synthetic wind force, the span data, a preset fluid resistance and a preset fluid density; determining a first swing amplitude of the sagging line according to the resistance, the lift and a preset aerodynamic force; extracting a target feature in the sagging line image sequence and a coordinate position of the target feature, determining a distance between coordinate positions of the target feature in a plurality of adjacent frame images in the sagging line image sequence, and screening a distance between coordinate positions of adjacent frame images with the largest distance as a second swing amplitude; performing weighted fusion on the first swing amplitude and the second swing amplitude to obtain a current swing amplitude; predicting the future swing amplitude of the sagging line based on the current swing amplitude and combining a pre-trained swing amplitude prediction model, wherein the pre-trained swing amplitude prediction model is obtained by training a long short-term memory network based on historical sagging line swing amplitude data.

4. The drop-off method of a real-time imaging robot according to claim 1, characterized by, the swing trend analysis of the sagging line according to the sagging line image sequence to obtain the swing trend data of the sagging line, comprises the following steps: Difference operation is performed on each adjacent frame image in the sagittal line image sequence to obtain a difference image; Edge extraction is performed on each difference image according to an edge extraction operator to obtain a plurality of edge feature images; Connected domain analysis is performed on each edge feature image to obtain a swing edge contour of the sagittal line; Based on the swing edge contour, a swing center position of the sagittal line in each frame image is calculated; According to a position change of the swing center position in each adjacent frame image and a time step of adjacent frames, a position change direction, a position change speed and a position change acceleration of the swing center position are determined; The position change direction, the position change speed and the position change acceleration of the swing center position are taken as swing trend data of the sagittal line.

5. The drop-off method of a real-time imaging robot according to claim 1, characterized in that, The prediction of the future position distribution of the sagittal line according to the swing trend data and the future swing amplitude includes: The swing trend data and the future swing amplitude are input into a pre-trained position prediction model, so that the pre-trained position prediction model outputs a future position heat map of the sagittal line; wherein the pre-trained position prediction model is obtained by training a convolutional neural network based on historical sagittal line position data, swing trend data samples and future swing amplitude samples; According to the future position heat map of the sagittal line, the future position distribution of the sagittal line is determined. 6.The drop method of a real-time imaging robot according to claim 1, wherein, The generation of a plurality of candidate placement arrangement schemes of the imaging robot to be placed according to the future position distribution of the sagittal line, and the screening of a candidate placement arrangement scheme best for imaging the target placement line from the plurality of candidate placement arrangement schemes as the placement arrangement scheme of the imaging robot to be placed, includes: Based on the future position distribution of the sagittal line, a plurality of candidate placement arrangement schemes are determined; wherein each candidate placement arrangement scheme includes a candidate placement point and a candidate placement pose corresponding to the candidate placement point; the candidate placement pose includes a candidate placement angle; For each candidate placement arrangement scheme, according to the relative position relationship between the candidate placement point and the target placement line, and the coverage range of the target placement line according to the candidate placement angle, the imaging quality of the imaging robot to be placed on the target placement line under the candidate placement arrangement scheme is determined; From a plurality of candidate placement arrangement schemes, a candidate placement arrangement scheme with best imaging quality is screened as the placement arrangement scheme of the imaging robot to be placed.

7. The drop-off method of a real-time imaging robot according to claim 6, characterized in that, The determination of the imaging quality of the imaging robot to be placed on the target placement line according to the relative position relationship between the candidate placement point and the target placement line, and the coverage range of the target placement line according to the candidate placement angle, includes: According to the relative position relationship between the candidate placement point and the target placement line, the distance between the imaging robot to be placed and the target placement line is determined; According to the candidate placement angle and the distance, the theoretical coverage range of the imaging robot to be placed on the target placement line is calculated; Determine an actual imaging range of the imaging robot to be deployed on the target deployment route under the candidate deployment arrangement scheme according to the theoretical coverage range and imaging parameters of the imaging robot to be deployed. Determine an imaging quality of the imaging robot to be deployed on the target deployment route under the candidate deployment arrangement scheme according to a difference between the actual imaging range and a preset expected imaging range, wherein the smaller the difference is, the better the imaging quality is.

8. A drop-off system for a real-time imaging robot, characterized in that Comprise: A data acquisition module configured to acquire a sag curve image sequence of a target deployment route over time, environmental wind data, and line parameters; A swing amplitude prediction module configured to determine a current swing amplitude of a sag curve according to the sag curve image sequence, the environmental wind data, and the line parameters, and predict a future swing amplitude of the sag curve based on the current swing amplitude; A swing trend analysis module configured to perform swing trend analysis on the sag curve according to the sag curve image sequence to obtain swing trend data of the sag curve; A position distribution prediction module configured to predict a future position distribution of the sag curve according to the swing trend data and the future swing amplitude; A deployment arrangement screening module configured to generate a plurality of candidate deployment arrangement schemes of an imaging robot to be deployed according to the future position distribution of the sag curve, and screen a candidate deployment arrangement scheme that best images the target deployment route from the plurality of candidate deployment arrangement schemes as a deployment arrangement scheme of the imaging robot to be deployed; A deployment execution module configured to deploy the imaging robot to be deployed according to the deployment arrangement scheme.

9. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the real-time imaging robot deployment method according to any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed to implement the steps of the real-time imaging robot deployment method according to any one of claims 1-7.

Images