Method and device for adjusting poses of multiple cameras of inspection robot of two-for-one twister

Abstract

The invention discloses a two-for-one twister inspection robot multi-camera pose adjustment method and device, and the method comprises the steps: extracting the center point coordinates of images at the broken line detection positions of four cameras of a two-for-one twister through YOLO, comparing the center point coordinates with set camera view center coordinates, and carrying out the multi-camera pose adjustment of the two-for-one twister inspection robot through multiple target detection collection results. Kalman filtering is used for predicting the position of a target, data association is carried out on the result, a cost matrix is constructed, then a Hungary algorithm is used for solving the cost matrix, and the optimal matching between each predicted broken line detection position and the detection target is found. And for each successfully matched prediction target and detection target, updating the position state of the target by using a Kalman filter, outputting a prediction result, judging whether the pitching pose of the camera is appropriate or not according to the prediction result, and adjusting the position of an automatic motor through a PID algorithm to adjust the pitching angle of the camera so as to ensure that the broken line detection position is located at the center of the view of the camera.

Description

Technical Field

[0001] The present application relates to the technical field of two-for-one twister inspection robots, and in particular to a method and device for adjusting the multi-camera posture of a two-for-one twister inspection robot. Background Art

[0002] With the vigorous development of automation technology, inspection robots for two-for-one twisters have been introduced into the production process. This type of robot shuttles between the two-for-one twisters along the preset trajectory. With the help of a multi-camera system, it captures real-time images of key parts of the two-for-one twisters from different angles, providing massive visual data for rapid troubleshooting of hidden faults and accurate assessment of machine status. However, in actual operation, the robot is affected by factors such as the flatness of the workshop floor and navigation accuracy, and the left and right paths often deviate during patrol. Once the deviation occurs, the distances between the inspection paths on both sides are no longer equal, and the images collected by the camera deviate from the ideal center position, resulting in incomplete and distorted imaging of key components, and the subsequent fault diagnosis accuracy based on image recognition algorithms has dropped significantly.

[0003] At present, the industry mainly uses the following methods to adjust the camera position on the twister inspection robot:

[0004] 1) Manual adjustment: Staff members regularly patrol the workshop to inspect the robot, visually check the machine camera and image position, and manually adjust the camera posture.

[0005] 2) Electric sensing adjustment: Use sensors and other detection devices to detect whether the camera position is appropriate, and use electric actuators to drive the camera's posture adjustment.

[0006] 3) Laser measurement and adjustment: Calculate the optimal position of the camera based on the reference characteristic value or distance of the laser rangefinder, and then adjust the camera posture with the help of the motor.

[0007] The defects of the above scheme are:

[0008] 1) Limitations of manual adjustment:

[0009] Low efficiency: Manual inspections are required regularly, manual operations are cumbersome, and adjusting multiple devices is time-consuming and labor-intensive.

[0010] Poor accuracy: Visual inspection is highly subjective, different people make different judgments, and slight deviations are difficult to accurately identify.

[0011] Weak real-time performance: Non-real-time monitoring, and changes in camera posture during inspection intervals are difficult to adjust in time.

[0012] High labor intensity: Workers walk around to check and make manual adjustments, which can easily lead to fatigue and affect work quality.

[0013] 2) Fixed sensor detection adjustment deficiency:

[0014] Limited coverage: Only preset fixed locations can be monitored, making it difficult to achieve full coverage.

[0015] Poor flexibility: It is impossible to dynamically adjust the monitoring focus according to the actual production situation.

[0016] High cost: Large-scale deployment of sensors requires high initial investment and maintenance costs.

[0017] 3) Laser measurement adjustment issues:

[0018] The measurement has limitations: it is greatly affected by environmental factors such as light and dust, and it is difficult to determine the position in all directions based on distance information alone.

[0019] High cost: The development cost of laser rangefinders and supporting equipment and algorithms is high.

[0020] Dependence on reference features: Changes or obscuration of reference features can lead to measurement and adjustment deviations.

[0021] 4) Generality issues:

[0022] Conflict between efficiency and precision: Existing technologies make it difficult to improve operational efficiency while ensuring precision.

[0023] High cost: Hardware purchase, software development and long-term maintenance all involve high costs.

[0024] Weak environmental adaptability: easily affected by external environmental factors (such as light, dust), affecting performance.

[0025] Lack of flexibility: It is difficult to adjust quickly according to production needs and lacks sufficient dynamic adaptability. Summary of the invention

[0026] The present application provides a method and device for adjusting the posture of multiple cameras of a two-for-one twister inspection robot, which has the advantage of being able to adjust the camera posture so that the captured image is always in an ideal position, ensuring the continuity and accuracy of the inspection work, and thus stabilizing the efficient operation of the textile production line.

[0027] The technical solution of this application is as follows:

[0028] On the one hand, the present application provides a method for adjusting the multi-camera posture of a twister inspection robot, comprising the following steps:

[0029] S1: Obtain the collected data of multiple cameras of the twister inspection robot;

[0030] S2: Perform target recognition and detection based on the collected data to obtain target location information;

[0031] S3: Use Kalman filtering to predict the target position, use data association to construct the cost matrix and match it with the Hungarian algorithm to obtain the prediction result of the target position;

[0032] S4: The camera is controlled according to the prediction result of the target position. The camera is controlled by a motor installed on the camera.

[0033] Furthermore, in step S2, the YOLO algorithm is used to detect the broken line target, and the bounding box coordinates are output. The center point coordinates of the target are calculated according to the bounding box coordinates as the position information of the broken line detection target.

[0034] Further, step S3 includes the following steps:

[0035] S31: Use Kalman filter to predict the target position;

[0036] Assume that the state vector of the disconnection detection target is x t , represents the position and velocity of the target at time t; for a target in two-dimensional space, the state vector is expressed as:

[0037]

[0038] Among them, x t and t Respectively represent the position of the target in the X and Y directions, and Represent the speed of the target in the X and Y directions respectively;

[0039] The target at the wire break detection point moves at a uniform speed relative to the inspection robot, and the dynamic model is expressed as:

[0040] x t =Fx t-1 +w t-1

[0041] Where F is the state transfer matrix, w t-1 is the process noise; the state transfer matrix F can be expressed as:

[0042]

[0043] Δt is the time interval;

[0044] According to the dynamic model, predict the state of the target in the current frame:

[0045]

[0046] in, is the predicted state at time t, is the estimated state at time t-1;

[0047] S314: Covariance matrix prediction;

[0048] The covariance matrix of the predicted state is:

[0049] P t|t-1 =FP t-1|t-1 F T +Q

[0050] Among them, P t|t-1 is the prediction covariance matrix at time t, P t-1|t-1 is the estimated covariance matrix at time t-1, Q is the covariance matrix of the process noise;

[0051] S32: Construct cost matrix using data association;

[0052] Assume that there are N predicted positions of disconnection detection targets and M actual positions of detection targets, and the predicted target positions are p1, p2, …, p N , the detection target position is d1,d2,…,d M ; Each predicted target position p i and each detected target position d j The Euclidean distance between is:

[0053]

[0054] Among them, the predicted target position: p1 = [x1, y1], p2 = [x2, y2], ..., p N =[x N ,y N ]; Detection target position: d1 = [u1, v1], d2 = [u2, v2], …, d M =[u M ,v M ];

[0055] S33: Hungarian algorithm matching;

[0056] Initialize the cost matrix Its size is N*N;

[0057] For each row i of the cost matrix, find the minimum value min(C i1 ,C i2 ,…,C iN ), and then reduce each element of each row: C ij =C ij -min(C i1 ,C i2 ,…,C iN ), for each column j, find the minimum value min(C 1j ,C 2j ,…,C Nj ), and then reduce each element of each column: C ij =Cij -min(C 1j ,C 2j ,…,C Nj ), find zero elements in the reduced cost matrix, mark these zero elements, cover all marked zero elements with the least number of lines, reduce the number of uncovered zero elements, check whether the number of covered lines is equal to the number of rows or columns of the matrix to determine whether the optimal match is found, output the matching result, adjust the cost matrix so that the number of uncovered zero elements increases, so as to find more matches; output the final matching result, indicating the corresponding relationship between the target prediction position and a detection target position at each broken line detection;

[0058] S34: prediction status update;

[0059] Determine the Kalman gain K t :

[0060] K t =P t|t-1 H T (HP t|t-1 H T +R) -1

[0061] Update state estimate The weighted sum of the residuals of the predicted state and the observed data is obtained:

[0062]

[0063] Update the covariance matrix P t|t :

[0064] P t|t =(IK t H)P t|t-1 .

[0065] Further, step S4 includes the following steps:

[0066] S41: Determine the camera position;

[0067] Four different state positions predicted from the broken wire detection point after the four cameras of the twister inspection robot are updated

[0068] Extract the current positions of the four targets [x t|t ,y t|t The camera center field of view is [x center ,y center ];

[0069] Pitch deviation: Δy = y t|t -y center; If |Δy| exceeds the set threshold, it is considered that the camera pose needs

[0070] Adjustment;

[0071] S42: Control the pitch angle of multiple cameras by adjusting the motor through the PID controller;

[0072] Calculate the control signal: Assuming the current time is t and the previous time is t-1, the control signal u(t) is expressed as:

[0073]

[0074] Where K p , K i , K d They are proportional, integral and derivative gains respectively.

[0075] On the other hand, the present application provides a multi-camera posture adjustment device for a twister inspection robot, comprising:

[0076] A data acquisition unit, used to acquire data collected by multiple cameras of the twister inspection robot;

[0077] A target detection unit is used to perform target recognition and detection based on the collected data to obtain target position information;

[0078] The current prediction unit is used to predict the target position using Kalman filtering, construct the cost matrix using data association and match it using the Hungarian algorithm to obtain the prediction result of the target position;

[0079] and a camera control unit for controlling the camera according to the prediction result of the target position.

[0080] On the other hand, the present application provides an electronic device, characterized in that it includes a processor and a memory, the memory stores a computer program, and when the computer program is called and executed by the processor, the method as described above is implemented.

[0081] On the other hand, the present application provides a computer-readable medium, characterized in that the computer-readable medium stores a computer program, and when the computer program is called and executed by a computer, the method as described above is implemented.

[0082] In summary, the beneficial effects of this application are:

[0083] 1. Real-time adaptive adjustment to improve inspection efficiency and accuracy:

[0084] The YOLO target detection algorithm is used to capture the key parts of the two-for-one twister's broken wire detection point in real time, combined with the Kalman filter to predict the target's motion trajectory, and the Hungarian algorithm is used to achieve the optimal matching of data association. The system dynamically adjusts the camera's position to ensure that the broken wire detection target is always located in the center of the camera's field of view, effectively eliminating image distortion and incompleteness caused by the robot's path deviation, and significantly improving the accuracy of fault diagnosis.

[0085] 2. Fully automated operation reduces manual dependence and labor intensity:

[0086] The PID adaptive control algorithm is used to drive the motor to adjust the camera pitch angle, completely replacing the traditional manual inspection and manual adjustment methods. The system can monitor and respond to environmental changes in real time, reduce the frequency of manual intervention, and avoid adjustment errors caused by human fatigue or subjective judgment differences.

[0087] 3. Strong environmental adaptability and robustness:

[0088] By integrating camera data with dynamic filtering algorithms, the system has strong adaptability to complex working conditions such as uneven workshop floors, light changes, and dust interference. Compared with existing laser measurement or fixed sensor solutions, this method does not need to rely on external reference features and can still maintain stable performance in a dynamic production environment.

[0089] 4. Significant cost-effectiveness:

[0090] Avoid large-scale deployment of high-precision sensors or laser ranging equipment, and achieve intensive use of hardware resources through software algorithm optimization. The initial investment and long-term maintenance costs of the system are lower than those of existing solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] Figure 1 This is a schematic diagram of the overall structure of a typical twister inspection robot;

[0092] Figure 2 This is a top view of a typical twister inspection robot;

[0093] Figure 3 This is a typical schematic diagram of camera posture adjustment in a twister inspection robot;

[0094] Figure 4 This is a schematic diagram of a target for detecting disconnection during normal inspection in a specific embodiment of the present application;

[0095] Figure 5 It is a target prediction and tracking view for line break detection in a specific embodiment of the present application;

[0096] Figure 6 This is a schematic diagram of the target of line break detection returning to the center of the field of view in a specific embodiment of the present application. DETAILED DESCRIPTION

[0097] The specific implementation of the present application is described in detail below with reference to the accompanying drawings.

[0098] An embodiment of the present application provides a multi-camera posture adjustment method for a two-for-one twister inspection robot. The embodiment of the present application is described with a typical two-for-one twister inspection robot. The structure of the two-for-one twister inspection robot is described in Chinese patents with application numbers 2025204546634 and 2025103033560, such as Figure 1-3 As shown in the figure, the robot is equipped with four high-definition industrial cameras installed in multiple directions of the robot. The bottom is the inspection chassis, which is used to control the movement of the inspection machine. The robot is equipped with a brushless motor and a stepper motor to control the pitch angle of the four high-definition cameras and the up and down height of the four cameras.

[0099] It is understandable that the method provided in this application can also be applied to other types of two-for-one twister inspection robots.

[0100] The multi-camera posture adjustment method of the twister inspection robot comprises the following steps:

[0101] S1: Obtain the collected data from multiple cameras of the twister inspection robot.

[0102] S2: Perform target recognition and detection based on the collected data to obtain target location information; specifically, use the YOLO algorithm to detect broken line targets, output the bounding box coordinates, that is, the four corner coordinates of the frame, and calculate the center point coordinates of the target based on the bounding box coordinates as the location information of the broken line detection target.

[0103] S3: Use Kalman filtering to predict the target position, use data association to construct the cost matrix and match it with the Hungarian algorithm to obtain the prediction result of the target position.

[0104] S31: Use Kalman filter to predict target position;

[0105] Construct the state representation of the disconnection detection target;

[0106] Assume that the state vector of the disconnection detection target is x t , represents the position and velocity of the target at time t; for a target in two-dimensional space, the state vector is expressed as:

[0107]

[0108] Among them, x t and t Respectively represent the position of the target in the X and Y directions, and Represents the speed of the target in the X and Y directions respectively;

[0109] Construct a dynamic model of the disconnection detection target;

[0110] The target at the wire break detection point moves at a uniform speed relative to the inspection robot, and the dynamic model is expressed as:

[0111] x t =Fx t-1 +w t-1

[0112] Where F is the state transfer matrix, w t-1 is the process noise, set to zero-mean Gaussian noise; the state transfer matrix F can be expressed as:

[0113]

[0114] Δt is the time interval.

[0115] The prediction steps are:

[0116] (1) Status prediction:

[0117] According to the dynamic model, predict the state of the target in the current frame:

[0118]

[0119] in, is the predicted state at time t, is the estimated state at time t-1;

[0120] (2) Covariance matrix prediction;

[0121] The covariance matrix of the predicted state is:

[0122] P t|t-1 =FP t-1|t-1 F T +Q

[0123] Among them, P t|t-1 is the prediction covariance matrix at time t, P t-1|t-1 is the estimated covariance matrix at time t-1, Q is the covariance matrix of the process noise;

[0124] The specific calculation steps are:

[0125] Assume that at time t-1, the estimated state of the disconnection detection target is The covariance matrix is ​​P t-1|t-1 , now we need to predict the target state at time t.

[0126] (1) Status prediction:

[0127]

[0128] The calculation results are:

[0129]

[0130] (2) Covariance matrix prediction;

[0131] Assume that the covariance matrix Q of the process noise is:

[0132]

[0133] Among them, q1, q2, q3, q4 are the variances of process noise, which can be adjusted according to actual conditions. Calculate the predicted value of the covariance matrix:

[0134]

[0135] S32: Construct cost matrix using data association;

[0136] Assume that there are N predicted positions of disconnection detection targets and M actual positions of detection targets, and the predicted target positions are p1, p2, …, p N , the detection target position is d1,d2,…,d M ; Each predicted target position p i and each detected target position d j The Euclidean distance between is:

[0137]

[0138] Among them, the predicted target position: p1 = [x1, y1], p2 = [x2, y2], ..., p N =[x N ,y N ]; Detection target position: d1 = [u1, v1], d2 = [u2, v2], …, d M =[u M ,v M ];

[0139] S33: Hungarian algorithm matching;

[0140] Initialize the cost matrix Its size is N*N;

[0141] For each row i of the cost matrix, find the minimum value min(C i1 ,C i2 ,…,C iN ), and then reduce each element of each row: C ij =C ij -min(C i1 ,C i2 ,…,CiN ), for each column j, find the minimum value min(C 1j ,C 2j ,…,C Nj ), and then reduce each element of each column: C ij =C ij -min(C 1j ,C 2j ,…,C Nj ), find zero elements in the reduced cost matrix, mark these zero elements, cover all marked zero elements with the least number of lines, reduce the number of uncovered zero elements, check whether the number of covered lines is equal to the number of rows or columns of the matrix to determine whether the optimal match is found and output the matching result; adjust the cost matrix so that the number of uncovered zero elements increases, thereby finding more matches; output the final matching result, indicating the corresponding relationship between the target prediction position at each broken line detection and a detected target position;

[0142] In the Hungarian algorithm:

[0143] 1. The zero element coverage rule is:

[0144] Mark all rows where zero elements are not covered;

[0145] Cover the columns of all zero elements in the marked rows;

[0146] Mark all rows with matching zero elements in the covered columns;

[0147] Repeat the above process until no more marking is possible;

[0148] 2. The cost matrix adjustment method is:

[0149] Find the smallest element δ of the uncovered area;

[0150] Subtract δ from all uncovered elements;

[0151] Add δ to all elements covered by two lines;

[0152] Other elements remain the same;

[0153] 3. The matching quality judgment criteria are:

[0154] When the number of coverage lines is equal to the matrix dimension (i.e., N lines), there are N independent zero elements, which constitutes the optimal match. Otherwise, the cost matrix needs to be adjusted further.

[0155] The output of the Hungarian algorithm is the correspondence between the predicted position of each object and the detected object position.

[0156] This correspondence can be used to update the state of the target. For example, suppose target A is at the predicted position The detection target position matched by the Hungarian algorithm is (u j , v j ), then the update formula of Kalman filter can be used to update the state of target A.

[0157] S34: prediction status update;

[0158] Determine the Kalman gain K t :

[0159] K t =P t|t-1 H T (HP t|t-1 H T +R) -1

[0160] Volume calculation:

[0161]

[0162] Weighing the credibility of the predicted state and the observed data, the greater the gain, the greater the influence of the observed data on the state update.

[0163] Update state estimate The weighted sum of the residuals of the predicted state and the observed data is obtained:

[0164]

[0165] Specific calculation:

[0166]

[0167] The weighted sum of the residuals of the predicted state and the observed data reflects the fusion result of the predicted state and the observed data, which improves the accuracy of the target state estimation.

[0168] Update the covariance matrix P t|t :

[0169] P t|t =(IK t H)P t|t-1

[0170] Specific calculation:

[0171]

[0172] where z t For observation data H is the observation matrix R is the covariance matrix of the observation noise I is the identity matrix. The smaller the covariance, the more reliable the updated state estimate.

[0173] S4: The camera is controlled according to the prediction result of the target position. The camera is controlled by a motor installed on the camera.

[0174] S41: Determine the camera position;

[0175] Four different state positions predicted from the broken wire detection point after the four cameras of the twister inspection robot are updated Extract the current positions of the four targets [x t|t ,y t|t The camera center field of view is [x center ,y center ]; pitch deviation: Δy = y t|t -y center ; If |Δy| exceeds the set threshold, it is considered that the camera pose needs to be adjusted;

[0176] S42: The motor is adjusted by a PID controller to control the pitch angle of the multi-camera; in the PID controller: the proportional term P is proportional to the deviation. The integral term I is proportional to the integral of the deviation and is used to eliminate the steady-state error. The differential term D is proportional to the rate of change of the deviation and is used to reduce overshoot.

[0177] Calculate the control signal: Assuming the current time is t and the previous time is t-1, the control signal u(t) is expressed as:

[0178]

[0179] Where K p , K i , K d They are proportional, integral and derivative gains respectively.

[0180] According to the Δy of the four cameras, if Δy>0, it means the target is above the center of the field of view and the camera needs to be adjusted downward; conversely, if Δy<0, the camera needs to be adjusted upward. Then, a control command is sent to adjust the motor position, and the motor adjusts the pitch angle of the camera according to the control command to bring the target back to the center of the field of view.

[0181] The prediction process and adjustment are shown below:

[0182] (1) Initial state: The line of sight and imaging area of ​​the wire break detection point during normal inspection by the robot are as follows: Figure 4 shown.

[0183] (2) Offset and prediction process: The line break detection point gradually shifts upward. The red rectangle is the actual position, the yellow rectangle is the predicted position, and the green rectangle is the center point trajectory prediction. Figure 5 shown.

[0184] (3) Camera posture adaptation process: According to the detection and prediction results, the brushless motor in the robot system is adjusted to perform PID closed-loop control, raising the pitch angle so that the detection point returns to the center of the field of view, such as Figure 6 shown.

[0185] Another embodiment of the present application provides a multi-camera posture adjustment device for a twister inspection robot, comprising:

[0186] A data acquisition unit, used to acquire data collected by multiple cameras of the twister inspection robot;

[0187] A target detection unit is used to perform target recognition and detection based on the collected data to obtain target position information;

[0188] The current prediction unit is used to predict the target position using Kalman filtering, construct the cost matrix using data association and match it using the Hungarian algorithm to obtain the prediction result of the target position;

[0189] and a camera control unit for controlling the camera according to the prediction result of the target position.

[0190] The multi-camera posture adjustment method of the two-for-one twister inspection robot is as described above.

[0191] Another embodiment of the present application provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is called and executed by the processor, the method described above is implemented.

[0192] Another embodiment of the present application provides a computer-readable medium, wherein the computer-readable medium stores a computer program, and when the computer program is called and executed by a computer, the method described above is implemented.

[0193] The above is only a preferred implementation of the present application. It should be pointed out that a person skilled in the art can make several modifications and improvements without departing from the inventive concept of the present application, and these all fall within the scope of protection of the present application.

Claims

1. A method for adjusting the posture of multiple cameras of a two-for-one twister inspection robot, characterized in that: The following steps are involved: S1: Obtain the collected data of multiple cameras of the twister inspection robot; S2: Perform target recognition and detection based on the collected data to obtain target location information; S3: Use Kalman filtering to predict the target position, use data association to construct the cost matrix and match it with the Hungarian algorithm to obtain the prediction result of the target position; S4: The camera is controlled according to the prediction result of the target position. The camera is controlled by a motor installed on the camera.

2. The multi-camera posture adjustment method of the two-for-one twister inspection robot according to claim 1 is characterized in that: In step S2, the YOLO algorithm is used to detect the broken line target, and the bounding box coordinates are output. The center point coordinates of the target are calculated according to the bounding box coordinates as the position information of the broken line detection target.

3. The multi-camera posture adjustment method of the two-for-one twister inspection robot according to claim 1 is characterized in that: Step S3 includes the following steps: S31: Use Kalman filter to predict target position; Assume that the state vector of the disconnection detection target is x t , represents the position and velocity of the target at time t; for a target in two-dimensional space, the state vector is expressed as: Among them, x t and t Respectively represent the position of the target in the X and Y directions, and Represent the speed of the target in the X and Y directions respectively; The target at the wire break detection point moves at a uniform speed relative to the inspection robot, and the dynamic model is expressed as: x t =Fx t-1 +w t-1 Where F is the state transfer matrix, w t-1 is the process noise; the state transfer matrix F can be expressed as: Δt is the time interval; According to the dynamic model, predict the state of the target in the current frame: in, is the predicted state at time t, is the estimated state at time t-1; S314: Covariance matrix prediction; The covariance matrix of the predicted state is: P t|t-1 =FP t-1|t-1 F T +Q Among them, P t|t-1 is the predicted covariance matrix at time t, P t-1|t-1 is the estimated covariance matrix at time t-1, Q is the covariance matrix of the process noise; S32: Construct cost matrix using data association; Assume that there are N predicted positions of disconnection detection targets and M actual positions of detection targets, and the predicted target positions are p1, p2, …, p N , the detection target position is d1,d2,…,d M ; Each predicted target position p i and each detected target position d j The Euclidean distance between is: Among them, the predicted target position: p1 = [x1, y1], p2 = [x2, y2], ..., p N =[x N ,y N ]; Detection target position: d1 = [u1, v1], d2 = [u2, v2], …, d M =[u M ,v M ]; S33: Hungarian algorithm matching; Initialize the cost matrix Its size is N*N; For each row i of the cost matrix, find the minimum value min(C i1 ,C i2 ,…,C iN ), and then reduce each element of each row: C ij =C ij -min(C i1 ,C i2 ,…,C iN ), for each column j, find the minimum value min(C 1j ,C 2j ,…,C Nj ), and then reduce each element of each column: C ij =C ij -min(C 1j ,C 2j ,…,C Nj ), find zero elements in the reduced cost matrix, mark these zero elements, cover all marked zero elements with the least number of lines, reduce the number of uncovered zero elements, check whether the number of covered lines is equal to the number of rows or columns of the matrix to determine whether the optimal match is found, output the matching result, adjust the cost matrix so that the number of uncovered zero elements increases, so as to find more matches; output the final matching result, indicating the corresponding relationship between the target prediction position and a detection target position at each broken line detection; S34: prediction status update; Determine the Kalman gain K t : K t =P t|t-1 H T (HP t|t-1 H T +R) -1 Update state estimate The weighted sum of the residuals of the predicted state and the observed data is obtained: Update the covariance matrix P t|t : P t|t =(I-K t H)P t|t-1 。 4. The multi-camera posture adjustment method of the two-for-one twister inspection robot according to claim 1 is characterized in that: Step S4 includes the following steps: S41: Determine the camera position; Four different state positions predicted from the broken wire detection point after the four cameras of the twister inspection robot are updated Extract the current positions of the four targets [x t|t ,y t|t The camera center field of view is [x ceNter ,y center ]; Pitch deviation: Δy = y t|t -y center ; If |Δy| exceeds the set threshold, it is considered that the camera pose needs to be adjusted; S42: Control the pitch angle of multiple cameras by adjusting the motor through the PID controller; Calculate the control signal: Assuming the current time is t and the previous time is t-1, the control signal u(t) is expressed as: Where K p , K i , K d They are proportional, integral and derivative gains respectively.

5. A multi-camera posture adjustment device for a two-for-one twister inspection robot, characterized in that: include: A data acquisition unit, used to acquire data collected by multiple cameras of the twister inspection robot; A target detection unit is used to perform target recognition and detection based on the collected data to obtain target position information; The current prediction unit is used to predict the target position using Kalman filtering, construct the cost matrix using data association and match it using the Hungarian algorithm to obtain the prediction result of the target position; and a camera control unit for controlling the camera according to the prediction result of the target position.

6. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program, and when the computer program is called and executed by the processor, the method according to any one of claims 1 to 4 is implemented.

7. A computer-readable medium, characterized in that The computer-readable medium stores a computer program, and when the computer program is called and executed by a computer, the method according to any one of claims 1 to 4 is implemented.

Images