A centrifuge rotating speed control method and system based on image recognition

Abstract

The application discloses a centrifuge rotating speed control method and system based on image recognition. The method comprises the following steps: acquiring each frame of pretreated rotor region image; extracting feature points from each two continuous frames of the pretreated rotor region image, so as to obtain rotation angle and angular velocity estimation value and rotor motion trajectory; obtaining fused rotation angle and angular velocity estimation value according to the pretreated rotor region image; obtaining self-adaptive fused rotation angle and angular velocity estimation value according to the fused rotation angle and angular velocity estimation value; calculating real-time rotating speed estimation value of the centrifuge; generating corrected real-time rotating speed estimation value according to the real-time rotating speed estimation value; and obtaining a control signal according to the corrected real-time rotating speed estimation value. The technical scheme can significantly improve the system anti-interference, control precision and environmental adaptability, and meet the high-precision and high-stability control requirements of the high-speed centrifuge.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, in particular to a centrifuge rotating speed control method based on image recognition and a centrifuge rotating speed control system based on image recognition. BACKGROUND

[0002] In the field of centrifuge rotating speed control, the traditional technical solution is mainly based on the PID control algorithm of fixed parameters, and the rotating speed is adjusted through the preset proportional, integral and differential gain. The typical technical route is: 1) obtaining real-time rotating speed through an encoder or a laser speedometer; 2) calculating the error between the target rotating speed and the actual rotating speed; 3) generating a control signal to drive the motor according to the fixed PID parameters. However, this solution has three major defects: first, when the model parameters change due to sudden load changes or mechanical wear of the centrifuge, the fixed PID parameters are easy to cause rotating speed oscillation or response lag; second, the anti-interference ability is weak, and when the sensor data is affected by electromagnetic interference or mechanical vibration, the measurement noise will directly reduce the control precision; third, there is a contradiction between control precision and stability, and the traditional PID needs to balance between fast response and overshoot suppression, which is difficult to meet the requirements of high-precision rotating speed control (error <0.1%) and millisecond-level dynamic response at the same time.

[0003] Therefore, it is desirable to have a technical solution to solve or at least alleviate the above-mentioned deficiencies of the prior art. SUMMARY

[0004] The purpose of the present application is to provide a centrifuge rotating speed control method based on image recognition to at least solve one of the above technical problems.

[0005] The present application provides the following solutions:

[0006] According to one aspect of the present application, a centrifuge rotating speed control method based on image recognition is provided, which comprises:

[0007] acquiring a centrifuge rotor video stream collected by an industrial camera in real time;

[0008] preprocessing each frame of image in the centrifuge rotor video stream to obtain a rotor region image after preprocessing of each frame;

[0009] extracting feature points from each two consecutive frames of the rotor region image after preprocessing of each frame to obtain a rotating angle and angular velocity estimation value and a rotor motion trajectory;

[0010] performing micro-texture motion analysis on the rotor region image after preprocessing of each frame to obtain a fused rotating angle and angular velocity estimation value;

[0011] Adaptive multi-modal fusion is performed on the fused rotation angle and angular velocity estimation values, so as to obtain adaptive fused rotation angle and angular velocity estimation values;

[0012] A real-time rotation speed estimation value of the centrifuge is calculated according to the adaptive fused rotation angle and angular velocity estimation values;

[0013] A corrected real-time rotation speed estimation value is generated according to the real-time rotation speed estimation value and each frame image in the centrifuge rotor video stream;

[0014] The corrected real-time rotation speed estimation value is input into a PID controller, so as to obtain a control signal output by the PID controller.

[0015] Optionally, the preprocessing of each frame image in the centrifuge rotor video stream, so as to obtain each frame preprocessed rotor region image, comprises:

[0016] Gaussian filtering is applied to each frame image, so as to obtain a filtered image, so as to reduce sensor noise;

[0017] CLAHE is used on each frame filtered image to enhance image contrast under low light, so as to obtain a high-contrast image;

[0018] YOLOv5 is used to locate and crop the region where the rotor is located for each frame high-contrast image, so as to obtain each frame preprocessed rotor region image.

[0019] Optionally, the feature point extraction from each two consecutive frames of each frame preprocessed rotor region image, so as to obtain rotation angle and angular velocity estimation values and rotor motion trajectory, comprises:

[0020] SIFT algorithm is used to detect feature points from the rotor region image of the kth frame and the k+1th frame, so as to obtain a feature point set of the kth frame and a feature point set of the k+1th frame ;

[0021] According to the feature point set of the kth frame , a feature point descriptor set of the kth frame is obtained ;

[0022] According to the feature point set of the k+1th frame , a feature point descriptor set of the k+1th frame is obtained ;

[0023] According to the feature point descriptor set of the kth frame , the feature point descriptor set of the kth frame , and the feature point set of the kth frame and a feature point set of the k+1th frame Performing descriptor matching to obtain a matched feature point pair set and the matched feature point pair set , the matched feature point pair set and the matched feature point pair set Each element in the matched feature point pair set is (x, y, z), where , , and The descriptor distance between x and y is the smallest;

[0024] The matched feature point pair set and the matched feature point pair set respectively are subjected to adaptive RANSAC outlier rejection processing to obtain an inlier set after outlier rejection and Each element in the inlier set after outlier rejection is (x, y, z), and satisfies ; ;

[0025] According to the inlier set after outlier rejection , a feature point motion trajectory set of the kth frame is generated and according to the inlier set after outlier rejection , a feature point motion trajectory set of the k+1th frame is generated ;

[0026] According to the inlier set after outlier rejection , a rotation angle and an angular velocity estimate value are obtained;

[0027] The feature point motion trajectory set is subjected to Kalman filter trajectory prediction to obtain a rotor motion trajectory.

[0028] Optionally, the micro-texture motion analysis according to the pre-processed rotor region image of each frame to obtain the fused rotation angle and angular velocity estimate value comprises:

[0029] According to the inlier set after outlier rejection and the feature point set of the kth frame , a texture primitive set of the kth frame is extracted ;

[0030] According to the inlier set after outlier rejection and the feature point set of the k+1th frame , a texture primitive set of the k+1th frame is extracted​ ;

[0031] According to the texture primitive set of the kth frame and the texture primitive set of the k+1th frame Obtain the main direction information of the texture motion and the average texture motion speed ;

[0032] Obtain the fused rotation angle ;

[0033] According to the main direction information of the texture motion and the fused rotation angle Calculate the consistency index C.

[0034] Optionally, the calculation of the real-time rotation speed estimation value of the centrifuge according to the adaptive fused rotation angle and the angular velocity estimation value comprises:

[0035] According to the rotation angle , the angular velocity estimation value and the real-time working condition information, perform modal reliability evaluation, thereby obtaining the confidence of the texture motion modal ;

[0036] According to the confidence of the texture motion modal , the rotation angle , the angular velocity estimation value , the main direction information of the texture motion and the average texture motion speed Generate the weight of the macro motion modal and the weight of the micro texture motion modal ;

[0037] According to the rotation angle , the angular velocity estimation value , the main direction information of the texture motion and the average texture motion speed , the weight of the macro motion modal and the weight of the micro texture motion modal Generate the adaptive fused rotation angle and the adaptive fused angular velocity .

[0038] Optionally, the calculation of the real-time rotation speed estimation value of the centrifuge according to the adaptive fused rotation angle and the angular velocity estimation value comprises:

[0039] Initialize the dynamic system state of the centrifuge rotor, thereby obtaining the initial state vector and the initial state covariance matrix ;

[0040] obtain a prior estimate of the current state and a prior covariance of the current state ;

[0041] obtain an observation value at the current time and a Jacobian matrix of the observation model ;

[0042] obtain a posterior estimate of the current state according to the prior estimate of the current state and the prior covariance , the observation value at the current time and the Jacobian matrix of the observation model obtain a posterior covariance of the current state ;

[0043] obtain a real-time rotation speed estimation value according to the posterior estimate of the current state .

[0044] Optionally, the generating a corrected real-time rotation speed estimation value according to the real-time rotation speed estimation value and each frame image in the video stream of the centrifuge rotor comprises:

[0045] obtain a set of micro-topography descriptors of the kth frame according to the set of inliers after removing outliers and the set of feature points of the kth frame ;

[0046] obtain a set of micro-topography descriptors of the k+1th frame according to the set of inliers after removing outliers and the set of feature points of the k+1th frame ;

[0047] match the micro-topography descriptors in and , calculate the Euclidean distance between the descriptors, retain the descriptor pairs with a Euclidean distance less than a threshold , and form a set of matched micro-topography pairs ;

[0048] for each micro-topography pair ( , ) in the set of matched micro-topography pairs, calculate the topography change amount of the micro-topography pair, wherein is an L2 norm, thereby obtaining an actual topography change amount

[0049] use historical data or real-time data to construct a nonlinear regression model to fit​​​ , thereby obtaining the speed-topography correlation model ;

[0050] using the speed-topography correlation model , predicting the topography change amount at the adaptive fused angular velocity , thereby obtaining the predicted topography change amount ;

[0051] calculating the residual between the predicted topography change amount and the actual topography change amount ;

[0052] assuming that the residual between the predicted topography change amount and the actual topography change amount and the speed correction amount has a linear relationship: , wherein K is a correction gain, is noise, wherein the correction gain K is estimated by a least square method or a recursive least square method;

[0053] using the correction gain K and the residual , calculating the speed correction amount: ;

[0054] correcting the fused angular velocity: , thereby obtaining the corrected real-time speed estimation value .

[0055] Optionally, the inputting the corrected real-time speed estimation value into the PID controller, thereby obtaining the control signal output by the PID controller comprises:

[0056] obtaining a speed error according to the centrifuge preset target speed and the corrected real-time speed estimation value ;

[0057] judging whether the speed error exceeds an error threshold, and if yes, calculating the PID control output by the PID control.

[0058] The application further provides a centrifuge speed control system based on image recognition, comprising:

[0059] a centrifuge rotor video stream acquisition module, configured to acquire a centrifuge rotor video stream collected by an industrial camera in real time;

[0060] a preprocessing module, configured to pre-process each frame of image in the centrifuge rotor video stream, thereby obtaining each frame of pre-processed rotor region image;

[0061] a first estimation module, configured to perform feature point extraction on each two continuous frames of the pre-processed rotor region images of each frame, so as to obtain rotation angle and angular velocity estimation values and rotor motion trajectory;

[0062] a second estimation module, configured to perform micro-texture motion analysis on the pre-processed rotor region images of each frame, so as to obtain fused rotation angle and angular velocity estimation values;

[0063] a fusion module, configured to perform adaptive multi-modal fusion on the fused rotation angle and angular velocity estimation values, so as to obtain adaptive fused rotation angle and angular velocity estimation values;

[0064] a real-time rotation speed estimation value calculation module, configured to calculate a real-time rotation speed estimation value of the centrifuge according to the adaptive fused rotation angle and angular velocity estimation values;

[0065] a correction module, configured to generate a corrected real-time rotation speed estimation value according to the real-time rotation speed estimation value and each frame of image in the centrifuge rotor video stream;

[0066] an input module, configured to input the corrected real-time rotation speed estimation value to a PID controller, so as to obtain a control signal output by the PID controller.

[0067] The centrifuge rotation speed control method based on image recognition has the following advantages:

[0068] (1) The SIFT feature point detection, FLANN fast matching and adaptive RANSAC outlier elimination technology are used, so that the system can still stably track the feature points when the rotor surface is worn or the illumination changes. The RANSAC algorithm eliminates outliers, reduces false matching and improves tracking accuracy. Through FLANN accelerated matching and RANSAC optimization, the angular velocity estimation error is reduced, meeting the high-precision control requirement.

[0069] (2) Through micro-texture motion analysis, the Gabor filter can still extract texture features and provide independent motion estimation source when the rotor surface is smooth or the feature points are missing. After fusion with the feature point tracking result, the system fault tolerance is improved, and even if one method fails, the other method can still provide reliable motion information.

[0070] (3) By analyzing real-time working condition information (such as rotor acceleration, image quality index, etc.), the system can perceive the current running state and dynamically adjust the weight of feature point tracking and texture analysis. In the acceleration or deceleration stage, the system can automatically adjust the weight, so that the rotation speed estimation value can follow the target value more quickly.

[0071] (4) Extended Kalman Filter (EKF) fuses angle change estimates by linearizing the nonlinear system, constructs state equations and measurement equations, and realizes accurate estimation of the rotational speed. EKF has good filtering effect on sensor noise and model uncertainty, making the rotational speed estimate more smooth and accurate.

[0072] (5) By extracting the micro-topographic features (such as scratches, pits, etc.) on the surface of the rotor and matching them with the pre-collected templates, the topographic displacement is calculated to assist in correcting the rotational speed estimate. Micro-topographic features are not sensitive to rotor deformation, so even in the case of rotor thermal deformation or mechanical wear, accurate rotational speed estimates can still be provided. Through topographic motion analysis correction, the system can further eliminate estimation errors and improve the accuracy of rotational speed control.

[0073] (6) The image recognition-based centrifuge rotational speed control method of the present application improves image quality through Gaussian filtering and CLAHE enhancement, realizes robust feature tracking through SIFT+FLANN+RANSAC, captures micro-texture through Gabor filter, optimizes estimation accuracy through adaptive multi-modal fusion, suppresses noise through EKF filtering, and enhances anti-deformation ability through micro-topographic correction. The technical solution of the present application can significantly improve the system's anti-interference, control accuracy and environmental adaptability, meeting the high-precision and high-stability control requirements of high-speed centrifuges. BRIEF DESCRIPTION OF DRAWINGS

[0074] Figure 1 is a flowchart of the image recognition-based centrifuge rotational speed control method in an embodiment of the present application. DETAILED DESCRIPTION

[0075] The technical solutions of the present application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0076] As shown in the image recognition-based centrifuge rotational speed control method includes: Figure 1

[0077] Acquire the centrifuge rotor video stream collected in real time by the industrial camera (in this embodiment, the resolution is ≥720p and the frame rate is ≥30FPS);

[0078] Preprocess each frame of image in the centrifuge rotor video stream to obtain the preprocessed rotor region image of each frame;

[0079] ​extract feature points from each two continuous frames of the pre-processed rotor region images of each frame, so as to obtain the rotation angle and angular velocity estimation values and the rotor motion trajectory;

[0080] perform microscopic texture motion analysis according to the pre-processed rotor region images of each frame, so as to obtain fused rotation angle and angular velocity estimation values;

[0081] perform adaptive multi-modal fusion on the fused rotation angle and angular velocity estimation values, so as to obtain adaptive fused rotation angle and angular velocity estimation values;

[0082] calculate the real-time rotation speed estimation value of the centrifuge according to the adaptive fused rotation angle and angular velocity estimation values;

[0083] generate a corrected real-time rotation speed estimation value according to the real-time rotation speed estimation value and each frame image in the centrifuge rotor video stream;

[0084] input the corrected real-time rotation speed estimation value into a PID controller, so as to obtain a control signal output by the PID controller.

[0085] In the embodiment, the pre-processing of each frame image in the centrifuge rotor video stream to obtain the pre-processed rotor region images of each frame includes:

[0086] applying Gaussian filtering (kernel size 5x5) to each frame image to obtain a filtered image, so as to reduce sensor noise;

[0087] using CLAHE to enhance the contrast of the image under low light on each filtered image, so as to obtain a high-contrast image;

[0088] using YOLOv5 to locate and crop the region where the rotor is located on each high-contrast image, so as to obtain the pre-processed rotor region images of each frame.

[0089] In the embodiment, the feature point extraction from each two continuous frames of the pre-processed rotor region images of each frame to obtain the rotation angle and angular velocity estimation values and the rotor motion trajectory includes:

[0090] using SIFT algorithm to detect feature points from the rotor region images of the kth frame and the k+1th frame, so as to obtain a feature point set of the kth frame and a feature point set of the k+1th frame ; Specifically, the SIFT algorithm parameters are set as follows: the maximum number of feature points is 1000, the number of layers per octave is 3, the contrast threshold is 0.04, the edge suppression threshold is 10, and the Gaussian blur scale is 1.6. The feature point sets are detected on the kth frame and the k+1th frame, respectively, and are denoted as and ;

[0091] According to the feature point set of the kth frame Obtain the feature point descriptor set of the kth frame ; Specifically, a 16x16 neighborhood window is selected as the center of each feature point, divided into 4x4 sub-regions, and the gradient direction histogram of 8 directions is calculated in each sub-region to generate a 4x4x8=128-dimensional descriptor vector. The descriptor vector is normalized to eliminate the influence of light changes.

[0092] According to the feature point set of the kth frame Obtain the feature point descriptor set of the k+1th frame (Same as the above method, not described here);

[0093] According to the feature point descriptor set of the kth frame , the feature point descriptor set of the kth frame , the feature point set of the kth frame and the feature point set of the k+1th frame Match the descriptors to obtain the matched feature point pair set and the matched feature point pair set ;

[0094] Specifically, the descriptors in and are matched by FLANN, the L2 distance is calculated, the high-quality matching pairs with L2 distance less than 0.7 are retained, and the fuzzy matching is removed to obtain the matched feature point pair set and the matched feature point pair set Each element in the matched feature point pair set is ( ), wherein , , and The descriptor distance of is the smallest.

[0095] The matched feature point pair set and the matched feature point pair set are respectively subjected to adaptive RANSAC outlier removal processing to obtain the inlier set after outlier removal and , wherein each element in the inlier set after outlier removal is ( ), and satisfies ;

[0096] The matched feature point pair set and the matched feature point pair set respectively perform adaptive RANSAC outlier rejection processing, thereby obtaining an inlier set after outlier rejection and wherein each element in the inlier set after outlier rejection is (H, x, y, x', y') and satisfies and comprises:

[0097] performing the following processing on the matched feature point pair set :

[0098] initializing parameters, thereby obtaining an initialized parameter set, in this embodiment, the parameter set can comprise a maximum iteration number , an initial inlier threshold , an inlier proportion lower limit , and an adaptive adjustment factor α;

[0099] obtaining a sampling subset by random sampling ;

[0100] obtaining a homography matrix H by model estimation;

[0101] according to the homography matrix H, calculating a re-projection error of each matched point pair in the feature point pair set ;

[0102] obtaining a number of matched point pairs whose re-projection errors satisfy a projection error threshold and a set ;

[0103] calculating a current inlier proportion ;

[0104] if γ> and , then updating the optimal inlier set = and increasing the threshold to include more potential inliers, thereby generating an updated inlier threshold; if γ ,

[0105] Specifically, increasing the threshold can be realized by the following formula:

[0106] ; wherein is the current inlier threshold; α is the adaptive adjustment factor; is a preset maximum allowed threshold;

[0107] The reduced threshold value can be obtained by the following formula:

[0108] wherein, is the current inlier threshold value; and a is an adaptive adjustment factor; is a preset minimum allowable threshold value;

[0109] repeating the updating of the optimal inlier set until the iteration is terminated, thereby obtaining the inlier set after the outliers are removed .

[0110] The present application can balance the strictness and inclusiveness of inlier screening by dynamically adjusting the inlier threshold value When the inlier proportion is high, the threshold value is increased to avoid missing potential inliers; when the inlier proportion is low, the threshold value is reduced to strictly remove outliers

[0111] The adaptive RANSAC outlier removal method of the present application will be further described in detail below by way of example, and it can be understood that the example does not constitute any limitation on the present application.

[0112] Suppose that 100 pairs of feature point matches are obtained , of which 30% are outliers (such as false matches or abnormal points caused by motion blur).

[0113] The goal is to remove outliers by the adaptive RANSAC algorithm, retain the inlier set , and estimate the accurate homography matrix H.

[0114] Parameter initialization:

[0115] = 100 (maximum number of iterations);

[0116] = 3 pixels (initial inlier threshold value);

[0117] = 0.4 (lower limit of inlier proportion);

[0118] = 1.2 (adaptive adjustment factor);

[0119] 10 pixels (maximum allowable threshold value);

[0120] = 1 pixel (minimum threshold value).

[0121] Step 1: Initialize parameters

[0122] k=0, =0, =3 pixels.

[0123] Step 2: Random Sampling and Model Estimation

[0124] Randomly sample 4 pairs of matching points (a minimum of 4 pairs of points are required for homography matrix estimation):

[0125] .

[0126] Estimate the homography matrix H using the direct linear transformation (DLT):

[0127] H= ;

[0128] (The specific numerical value is obtained using the least squares method; the calculation process is omitted here, and the estimated result is assumed to be...) ).

[0129] Step 3: Interior point filtering and counting:

[0130] Matching 100 pairs of feature points Each pair of matching points Calculate the reprojection error ;

[0131] Assume that There are 65 pairs of interior points in a pixel, and the ratio of interior points γ = 65 / 100 = 0.65.

[0132] Record interior point set It contains 65 pairs of matching points.

[0133] Step 4: Adaptive threshold adjustment:

[0134] Because γ=0.65> =0.4 and 65> Update the optimal set of interior points And increase the threshold:

[0135] =min(3×1.2,10)=3.6 pixels.

[0136] Step 5: Iterative optimization and termination judgment:

[0137] Update k=1, because and If no update is performed for five consecutive times, continue iterating.

[0138] Iterative convergence result (assuming convergence after 20 iterations)

[0139] Final interior point threshold: = 5.2 pixels (stable after multiple adaptive adjustments).

[0140] Optimal inlier set: Contains 82 matched points (18 outliers are removed).

[0141] According to the inlier set after removing outliers Generate the feature point motion trajectory set of the kth frame and according to the inlier set after removing outliers Generate the feature point motion trajectory set of the (k+1)th frame ;

[0142] According to the inlier set after removing outliers Obtain the rotation angle and the angular velocity estimate ; Specifically, for the inlier pair in the inlier set , fit the rotation matrix by least squares method, satisfying ≈ ; Extract the rotation angle: based on the rotation matrix , convert it to a rotation vector by the Rodrigues formula, and the length of this vector is the rotation angle (unit: radian) from the kth frame to the (k+1)th frame; Calculate the angular velocity: given that the time interval between the adjacent two frames is ∆t (determined by the camera frame rate, such as ∆t = 1 / 30 s when the frame rate is 30 FPS), then the angular velocity estimate = (unit: rad / s).

[0143] Perform Kalman filter trajectory prediction on the feature point motion trajectory set , thereby obtaining the rotor motion trajectory.

[0144] In this embodiment, the micro-texture motion analysis based on the pre-processed rotor region images of each frame to obtain the fused rotation angle and angular velocity estimate includes:

[0145] According to the inlier set after removing outliers Generate the texture primitive set of the kth frame and according to the inlier set after removing outliers Generate the texture primitive set of the (k+1)th frame ;

[0146] In this embodiment, whether it is the feature point set of the kth frame and the feature point set of the (k+1)th frame Each element contains feature point coordinates (x, y), scale σ and dominant orientation θ.

[0147] The processing method is the same for the Kth frame and the K+1th frame, so the following takes the Kth frame as an example:

[0148] For each feature point (x, y, σ, θ) in the feature point set of the Kth frame, a local texture block with a size of 16σ x 16σ is extracted around it.

[0149] For each local texture block, a Gabor filter bank is applied to extract texture primitives to capture the multi-scale and multi-orientation characteristics of the texture.

[0150] The response of the Gabor filter is normalized to generate a texture primitive descriptor Each descriptor is a multi-dimensional vector reflecting the structure and orientation information of the local texture, thereby obtaining a feature point motion trajectory set .

[0151] According to the texture primitive set of the Kth frame and the texture primitive set of the K+1th frame , the main direction information of texture motion and the average texture motion speed are obtained; specifically, the texture primitives in and are matched to form a set of matched texture primitive pairs In this embodiment, the similarity is calculated using the following formula:

[0152] ;

[0153] wherein is the texture primitive of the Kth frame; is the texture primitive of the K+1th frame; is the mean value of the texture primitive of the Kth frame at the vth scale; is the mean value of the texture primitive of the K+1th frame at the vth scale; is the variance of the texture primitive of the Kth frame and the K+1th frame at the vth scale; is the variance of the texture primitive of the Kth frame at the vth scale; is the variance of the texture primitive of the K+1th frame at the vth scale; and are constant terms used to avoid numerical instability when the denominator is zero or close to zero; and ​​​These are weight parameters; This indicates the three different scales ( Perform product operations on similarity measurements on (=1,2,3).

[0154] The similarity formula described above is used for calculation. By weighting the mean, standard deviation and covariance of multiple scales, it captures both the structural similarity and directional consistency of textures, and has stronger robustness to noise and local deformation.

[0155] For the set of matched texture primitive pairs Each texture primitive pair in ( ), calculate its displacement vector ,in( )and( ) are respectively The center coordinates in frame k and frame k+1.

[0156] For all displacement vectors Perform cluster analysis (such as K-means clustering) to obtain the main texture motion directions. and speed .

[0157] Obtain the rotation angle after fusion ;

[0158] Based on the main direction information of texture movement With the fused rotation angle Calculate the consistency index C.

[0159] In this embodiment, the fused rotation angle It can be obtained in the following ways:

[0160] Based on the reliability of feature point matching, weights are dynamically assigned, and the confidence level of feature point matching is defined. (Interior point percentage, i.e.) ),in, Given the total number of matching points, the weight of the macroscopic rotation angle (feature point trajectory) is: W θ = Microtexture direction weight: W β = ;

[0161] Rotation angle after fusion Calculated by weighted average:

[0162] =W θ × +W β × ;

[0163] In this embodiment, the consistency index C can be obtained using the following formula:

[0164] C= ;

[0165] in, The rotation angle after fusion; This provides information on the main direction of texture movement. The joint entropy, representing the fused rotation angle and the main direction of texture motion, is used to measure the uncertainty or randomness of the amount of information between the two directions. The logarithm of the number N discretized bins is used to normalize the joint entropy. C represents the consistency index (ranging from [0,1]. The closer the value is to 1, the more consistent the directions are and the more definite the direction distribution; the closer the value is to 0, the more inconsistent the directions are or the more dispersed the direction distribution is).

[0166] In this embodiment, It can be obtained using the following formula:

[0167] Where p(i,j) is the joint statistical probability distribution; in this embodiment, and Mapping to discrete intervals:

[0168] right Calculate the index of its interval i= (i=0,1,...,N-1);

[0169] right Calculate the index of its interval i= (i=0,1,...,N-1);

[0170] Statistical joint probability distribution p(i,j):

[0171] Based on historical K-frame data (e.g., K=10), calculate the frequency of occurrence of (i,j): p(i,j)= Using the formula for the consistency index C from the above, we quantify the determinism of both directional consistency and directional distribution through a nonlinear combination of joint entropy and directional difference. When directions are consistent and concentrated, C approaches 1; when directions are inconsistent or dispersed, C approaches 0.

[0172] In this embodiment, if C > 0.75, it is considered that the macroscopic and microscopic motion directions are highly consistent, and the texture motion speed is... With angular velocity estimate By performing nonlinear fusion, the combined motion velocity is obtained:

[0173]

[0174] ; wherein r is the average distance of feature points to the center of image (used to convert angular velocity to linear velocity); is the average texture motion velocity; is the angular velocity estimation; is the main direction information of texture motion; is the fused rotation angle; is the average texture motion velocity.

[0175] It can be understood that if C is less than or equal to 0.75, it is considered that the macro and micro motion directions are inconsistent, and the fused rotation angle and the main direction information of texture motion are reserved as independent motion analysis results.

[0176] In the embodiment, the calculation of the real-time rotation speed estimation of the centrifuge according to the fused rotation angle and the angular velocity estimation includes:

[0177] According to the rotation angle , the angular velocity estimation , and the real-time working condition information, the modal reliability is evaluated, so as to obtain the confidence of the texture motion modal;

[0178] In the embodiment, the real-time working condition information includes rotor acceleration , image noise standard deviation .

[0179] In the embodiment, first, each data (rotation angle , angular velocity estimation , main direction information of texture motion , average texture motion velocity , rotor acceleration , and image noise standard deviation ) is normalized, which is the prior art and will not be described here.

[0180] In the embodiment, the confidence of the texture motion modal is obtained by the following formula:

[0181] ; wherein,

[0182] is the confidence of the texture motion modal; is the confidence of the texture motion velocity; is the confidence of the texture motion direction.

[0183] In the present embodiment, is obtained by the following formula:

[0184] wherein, is the normalized rotor acceleration;

[0185] In the present embodiment, is obtained by the following formula:

[0186] wherein, is the normalized image noise standard deviation;

[0187] According to the confidence of the texture motion modal , the rotation angle , the angular velocity estimate value , the texture motion main direction information and the average texture motion speed , the weight of the macro motion modal and the weight of the micro texture motion modal are generated.

[0188] In the present embodiment, the weight of the macro motion modal is obtained by the following formula:

[0189]

[0190] In the present embodiment, the weight of the micro texture motion modal is obtained by the following formula:

[0191]

[0192] The weight of the macro motion modal and the weight of the micro texture motion modal are normalized to ensure + =1.

[0193] According to the rotation angle , the angular velocity estimate value , the texture motion main direction information and the average texture motion speed , the weight of the macro motion modal and the weight of the micro texture motion modal , the adaptive fused rotation angle and the adaptive fused angular velocity are generated.

[0194] ​​In the embodiment, the fusion rotation angle is calculated by the following formula:

[0195] ; wherein, is the fusion rotation angle; is the weight of the macro motion mode; is the weight of the micro texture motion mode; is the normalized rotor acceleration;

[0196] In the embodiment, the fusion angular velocity is calculated by the following formula:

[0197] ; wherein, is the fusion angular velocity; is the weight of the macro motion mode; is the normalized angular velocity estimate; is the weight of the micro texture motion mode; is the normalized average texture motion velocity;

[0198] The obtained and are denormalized as follows:

[0199] ;

[0200] .

[0201] In the embodiment, the calculation of the real-time rotation speed estimate of the centrifuge according to the adaptive fusion rotation angle and angular velocity estimate includes:

[0202] The dynamic system state of the centrifuge rotor is initialized (using an extended Kalman filter) to obtain an initial state vector and an initial state covariance matrix ; specifically, the system state vector is defined, wherein ω is the rotation speed (angular velocity) and a is the rotor acceleration.

[0203] The state vector is initialized.

[0204] The state covariance matrix is initialized, wherein and are the initial standard deviations of the rotation speed and acceleration (known quantities), respectively.

[0205] The prior estimate of the current state and the prior covariance of the current state Specifically, the system dynamic model is defined as a uniformly accelerated motion model:

[0206] ;

[0207] For the current moment The state vector; For the previous moment The state vector; This is the state transition function; The process noise follows a normal distribution N(0,Q); The time interval, i.e. Q is the process noise covariance matrix. ,in and The process noise intensity refers to the rotational speed and acceleration, respectively.

[0208] Calculate the Jacobian matrix :

[0209] ;

[0210] Predict the current state:

[0211] ;

[0212] Predicted state covariance:

[0213] Q ;

[0214] Get the observation value at the current time. and the Jacobian matrix of the observation model Specifically, the observation model is defined as directly measuring rotational speed:

[0215] ;

[0216] Calculate the Jacobian matrix : For the current moment Observed values; For observation functions; To observe noise; To observe the noise variance;

[0217] ;

[0218] Based on prior estimates of the current state and prior covariance The current observation value And the Jacobian matrix of the observation model Obtain the posterior estimate of the current state and the posterior covariance of the current state ; in particular, the Kalman gain is computed as:

[0219] ; where is the Kalman gain; is the prior covariance matrix of the current state; is the transpose of the observation model Jacobian matrix;

[0220] update the state estimate:

[0221] ;

[0222] update the state covariance:

[0223] ;

[0224] from the posterior estimate of the current state obtain a real-time speed estimate. In this embodiment, the posterior estimate of the current state where is the filtered speed estimate, i.e., the real-time speed estimate.

[0225] In this embodiment, the generating a corrected real-time speed estimate from the real-time speed estimate and each frame image in the video stream of the centrifuge rotor includes:

[0226] from the inlier set after outlier rejection and the set of feature points of the k-th frame obtain a set of micro-topography descriptors of the k-th frame ; in particular, for each feature point (x, y, σ, θ) in the set of feature points of the k-th frame , extract a local topography block of size 32σ x 32σ centered at it from the inlier set after outlier rejection .

[0227] For each local topography block, the following processing is applied:

[0228] compute the Local Binary Pattern (LBP) feature to capture the roughness information of the surface texture.

[0229] compute the Local Phase Quantization (LPQ) feature to capture the blur invariance information of the surface texture.

[0230] compute the gradient magnitude and direction histogram (HOG) of the local topography to capture the edge information of the surface topography.

[0231] LBP, LPQ, and HOG features are concatenated to form a micromorphological descriptor. Each descriptor is a multi-dimensional vector that reflects the multi-scale characteristics of the local topography.

[0232] Based on the set of interior points after removing outliers and the feature point set of the (k+1)th frame Obtain the micromorphological descriptor subset of frame k+1. The specific method is the same as above, and will not be repeated here.

[0233] Subset of micromorphological descriptions and micromorphological description subset The micromorphological descriptors in the model are matched, and the Euclidean distance between the descriptors is calculated. Descriptors with an Euclidean distance less than a threshold are retained. The descriptor pairs form a set of matched micromorphological pairs. ;

[0234] right Each micromorphological pair in ( ), calculate its morphological changes. ,in It is an L2 norm;

[0235] Use historical data (such as offline calibration data) or real-time data to build a nonlinear regression model to fit the data. Thus, a rotational speed-morphology correlation model is obtained. ;

[0236] Using a rotation speed-morphology correlation model Predicting the angular velocity after adaptive fusion Morphological changes ;

[0237] Calculate the residual between the predicted morphological change and the actual morphological change: ;

[0238] Assuming residuals With speed correction amount There is a linear relationship between them: Where K is the correction gain. For noise, the correction gain K is estimated by least squares or recursive least squares;

[0239] Using the correction gain K and residual Calculate the speed correction amount: ;

[0240] Correction of fusion angular velocity: This allows us to obtain the corrected real-time speed estimate. .

[0241] In the embodiment, the inputting the corrected real-time rotating speed estimation value into the PID controller to obtain a control signal output by the PID controller comprises:

[0242] obtaining a rotating speed error according to the preset target rotating speed of the centrifuge and the corrected real-time rotating speed estimation value; obtaining a rotating speed error according to the preset target rotating speed of the centrifuge and the corrected real-time rotating speed estimation value;

[0243] judging whether the rotating speed error exceeds an error threshold, and if yes, calculating a PID control output through PID control.

[0244] The application also provides a centrifuge rotating speed control system based on image recognition, which comprises a centrifuge rotor video stream acquisition module, a preprocessing module, a first estimation module, a second estimation module, a fusion module, a real-time rotating speed estimation value calculation module, a correction module and an input module, wherein,

[0245] The centrifuge rotor video stream acquisition module is used to acquire a centrifuge rotor video stream collected by an industrial camera in real time.

[0246] The preprocessing module is used to pre-process each frame of image in the centrifuge rotor video stream to obtain a rotor region image after pre-processing of each frame.

[0247] The first estimation module is used to extract feature points from each two continuous frames of the rotor region image after pre-processing to obtain rotating angle and angular velocity estimation values and rotor motion trajectories.

[0248] The second estimation module is used to perform micro-texture motion analysis according to the rotor region image after pre-processing to obtain fused rotating angle and angular velocity estimation values.

[0249] The fusion module is used to perform adaptive multi-modal fusion on the fused rotating angle and angular velocity estimation values to obtain adaptive fused rotating angle and angular velocity estimation values.

[0250] The real-time rotating speed estimation value calculation module is used to calculate a real-time rotating speed estimation value of the centrifuge according to the adaptive fused rotating angle and angular velocity estimation values.

[0251] The correction module is used to generate a corrected real-time rotating speed estimation value according to the real-time rotating speed estimation value and each frame of image in the centrifuge rotor video stream.

[0252] The input module is used to input the corrected real-time rotating speed estimation value into a PID controller to obtain a control signal output by the PID controller.

[0253] ​It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A control method of a centrifuge rotational speed based on image recognition, characterized by, The image recognition-based centrifuge rotating speed control method comprises: acquiring a centrifuge rotor video stream collected by an industrial camera in real time; preprocessing each frame of image in the centrifuge rotor video stream to acquire a rotor region image after preprocessing of each frame; extracting feature points from each two continuous frames of the rotor region image after preprocessing to acquire a rotation angle and angular velocity estimation value and a rotor motion trajectory; performing micro-texture motion analysis on each frame of the rotor region image after preprocessing to acquire a fused rotation angle and angular velocity estimation value; performing adaptive multi-modal fusion on the fused rotation angle and angular velocity estimation value to acquire an adaptive fused rotation angle and angular velocity estimation value; calculating a real-time rotating speed estimation value of the centrifuge according to the adaptive fused rotation angle and angular velocity estimation value; generating a corrected real-time rotating speed estimation value according to the real-time rotating speed estimation value and each frame of image in the centrifuge rotor video stream; inputting the corrected real-time rotating speed estimation value into a PID controller to acquire a control signal output by the PID controller; the preprocessing of each frame of image in the centrifuge rotor video stream to acquire a rotor region image after preprocessing of each frame comprises: applying Gaussian filtering to each frame of image to acquire a filtered image to reduce sensor noise; applying CLAHE to each frame of filtered image to enhance image contrast under low light to acquire a high-contrast image; positioning and cropping the region where the rotor is located by using YOLOv5 on each frame of high-contrast image to acquire a rotor region image after preprocessing of each frame; the extraction of feature points from each two continuous frames of the rotor region image after preprocessing to acquire a rotation angle and angular velocity estimation value and a rotor motion trajectory comprises: The SIFT algorithm is used for detecting feature points of the rotor region images of the kth frame and the k+1th frame respectively, so as to obtain a feature point set of the kth frame and a feature point set of the k+1th frame . ​ a feature point set of the kth frame obtaining a feature point descriptor set of the kth frame ; a feature point set of the kth frame obtaining a feature point descriptor set of the k+1th frame ; Based on the feature point descriptor subset of the k-th frame The feature point descriptor subset of the k-th frame The set of feature points in the k-th frame and the feature point set of the (k+1)th frame Perform descriptor matching to obtain a set of matched feature point pairs. and the set of matched feature point pairs The set of matched feature point pairs and the set of matched feature point pairs Each element in is ( ),in , ,and and The descriptor distance is minimized; For the set of matched feature points respectively and the set of matched feature point pairs Adaptive RANSAC outlier removal is performed separately to obtain the set of interior points after outlier removal. as well as Among them, each element in the set of interior points after removing outliers is ( ), and satisfy ; According to the inlier set after removing outliers Generate the feature point motion trajectory set of the kth frame And according to the inlier set after removing outliers Generate the feature point motion trajectory set of the k+1th frame ; According to the inlier set after outlier rejection Obtaining a rotation angle And angular velocity estimates ; A set of feature point motion trajectories A Kalman filter trajectory prediction is performed to obtain a rotor motion trajectory.

2. The image recognition-based centrifuge rotational speed control method according to claim 1, characterized by, the micro-texture motion analysis on each frame of the rotor region image after preprocessing to acquire a fused rotation angle and angular velocity estimation value comprises: According to the inlier set after removing outliers And the feature point set of the kth frame Extract the texture primitive set of the kth frame ; According to the inlier set after removing outliers And the feature point set of the k+1th frame Extract the texture primitive set of the k+1th frame ; a set of texture primitives of the kth frame a set of texture primitives of the k+1th frame obtaining texture motion main direction information and average texture motion speed ; acquiring the fused rotation angle ; According to the texture motion main direction information With the fused rotation angle Calculate the consistency index C.

3. The image recognition-based centrifuge rotational speed control method according to claim 2, characterized by, the calculation of a real-time rotating speed estimation value of the centrifuge according to the adaptive fused rotation angle and angular velocity estimation value comprises: According to the rotation angle , the angular velocity estimation value , and real-time working condition information to perform modal reliability evaluation, thereby obtaining the confidence of the texture motion modal ; a confidence of the texture motion modality , a rotation angle , an angular velocity estimate , a main direction of texture motion information and an average texture motion speed a weight of the macro motion modality and a weight of the micro texture motion modality ; According to the rotation angle , the angular velocity estimate , the texture motion dominant direction information , and the average texture motion speed , the weight of the macro motion mode , and the weight of the micro texture motion mode , the adaptive fused rotation angle , and the adaptive fused angular velocity are generated.

4. The image recognition-based centrifuge rotational speed control method according to claim 3, characterized by, the calculation of a real-time rotating speed estimation value of the centrifuge according to the adaptive fused rotation angle and angular velocity estimation value comprises: initializing a dynamic system state of a centrifuge rotor, thereby obtaining an initial state vector and an initial state covariance matrix ; obtaining a prior estimate of the current state and a prior covariance of the current state ; Obtain the observation value at the current time and the Jacobian matrix of the observation model ; a priori estimate of the current state and a priori covariance an observation value at a current time and an observation model Jacobian matrix obtain a posteriori estimate of the current state and a posteriori covariance of the current state ; posterior estimate of the current state Obtain real-time speed estimate.

5. The image recognition-based centrifuge rotational speed control method according to claim 4, characterized by, the generation of a corrected real-time rotating speed estimation value according to the real-time rotating speed estimation value and each frame of image in the centrifuge rotor video stream comprises: According to the inlier set after removing outliers And the feature point set of the kth frame Obtain the micro-topography descriptor set of the kth frame ; According to the inlier set after removing outliers And the feature point set of the k+1th frame Obtain the micro-topography descriptor set of the k+1th frame ; a set of microtopography descriptors and microtopography descriptors in the set of microtopography descriptors are matched, euclidean distances between the descriptors are calculated, and pairs of descriptors with euclidean distances less than a threshold are retained to form a set of matched microtopography pairs ; right Each micromorphological pair in ( ), calculate its morphological changes. ,in The L2 norm is used to obtain the actual morphological changes; A nonlinear regression model is constructed using historical data or real-time data to fit a speed- topography correlation model ; Using a speed-topography correlation model , predicting a topography change amount at an angular speed after adaptive fusion , thereby obtaining a predicted topography change amount;​ calculating a residual of the predicted topography change amount and the actual topography change amount: ; a residual between a predicted profile change amount and an actual profile change amount and a rotational speed correction amount where K is a correction gain, is noise, wherein the correction gain K is estimated by a least square method or a recursive least square method;​ using a correction gain K and a residual error , a rotation speed correction amount is calculated: ; Corrected fusion angular velocity: Thus, the corrected real-time rotational speed estimation value is obtained .

6. The image recognition-based centrifuge rotational speed control method according to claim 5, wherein the input of the corrected real-time rotating speed estimation value into the PID controller to acquire a control signal output by the PID controller comprises: According to the preset target rotating speed of the centrifuge And the corrected real-time rotating speed estimation value Obtain a rotating speed error; judging whether a rotating speed error exceeds an error threshold, and if yes, calculating a PID control output by PID control.

7. An image recognition-based control system for centrifuge rotational speed, characterized by, The image recognition-based centrifuge rotating speed control system comprises: a centrifuge rotor video stream acquisition module, which is configured to acquire a centrifuge rotor video stream collected by an industrial camera in real time; The preprocessing module is configured to preprocess each frame of image in the centrifuge rotor video stream to obtain a preprocessed rotor region image of each frame; the preprocessing of each frame of image in the centrifuge rotor video stream to obtain a preprocessed rotor region image of each frame comprises: applying Gaussian filtering to each frame of image to obtain a filtered image to reduce sensor noise; applying CLAHE to each frame of filtered image to enhance image contrast under low light to obtain a high-contrast image; using YOLOv5 to locate and crop the region where the rotor is located in each frame of high-contrast image to obtain a preprocessed rotor region image of each frame; The first estimation module is configured to extract feature points from each two consecutive frames of the preprocessed rotor region image to obtain an estimated value of rotation angle and angular velocity and a rotor motion trajectory; the extraction of feature points from each two consecutive frames of the preprocessed rotor region image to obtain an estimated value of rotation angle and angular velocity and a rotor motion trajectory comprises: The SIFT algorithm is used to detect feature points in the rotor region images of the k-th frame and the (k+1)-th frame, respectively, to obtain the feature point set of the k-th frame. and the feature point set of the (k+1)th frame ; a set of feature points of the kth frame obtaining a set of feature point descriptors of the kth frame ; a feature point set of the kth frame obtaining a feature point descriptor set of the k+1th frame ; Based on the feature point descriptor subset of the k-th frame The feature point descriptor subset of the k-th frame The set of feature points in the k-th frame and the feature point set of the (k+1)th frame Perform descriptor matching to obtain a set of matched feature point pairs. and the set of matched feature point pairs The set of matched feature point pairs and the set of matched feature point pairs Each element in is ( ),in , ,and and The descriptor distance is minimized; For the set of matched feature points respectively and the set of matched feature point pairs Adaptive RANSAC outlier removal is performed separately to obtain the set of interior points after outlier removal. as well as Among them, each element in the set of interior points after removing outliers is ( ), and satisfy ; According to the inlier set after removing outliers Generate the feature point motion trajectory set of the kth frame And according to the inlier set after removing outliers Generate the feature point motion trajectory set of the k+1th frame ; According to the inlier set after outlier rejection Obtaining a rotation angle And an angular velocity estimate ; A set of feature point motion trajectories Perform Kalman filtering trajectory prediction, thereby obtaining a rotor motion trajectory; The second estimation module is configured to perform microscopic texture motion analysis on each frame of preprocessed rotor region image to obtain a fused estimated value of rotation angle and angular velocity; The fusion module is configured to perform adaptive multi-modal fusion on the fused estimated value of rotation angle and angular velocity to obtain an adaptively fused estimated value of rotation angle and angular velocity; The real-time speed estimation value calculation module is configured to calculate a real-time speed estimation value of the centrifuge according to the adaptively fused estimated value of rotation angle and angular velocity; The correction module is configured to generate a corrected real-time speed estimation value according to the real-time speed estimation value and each frame of image in the centrifuge rotor video stream; The input module is configured to input the corrected real-time speed estimation value to a PID controller to obtain a control signal output by the PID controller.

Images