A method and related device for motion particle recognition based on multi-view features

Abstract

The application discloses a kind of based on multi-view feature's moving abrasive particle identification method and related device, method includes obtaining moving abrasive particle video, and obtains abrasive particle multi-view image in moving abrasive particle video;Area change curve, length-width ratio change curve and roundness change curve are determined based on multi-view image;Based on the feature change curve of each acquisition identifies the abrasive particle type of moving abrasive particle.This application obtains the video of moving abrasive particle, then determines multi-angle image according to moving abrasive particle video, can obtain the multi-view information of abrasive particle by multi-angle image, can quickly determine the abrasive particle quantity and abrasive particle type contained in wear tissue fluid, so as to can improve abrasive particle analysis efficiency.Meanwhile, the feature change curve of area, length-width ratio and circularity of each view image is obtained in the application, and the abrasive particle type is identified by fusing each feature change curve, so that the accuracy and identification speed of abrasive particle type identified can be guaranteed.

Description

Technical Field

[0001] The present application relates to the technical field of artificial joint wear mechanism analysis and mechanical fault diagnosis, and in particular to a method for identifying motion wear particles based on multi-view features and related devices. Background Art

[0002] Wear particle (abrasive grain) types (e.g., flakes, strips, blocks, and spheres) are generated by various operating conditions, including joint load, lubrication, material, and operating time. Therefore, abrasive grain type is an important indicator for assessing the severity and mechanism of wear between workpieces. It is widely used in bearings, engines, and artificial joints. Therefore, abrasive grain type identification has become a focus of attention.

[0003] At present, machine learning algorithms and deep learning algorithms are commonly used to identify abrasive particle types. For machine learning algorithms, such as support vector machines (SVMs), neural networks, and search trees, it is usually necessary to manually extract features (for example, the size, contour, surface texture, and color of the abrasive particles) as input. However, since the features are manually selected, there is no guarantee that the input features are reliable for abrasive particle classification, resulting in low accuracy in the determined abrasive particle types. For deep learning algorithms, such as convolutional neural networks, multiple similar single-view images of the same abrasive particle type need to be given during training so that the deep learning algorithm can automatically extract abrasive particle features and identify the type. However, different types of abrasive particles have highly similar viewing angles (for example, thin flake particles look like strip particles when viewed from a vertical plane), and as the viewing angle changes during the tumbling of the abrasive particles, the abrasive particle type determined by the deep learning algorithm is highly unstable.

[0004] Therefore existing technology still needs to be improved and improved. Summary of the Invention

[0005] The technical problem to be solved by this application is to provide a method and related device for identifying moving wear particles based on multi-view features in response to the shortcomings of the existing technology.

[0006] In order to solve the above technical problems, the first aspect of the embodiments of the present application provides a method for identifying moving wear particles based on multi-view features, the method comprising:

[0007] Acquire a video of moving abrasive particles, and acquire multi-view images of the abrasive particles in the video of moving abrasive particles;

[0008] determining a plurality of characteristic change curves based on the multi-view images, wherein the plurality of characteristic change curves include an area change curve, an aspect ratio change curve, and a roundness change curve;

[0009] The abrasive particle type of the moving abrasive particles is identified based on the plurality of characteristic change curves.

[0010] The motion abrasive particle recognition method based on multi-view features, wherein the obtaining of the multi-view images of the abrasive particles in the motion abrasive particle video specifically comprises:

[0011] Obtaining a background model of the motion abrasive particle video;

[0012] Subtracting the motion abrasive particle video from the background model to obtain a motion target video;

[0013] Tracking the motion target video to obtain the multi-view images of the abrasive particles in the motion abrasive particle video.

[0014] The motion abrasive particle recognition method based on multi-view features, wherein the determination of the several feature variation curves based on the multi-view images specifically comprises:

[0015] Extracting the abrasive particle regions of each view image in the multi-view images, and determining the abrasive particle area, length-width ratio and roundness of each view image based on the abrasive particle regions;

[0016] Determining the area variation curve, length-width ratio variation curve and roundness variation curve of the abrasive particles according to the abrasive particle area, length-width ratio and roundness of each view image.

[0017] The motion abrasive particle recognition method based on multi-view features, wherein the recognition of the abrasive particle type of the motion abrasive particle based on the several feature variation curves specifically comprises:

[0018] Obtaining the variation period of each feature variation curve in the several feature variation curves, and selecting a single-period feature variation curve in each feature variation curve based on the variation period;

[0019] For each single-period feature region, extracting a feature vector of the single-period feature variation curve;

[0020] Recognizing the abrasive particle type of the motion abrasive particle based on the feature vectors.

[0021] The motion abrasive particle recognition method based on multi-view features, wherein the variation period is determined based on an autocorrelation function.

[0022] The motion abrasive particle recognition method based on multi-view features, wherein the extraction of the feature vector of the single-period feature variation curve specifically comprises:

[0023] Extracting several statistical features of the single-period feature variation curve based on a statistical method to obtain the feature vector.

[0024] The motion abrasive particle recognition method based on multi-view features, wherein the recognition of the abrasive particle type of the motion abrasive particle based on the feature vectors specifically comprises:

[0025] For each feature vector, obtaining an SVM classification model corresponding to the feature vector, and determining the probability of the wear particle type corresponding to the feature vector using the SVM classification model;

[0026] Based on the determined probabilities of the wear particle types, the wear particle types of the moving wear particles are identified.

[0027] A second aspect of the embodiments of the present application provides a moving wear particle identification system based on multi-view features, the system comprising:

[0028] An acquisition module is used to acquire a video of moving wear particles and acquire multi-view images of wear particles in the video of moving wear particles;

[0029] a determination module, configured to determine several characteristic change curves of the moving abrasive particles based on the multi-view images of the abrasive particles, wherein the several characteristic change curves include an area change curve, an aspect ratio change curve, and a roundness change curve;

[0030] The identification module is used to identify the abrasive particle type of the moving abrasive particles based on a plurality of characteristic change curves.

[0031] A third aspect of an embodiment of the present application provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in any of the above-described methods for identifying moving wear particles based on multi-perspective features.

[0032] A fourth aspect of an embodiment of the present application provides a terminal device, comprising: a processor, a memory, and a communication bus; the memory stores a computer-readable program executable by the processor;

[0033] The communication bus realizes the connection and communication between the processor and the memory;

[0034] When the processor executes the computer-readable program, the processor implements the steps of any of the above-mentioned methods for identifying moving wear particles based on multi-view features.

[0035] Beneficial effects: Compared with the prior art, the present application provides a method and related device for identifying moving wear particles based on multi-perspective features, the method comprising obtaining a video of moving wear particles, and obtaining multi-perspective images of the wear particles in the video of the moving wear particles; determining an area change curve, an aspect ratio change curve, and a roundness change curve based on the multi-perspective images; and identifying the wear particle type of the moving wear particles based on the obtained characteristic change curves. The present application obtains a video of the moving wear particles, and then determines a multi-angle image based on the video of the moving wear particles. The multi-perspective information of the wear particles can be obtained through the multi-angle images, and the number and type of wear particles contained in the wear tissue fluid can be quickly determined, thereby improving the efficiency of wear particle analysis. At the same time, the present application obtains the characteristic change curves of the area, aspect ratio, and roundness of the images of each perspective, and fuses the characteristic change curves to identify the wear particle type, thereby ensuring the accuracy and recognition speed of the identified wear particle type. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without inventive work.

[0037] Figure 1 This is a flowchart of the method for identifying moving wear particles based on multi-view features provided in this application.

[0038] Figure 2 A schematic diagram of the principle flow of the moving wear particle identification method based on multi-view features provided in this application.

[0039] Figure 3 This is a schematic diagram of the principle flow of determining the type of wear particles from multi-view images in the moving wear particle identification method based on multi-view features provided in this application.

[0040] Figure 4 Schematic diagram of a device for collecting videos of moving wear particles.

[0041] Figure 5 Multi-view images of wear particles.

[0042] Figure 6 An image frame in the moving wear particle video.

[0043] Figure 7 for Figure 6 The image frame shown is the target image frame after the background is removed.

[0044] Figure 8 Schematic diagram of wear tracking.

[0045] Figure 9 Multi-view images and characteristic change curves of flake-type abrasive particles.

[0046] Figure 10 Multi-view images and characteristic change curves of blocky abrasive particles.

[0047] Figure 11 Multi-view images and characteristic change curves of spherical abrasive particles.

[0048] Figure 12 Multi-view images and characteristic change curves of strip-shaped abrasive particles.

[0049] Figure 13 It is a schematic diagram of the area characteristic change curves of flake-type abrasive particles, block-type abrasive particles, spherical-type abrasive particles and strip-type abrasive particles.

[0050] Figure 14 for Figure 13 Schematic diagram after normalization.

[0051] Figure 15 A structural principle diagram of a moving wear particle identification system based on multi-view features is provided for this application.

[0052] Figure 16 This is a schematic diagram of the structure of the terminal device provided in this application. DETAILED DESCRIPTION

[0053] This application provides a method and related device for identifying moving wear particles based on multi-view features. To make the purpose, technical solution, and effects of this application more clear and explicit, the application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only intended to illustrate this application and are not intended to limit this application.

[0054] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0055] As will be understood by one of ordinary skill in the art, and unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0056] It should be understood that the sequence and size of each step in the embodiments do not mean the order of execution, and the execution order of each process is determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the application.

[0057] It is found through research that different types of wear particles (i.e., abrasion particles) (for example, flaky, strip-shaped, block-shaped, and spherical, etc.) are generated by different working conditions such as connecting load, lubrication, material, and running time, so that the type of wear particle is an important indicator for evaluating the severity of wear between workpieces and the wear mechanism, and is widely used in the fields of bearings, engines, and artificial joints, etc. Therefore, the identification of the type of wear particle has become the focus of attention.

[0058] At present, the identification of the type of wear particle is generally carried out by using machine learning algorithms and deep learning algorithms. For machine learning algorithms such as SVM, neural networks, and search trees, etc., it is usually necessary to manually extract features (such as the size, contour, surface texture, and color of the wear particle, etc.) as input. However, since the features are artificially selected, it cannot be guaranteed that the input features are reliable for wear particle classification, so that the accuracy of the determined type of wear particle is low. For deep learning algorithms such as convolutional neural networks, etc., during training, multiple similar single perspective images of the same type of wear particle need to be given, so that the deep learning algorithm automatically extracts the features of the wear particle and identifies the type. However, different types of wear particles have highly similar perspectives (for example, from the vertical plane, flaky particles look like strip-shaped particles, etc.), and with the change of the perspective of the wear particle during the rolling process, the type of wear particle determined by the deep learning algorithm has great instability.

[0059] To solve the above problems, in the embodiment of the application, a moving abrasive particle video is acquired, and a multi-view image of an abrasive particle in the moving abrasive particle video is acquired; an area change curve, an aspect ratio change curve and a roundness change curve are determined based on the multi-view image; and a type of the moving abrasive particle is identified based on the acquired characteristic change curves. The application can acquire a multi-angle image based on the moving abrasive particle video, and can acquire multi-view information of the abrasive particle based on the multi-angle image, so as to quickly determine the number and type of the abrasive particles contained in the wear tissue fluid, thereby improving the efficiency of abrasive particle analysis. Meanwhile, the application can acquire the area, aspect ratio and roundness of each view image, and fuse the characteristic change curves to identify the type of the abrasive particle, so as to ensure the accuracy and speed of the identified type of the abrasive particle.

[0060] The application content will be further described by the description of the embodiments in combination with the drawings.

[0061] The embodiment provides a moving abrasive particle identification method based on multi-view features, as shown in Figure 1 、 Figure 2 and Figure 3 , the method comprises the following steps.

[0062] S10, a moving abrasive particle video is acquired, and a multi-view image of an abrasive particle in the moving abrasive particle video is acquired.

[0063] Specifically, the moving abrasive particle video is obtained by collecting a mixed liquid containing abrasive particles, for example, the moving abrasive particle video is obtained by shooting a flow process of a mixed liquid of a liquid medium and artificial joint abrasive particles. The moving abrasive particle video includes multiple video frames, and at least some of the video frames carry abrasive particles. The video frame carrying the abrasive particles can carry one abrasive particle or multiple abrasive particles. When the video frame carries multiple abrasive particles, the types of the multiple abrasive particles can be the same or different. Therefore, the moving abrasive particle video can carry abrasive particles of one type or abrasive particles of multiple different types.

[0064] In one implementation, the moving abrasive particle video can adopt a method as shown in Figure 4The wear particle image acquisition device shown is captured, wherein the wear particle image acquisition device includes a host, an oil tank, a micropump, a microfluidic optical microscope, and a camera. The oil tank, micropump, and microfluidic channel are sequentially connected through an oil pipeline to form a circulating oil circuit. The micropump drives the mixed liquid in the oil tank to flow along the oil circuit, so that the mixed liquid flows through the microfluidic channel. The optical microscope is arranged opposite to the microfluidic channel and magnifies and images the mixed liquid flowing through the microfluidic channel. The camera is arranged opposite to the optical microscope and captures the image of the optical microscope. The host is connected to the camera to obtain image frames of the camera capture channel to obtain a video of moving wear particles. The magnification of the optical microscope, the flow rate of the mixed liquid in the oil circuit, and the shooting frame rate of the camera can all be determined according to actual needs. For example, the flow rate of the mixed liquid in the oil circuit can be a stable flow rate of 1-300 ml / h, the magnification of the optical microscope can be 50×-500×, the shooting frame rate of the camera can be 30fps, and the resolution of the camera can be 1920×1080. Of course, in practical applications, other acquisition devices can also be used to acquire moving wear particle videos.

[0065] Furthermore, the multi-perspective image includes multiple single-perspective images, wherein, when the moving wear particle video carries one wear particle, the multi-perspective image is a group, and when the moving wear particle video carries multiple wear particles, the multi-perspective image is multiple groups, and each group of multi-perspective images corresponds to one wear particle. In other words, obtaining the multi-perspective image of the wear particle in the moving wear particle video refers to obtaining the multi-perspective image of each wear particle in the moving wear particle video. For example, the moving wear particle video includes four types of wear particles, which are respectively recorded as #266, #337, #341 and #346. Then, if Figure 5 As shown in the figure, four groups of corresponding multi-view images can be extracted from the moving wear particle video. Figure 5 It can be seen that the multi-view images show the contours and morphologies of the wear particles at different viewpoints, which can help to identify the wear particle type, especially for irregular wear particles.

[0066] In one implementation, obtaining multi-view images of wear particles in the moving wear particle video specifically includes:

[0067] S11, obtaining a background model of the moving wear particle video;

[0068] S12, performing a differential analysis on the moving wear particle video and the background model to obtain a moving target video;

[0069] S13: performing target tracking on the moving target video to obtain multi-view images of the wear particles in the moving wear particle video.

[0070] Specifically, an image is composed of a foreground and a background. In a video of moving wear particles, the moving wear particles are the foreground, and the parts other than the moving wear particles are the background. Once the background is determined, the image and the background are differentiated to obtain a foreground image. In one implementation, the background model can be determined using a Gaussian mixture background modeling method, wherein the process of determining the background model using the Gaussian mixture background modeling method can be as follows: first, each pixel value is regarded as randomly changing and conforming to a Gaussian distribution, then, multiple Gaussian models with different weights are established for each pixel, and then, after continuous iteration and updating of the model parameters of the Gaussian model, the background model of the moving wear particle video is obtained.

[0071] Furthermore, after obtaining the background model, the moving wear particle video is differentiated from the background model to remove the background information of each video frame in the moving wear particle video to obtain a moving target video, wherein the moving target video includes a plurality of target video frames, each of the plurality of target video frames corresponds to a video frame in the moving wear particle video, and the target video frame is obtained by differentiating the video frame from the background image corresponding to the video frame in the background model. For example, Figure 6 The target video frame shown is the one in the moving wear particle video. Figure 7 The video frame shown is obtained by subtracting the corresponding background image in the background model.

[0072] Furthermore, during the rolling process of the abrasive particles, due to the high similarity and different sizes of the abrasive particles, adhesion and occlusion are likely to occur, which can easily lead to ID exchange and track loss during tracking. Figure 8 As shown in the figure, the deep_sort method is used to track the moving wear particles in the moving target video. The deep_sort method uses a cascade matching algorithm composed of Kalman filtering and Hungarian algorithm, which can effectively alleviate the large changes caused by occlusion, reduce the number of ID exchanges, and improve the accuracy of the multi-view images of the wear particles.

[0073] S20. Determine several feature change curves based on the multi-view images, wherein the several feature change curves include an area change curve, an aspect ratio change curve, and a roundness change curve.

[0074] Specifically, since flake, strip, block and spherical particles are common types of abrasive particles for artificial joints, such as Figure 5As shown, the abrasive particle profiles in each viewing angle graph of blocky abrasive particles are mostly irregular; the abrasive particle profiles in each viewing angle graph of strip-shaped abrasive particles are mostly slender and have a large aspect ratio; the abrasive particle profiles in each viewing angle graph of spherical abrasive particles are mostly circular or quasi-circular and have a large roundness; and the abrasive particle profiles in each viewing angle graph of flake-shaped abrasive particles are relatively complex. Thus, this embodiment obtains characteristic change curves for reflecting the abrasive particle profiles, wherein the characteristic change curves include an area change curve, an aspect ratio change curve, and a roundness change curve. The area change curve is used to reflect the area change of the abrasive particle at different viewing angles during movement, the aspect ratio change curve is used to reflect the aspect ratio change of the abrasive particle at different viewing angles during movement, and the roundness change curve is used to reflect the roundness change of the abrasive particle at different viewing angles during movement. By using the area change curve, the aspect ratio change curve, and the roundness change curve, this embodiment can fully utilize the characteristic information of the abrasive particle profiles of different abrasive particle types to distinguish the abrasive particles.

[0075] For example:

[0076] Assuming that the moving abrasive particle video includes flake-type abrasive particles, block-type abrasive particles, spherical-type abrasive particles, and strip-type abrasive particles, the multi-view images of the four types of abrasive particles and their corresponding characteristic change curves are as follows: Figure 9-12 As shown by Figure 9-12 It can be seen that for the area characteristics, the flake abrasive particles vary greatly between different viewing angles, with deeper troughs in the curve, while the area changes of the spherical abrasive particles between different viewing angles are relatively stable and tend to stabilize near a certain value; for the aspect ratio change curve, the flake type changes most dramatically, while the spherical abrasive particles are stable near 1, and the strip abrasive particles have slight fluctuations near a certain value; for the roundness characteristics, the spherical and strip abrasive particles change relatively steadily, while the flake and block abrasive particles change dramatically. Therefore, based on the area change curve, the aspect ratio change curve, and the roundness change curve, the abrasive particles can be distinguished to identify the abrasive particle type.

[0077] In one implementation, determining a plurality of feature change curves based on the multi-view images specifically includes:

[0078] S21, extracting the wear particle region of each viewing angle image in the multi-view images, and determining the wear particle area, aspect ratio, and roundness corresponding to each viewing angle image based on each wear particle region;

[0079] S22 , determining an area variation curve, an aspect ratio variation curve, and a roundness variation curve of the abrasive particles according to the abrasive particle areas, aspect ratios, and roundness corresponding to the images at each viewing angle.

[0080] Specifically, the area change curve is determined based on the abrasive grain area of ​​each single-view image, wherein the calculation formula for the abrasive grain area of ​​the single-view image can be:

[0081]

[0082] Where D represents the abrasive area, and (x, y) represents the pixel point in area D.

[0083] The aspect ratio variation curve is determined based on the aspect ratio of the abrasive particles in each single-view image, wherein the calculation formula of the aspect ratio of the abrasive particles can be:

[0084]

[0085] Wherein, W is the length of the smallest rectangle surrounding the abrasive grain area, and L is the width of the smallest rectangle.

[0086] The roundness variation curve is determined based on the roundness of the abrasive particles in each single-view image, wherein the calculation formula of the abrasive particle roundness can be:

[0087]

[0088] Wherein, A represents the area of ​​the abrasive region, and P represents the perimeter of the abrasive region.

[0089] Furthermore, the characteristic changes in aspect ratio and circularity of the graphics at each viewing angle are not related to the size of the abrasive grain itself, but are related to the type of abrasive grain. However, the area value is not only related to the shape, but also to the size of the abrasive grain. For example, Figure 13 As shown in the figure, the area change curves of spherical abrasive particles of different sizes show that the variance of the area curves of the four abrasive particles is small and the change trend is relatively stable. However, as the abrasive particles become larger, the corresponding area distribution range also becomes larger. Therefore, in order to eliminate the influence of abrasive particle size on abrasive particle type identification, this embodiment scales the area values ​​of all abrasive particles to the interval [0,1] and keeps the shape of the area change curve unchanged, so as to obtain the following: Figure 14 The area change curve shown in FIG. 1 , wherein the scaling formula for scaling the area values ​​of all abrasive particles to the interval [0,1] can be:

[0090]

[0091] Among them, X max represents the maximum wear particle area corresponding to a wear particle in the multi-view image, and X represents the wear particle area of ​​the wear particle in the single-view image.

[0092] S30: Identify the abrasive particle type of the moving abrasive particles based on the plurality of characteristic change curves.

[0093] Specifically, the abrasive particle type is the abrasive particle shape type of the moving abrasive particles, wherein the abrasive particle type can be a flake type, a strip type, a block type or a spherical type, etc. In addition, when the abrasive particle dynamic video includes multiple moving abrasive particles, the abrasive particle type of each moving abrasive particle will be determined, and the multiple moving abrasive particles may have the same abrasive particle type. It can be understood that this embodiment will identify all the moving abrasive particles included in the moving abrasive particle video and the abrasive particle type of each moving abrasive particle, thereby obtaining the number of abrasive particles and the abrasive particle distribution in the tissue fluid containing the abrasive particles corresponding to the moving abrasive particle video, so as to facilitate the evaluation of the degree of wear of the device corresponding to the tissue fluid. For example, the tissue fluid video formed by the abrasive particles corresponding to the artificial joint is collected by the method provided in this embodiment, and the collected moving abrasive particle video is processed to obtain the number of abrasive particles in the tissue fluid and the abrasive particle type of each abrasive particle, so as to evaluate the degree of wear of the artificial joint. In addition, it is worth noting that when the moving abrasive particle video includes multiple moving abrasive particles, each moving abrasive particle corresponds to several characteristic change curves, that is, the area change curve, aspect ratio change curve and roundness change curve of each moving abrasive particle are obtained.

[0094] In one implementation, identifying the abrasive particle type of the moving abrasive particle based on the plurality of characteristic change curves specifically includes:

[0095] Obtaining a change period of each characteristic change curve from among the plurality of characteristic change curves, and selecting a single-period characteristic change curve from among the characteristic change curves based on the change period;

[0096] For each single-period feature region, extracting a feature vector of the single-period feature change curve;

[0097] Based on each feature vector, the wear particle type of the moving wear particle is identified.

[0098] Specifically, according to fluid mechanics, abrasive particles perform periodic motion in the flow channel, and thus, the characteristic change curve of the abrasive particles will also show periodic changes. However, because the size and shape of the abrasive particles themselves affect the force and movement of the abrasive particles, the number of multi-view images obtained in the video of the moving abrasive particles with the same movement distance (field of view of the microscope) is different, resulting in different lengths of the characteristic change curves determined based on the multi-view images, and thus the use of the characteristic change curves may result in different accuracy in identifying the abrasive type of different abrasive particles. In this embodiment, to avoid the influence of the above situation, after obtaining the characteristic change curves of the moving abrasive particles, the change period of each characteristic change curve is determined, and then a single-cycle characteristic change curve is selected based on the change period, and the single-cycle characteristic change curve is used to identify the abrasive type of the moving abrasive particles.

[0099] In one implementation, the variation period is determined based on Fourier transform, and the variation period may be determined by:

[0100] H11. Convert the characteristic change curve into the frequency domain through Fourier transform to obtain a spectrum diagram of the characteristic change curve. The expression of Fourier transform is:

[0101]

[0102] Where x(n) is a discrete-time signal of finite length x(n) = [x(0), x(2), …, x(N-1)];

[0103] H12. Select the frequency corresponding to the waveform with the largest change amplitude in the spectrum diagram, and use the displayed frequency as the change period of the characteristic change curve.

[0104] Specifically, the characteristic change curve is a time-domain signal. Using a discrete Fourier transform, the characteristic change curve is decomposed into a sine wave in the frequency domain. Low-frequency components have larger amplitudes and contribute more to the original waveform, while high-frequency components have less influence on the original waveform. Therefore, in the spectrum, the frequency corresponding to the waveform with the largest amplitude change is often selected as the approximate variation period of the characteristic change curve.

[0105] In one implementation, the variation period of the characteristic variation curve is determined based on the autocorrelation function, and the variation period may be determined by:

[0106] H21. Determine the autocorrelation function curve of the characteristic change curve through the autocorrelation function, wherein the expression of the autocorrelation function is:

[0107]

[0108] Where x(t) = [x(0), x(2), …, x(N-1)] represents the characteristic change curve, and τ represents the displacement between the two signals;

[0109] H22. Obtain the interval between three adjacent peaks or three adjacent troughs in the autocorrelation function curve, and use the interval as the variation period.

[0110] Specifically, the autocorrelation function is used to analyze a time domain signal to describe the similarity or correlation between the time domain signal and itself at different moments.

[0111] In one implementation, extracting the feature vector of the single-period feature change curve specifically includes:

[0112] Extracting several statistical features of the single-cycle characteristic change curve based on a statistical method to obtain a feature vector;

[0113] Specifically, the same feature variation curves of different types of abrasive grains have similar shapes and variation trends, especially among the feature variation curves of the sheet, block and strip types, especially the length-width ratio curves of the block and strip types. However, the feature variation amplitudes and distribution intervals of the same feature variation curves of different types of abrasive grains are different. Therefore, in order to classify the types of abrasive grains, the embodiment extracts statistical features of the feature variation curves, and takes the extracted statistical features as feature vectors of the feature variation curves, wherein the extracted statistical features are shown in Table 1.

[0114] Table 1 Periodic curve features and descriptions

[0115]

[0116] In an implementation manner, the identifying the abrasive grain type of the moving abrasive grain based on the feature vectors specifically includes:

[0117] For each feature vector, an SVM classification model corresponding to the feature vector is obtained, and an abrasive grain type probability corresponding to the feature vector is determined through the SVM classification model;

[0118] Based on the determined abrasive grain type probabilities, the abrasive grain type of the moving abrasive grain is identified.

[0119] Specifically, after obtaining the abrasive grain type probabilities corresponding to the feature vectors, the abrasive grain type probabilities can be weighted, and the abrasive grain type of the moving abrasive grain is determined according to the weighted results, or the abrasive grain type with the largest abrasive grain type probability is directly selected from the obtained abrasive grain type probabilities as the abrasive grain type of the moving abrasive grain, etc. The embodiment considers that the features of different types of abrasive grains have similarities, and sets an SVM classification model for each feature vector, which can avoid the problem that using a single type for classification will cause a large error.

[0120] In an implementation manner, the abrasive grain type of the moving abrasive grain is determined according to the weighted results by weighting the abrasive grain type probabilities, wherein the weighting process is to weight each abrasive grain type probability estimation by increasing each abrasive grain type probability to the power of α, which can reduce the difference between the accuracy estimates of different classifiers, and further improve the accuracy of the identified abrasive grain type.

[0121] Support Vector Machine (SVM) is an excellent binary classifier. It can map data to higher dimensions using kernel functions and find the maximum margin between samples. It also works well for small sample sizes and has been widely used in various fields. The SVM classification model primarily uses "one-vs-many" and "one-vs-one" multi-classification methods. The "one-vs-many" method selects one type at a time during training and groups the remaining types of samples into one category, resulting in the construction of N classifiers. The one-vs-one method selects one type and another different type at a time, resulting in a total of N(N-1) / 2 classifications.

[0122] In addition, the SVM model is a model that connects the standard SVM model and the sigmoid model, where the sigmoid model maps the SVM output to the posterior output probability, as shown in the formula:

[0123]

[0124] In order to determine the parameters A and B, a cross entropy error function is defined to minimize the error:

[0125]

[0126] Train the sigmod function and define a new training set (f i ,t i ), use LM algorithm to find the optimal value. Among them, f i is the output value of the svm score function, t i The target probability value of the sample.

[0127] In order to avoid the problem of overfitting, Bayesian estimation is used, t i Defined as

[0128]

[0129] Among them, N + is the number of positive samples in the training set, N - is the number of negative samples.

[0130] For N-classification problems, a one-to-one classification method is used to construct N(N-1) / 2 classifiers. The Plattscaling algorithm can be used to obtain the posterior probability of a sample between any two classes. Combining all classifiers can calculate the final posterior probability of the sample. Currently, commonly used methods include pairwise coupling and voting. The pairwise coupling method requires an optimization algorithm to find the optimal value, which is not conducive to real-time recognition. The voting method, on the other hand, is simple to calculate and runs quickly. Its calculation process is as follows:

[0131]

[0132] Among them, Piaj (t|j; x) represents a two-class support vector machine classifier consisting of the i-th class and the j-th class, and the calculated posterior probability that x belongs to the i-th class.

[0133] In summary, this embodiment provides a method for identifying moving wear particles based on multi-perspective features, the method comprising obtaining a video of moving wear particles, and obtaining multi-perspective images of the wear particles in the video of moving wear particles; determining an area change curve, an aspect ratio change curve, and a roundness change curve based on the multi-perspective images; and identifying the wear particle type of the moving wear particles based on the obtained characteristic change curves. This application obtains a video of the moving wear particles, and then determines a multi-angle image based on the video of the moving wear particles. The multi-perspective information of the wear particles can be obtained through the multi-angle images, and the number and type of wear particles contained in the wear tissue fluid can be quickly determined, thereby improving the efficiency of wear particle analysis. At the same time, this application obtains the characteristic change curves of the area, aspect ratio, and roundness of the images of each perspective, and fuses the characteristic change curves to identify the wear particle type, thereby ensuring the accuracy and recognition speed of the identified wear particle type.

[0134] To further illustrate the effectiveness of the multi-view feature-based motion abrasive particle identification method provided in this embodiment, this embodiment uses a friction and wear testing machine to generate artificial joint abrasive particles. The upper pair of the artificial joint is made of forged stainless steel balls, and the lower pair is made of titanium alloy TC4. The experimental equipment is an Rtec friction and wear testing machine. During the experiment, the upper friction pair remains stationary, and the lower friction pair reciprocates with the fixture at a frequency of 2Hz and a sliding stroke of 2mm. By adjusting the load (50N-100N) and wear time (60min-120min), abrasive particle samples with different wear degrees and different wear modes are obtained. It should be noted that most of the abrasive particles generated in the sliding friction and wear test are flaky. In order to expand the variety of abrasive particle types and verify the adaptability of the abrasive particle type identification method in this article, a mixture of aerosolized spherical abrasive particles and water-atomized block abrasive particles were mixed in the abrasive particle samples. Among them, there are 172 abrasive particles of four types: flake, block, spherical, and strip. The specific number is shown in Table 3. These samples will be used in the following base classifier training.

[0135] The model training and testing were completed on the same computer with Windows 10-64 bit operating system, Core i9-9900K CPU, 3.6GHz and 32GB RAM, and GeForce RTX 2080 GPU with 8GB.

[0136] Table 3 Number of samples of different types of abrasives

[0137]

[0138] experiment

[0139] After completing the training of the base classifier, a video of titanium alloy wear particles in motion that was not used in the training was selected for testing. The video selected here is 4 minutes and 50 seconds long, with a total of 8711 frames. After detection and tracking, 158 valid wear particles were obtained. They were tested on the same dataset with Inception Time, HIVE-COTE2, and MultiRocket. The recognition accuracy, recall rate, and F-measure of each method for each type of wear debris were calculated. The calculation formulas for recognition accuracy, recall rate, and F-measure are:

[0140]

[0141]

[0142]

[0143] TP represents the number of wear chips that are recognized as positive samples, FP represents the number of wear chips that are recognized as negative samples, and MP represents the number of wear chips that are recognized as positive samples.

[0144] As shown in Table 4, the wear particle identification results for each method show that the method provided in this example achieved the best performance, reaching 90.51%. The HIVECOTE 2.0 algorithm, which integrates four different algorithms, including Rocket and DrCIF, also performed well, achieving an overall accuracy of 84.18%. The Inception and MultiRocket models performed poorly.

[0145] Table 4 Wear debris recognition results of different models

[0146]

[0147] Furthermore, in addition to evaluating accuracy, algorithm speed is also a very important metric. Table 5 compares the efficiency of four methods. In terms of runtime, our method has the fastest performance. MultiRocket, due to its use of the Rocket algorithm's core, exhibits a relatively fast runtime, achieving an average of 9.9 fps. Our method runs slightly faster than MultiRocket, achieving an average of 10.5 fps. The Inception Time algorithm, however, exhibits the slowest processing speed due to its high number of parameters in the internal network and therefore places higher demands on computing power.

[0148] Table 5 Comparison of operating efficiency of different methods

[0149]

[0150] Based on the above-mentioned method for identifying moving wear particles based on multi-view features, this embodiment provides a system for identifying moving wear particles based on multi-view features, such as Figure 15 As shown, the system includes:

[0151] An acquisition module 100 is configured to acquire a video of moving wear particles and to acquire multi-view images of the wear particles in the video of moving wear particles;

[0152] A determination module 200 is configured to determine several characteristic change curves of the moving abrasive particles based on the multi-view images of the abrasive particles, wherein the several characteristic change curves include an area change curve, an aspect ratio change curve, and a roundness change curve;

[0153] The identification module 300 is used to identify the abrasive particle type of the moving abrasive particles based on a number of characteristic change curves.

[0154] Based on the above-mentioned method for identifying moving wear particles based on multi-view features, this embodiment provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in the method for identifying moving wear particles based on multi-view features as described in the above-mentioned embodiment.

[0155] Based on the above-mentioned method for identifying moving wear particles based on multi-view features, the present application also provides a terminal device, such as Figure 16 As shown, it includes at least one processor 20; a display screen 21; and a memory 22. It may also include a communications interface 23 and a bus 24. The processor 20, display screen 21, memory 22, and communications interface 23 can communicate with each other via bus 24. The display screen 21 is configured to display a preset user guidance interface in the initial setup mode. The communications interface 23 can transmit information. The processor 20 can call the logic instructions in the memory 22 to execute the method in the above embodiment.

[0156] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0157] The memory 22, as a computer-readable storage medium, can be configured to store software programs or computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes the software programs, instructions, or modules stored in the memory 22 to perform functional applications and data processing, thereby implementing the methods in the above embodiments.

[0158] The memory 22 may include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data created based on the use of the terminal device. In addition, the memory 22 may include high-speed random access memory and non-volatile memory. For example, various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, may also be transient storage media.

[0159] In addition, the specific process of loading and executing the multiple instructions in the storage medium and the processor in the terminal device has been described in detail in the above method and will not be described here one by one.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for identifying moving wear particles based on multi-view features, characterized in that: The method comprises: Acquire a video of moving abrasive particles, and acquire multi-view images of the abrasive particles in the video of moving abrasive particles; determining a plurality of characteristic change curves based on the multi-view images, wherein the plurality of characteristic change curves include an area change curve, an aspect ratio change curve, and a roundness change curve; identifying the abrasive particle type of the moving abrasive particles based on the plurality of characteristic change curves; Determining a number of feature change curves based on the multi-view images specifically includes: Extracting the wear particle region of each viewing angle image in the multi-view images, and determining the wear particle area, aspect ratio, and roundness corresponding to each viewing angle image based on each wear particle region; Determine the area change curve, aspect ratio change curve and roundness change curve of the abrasive particles according to the abrasive particle area, aspect ratio and roundness corresponding to the images of each viewing angle; The identifying the type of the moving abrasive particles based on the plurality of characteristic change curves specifically includes: Obtaining a change period of each characteristic change curve from among the plurality of characteristic change curves, and selecting a single-period characteristic change curve from among the characteristic change curves based on the change period; The characteristic change curve is a time domain signal; For each single-period feature region, extracting a feature vector of the single-period feature change curve; identifying the abrasive particle type of the moving abrasive particle based on each feature vector; The identifying the abrasive particle type of the moving abrasive particles based on each characteristic vector specifically includes: For each feature vector, obtaining an SVM classification model corresponding to the feature vector, and determining the probability of the wear particle type corresponding to the feature vector using the SVM classification model; Based on the determined probabilities of the wear particle types, the wear particle types of the moving wear particles are identified.

2. The method for identifying moving wear particles based on multi-view features according to claim 1, characterized in that: The acquiring of the multi-view images of the abrasive particles in the moving abrasive particle video specifically includes: Obtaining a background model of the moving wear particle video; Subtracting the moving wear particle video from the background model to obtain a moving target video; Target tracking is performed on the moving target video to obtain multi-view images of the wear particles in the moving wear particle video.

3. The method for identifying moving wear particles based on multi-view features according to claim 1, characterized in that: The variation period is determined based on the autocorrelation function.

4. The method for identifying moving wear particles based on multi-view features according to claim 1, characterized in that: The extracting of the characteristic vector of the single-period characteristic change curve specifically includes: A plurality of statistical features of the single-period characteristic change curve are extracted based on a statistical method to obtain a feature vector.

5. A moving wear particle identification system based on multi-view features, characterized in that: The moving wear particle identification system based on multi-view features is applied to the moving wear particle identification method based on multi-view features according to any one of claims 1 to 4, and the system comprises: An acquisition module is used to acquire a video of moving wear particles and acquire multi-view images of wear particles in the video of moving wear particles; a determination module, configured to determine several characteristic change curves of the moving abrasive particles based on the multi-view images of the abrasive particles, wherein the several characteristic change curves include an area change curve, an aspect ratio change curve, and a roundness change curve; The identification module is used to identify the abrasive particle type of the moving abrasive particles based on a plurality of characteristic change curves.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the method for identifying moving wear particles based on multi-view features as described in any one of claims 1 to 4.

7. A terminal device, characterized in that: include: processor, memory, and communication bus; The memory stores a computer-readable program executable by the processor; The communication bus realizes the connection and communication between the processor and the memory; When the processor executes the computer-readable program, the steps of the method for identifying moving wear particles based on multi-view features according to any one of claims 1 to 4 are implemented.

Images