Carp tong operation fatigue monitoring method and system based on internet of things sensing

Abstract

The application provides a pincer operation fatigue monitoring method and system based on Internet of Things sensing, relates to the technical field of clamping tool fatigue monitoring, and comprises the following steps: synchronously collecting multi-angle jaw images and jaw point cloud data of the pincers, and simulating and constructing a 3D model of the jaws; performing jaw use feature extraction, predicting the front clamping fatigue degree of the pincers, and outputting a first operation fatigue index; obtaining a grip strength sequence of the pincers in a detection empty window time zone through pressure sensing monitoring, analyzing the operation strength and operation frequency according to the grip strength sequence, predicting the front clamping fatigue degree of the pincers based on the operation strength features and the operation frequency features, and outputting a second operation fatigue index; setting a dynamic weight proportion, fusing the first and second operation fatigue indexes to obtain a predicted operation fatigue coefficient of the pincers, and setting the predicted operation fatigue coefficient as a fatigue monitoring result. The technical problems of low accuracy and insufficient precision of the pincer fatigue monitoring in the prior art are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of clamping tool fatigue monitoring, in particular to a pincer operation fatigue monitoring method and system based on Internet of Things sensing. BACKGROUND

[0002] As a high-frequency manual clamping tool, pincers are widely used in automobile manufacturing, heavy equipment maintenance, mechanical assembly, industrial repair and other scenes, and their performance stability is directly related to operation efficiency and operation safety. In daily operation, pincers need to bear dynamic clamping force, high-frequency opening and closing impact, and metal friction for a long time, and are prone to fatigue damage such as crack extension of the jaw and bending deformation of the arm. If fatigue damage is not found in time, it will not only lead to early tool scrap and interruption of operation process, but also may cause safety accidents such as falling of clamped parts and hand injury of operators due to sudden failure during high-strength clamping.

[0003] However, the existing pincer fatigue monitoring method is mainly based on manual experience judgment, which relies on operators to observe the surface damage of the jaw with their eyes, perceive the clamping stability by hand feeling, or send it to the detection point for offline detection regularly. This method has the problems of low accuracy and insufficient precision, which further leads to untimely early fatigue detection or high replacement frequency resulting in waste.

[0004] Therefore, there is an urgent need for a pincer operation fatigue monitoring method that integrates Internet of Things sensing, three-dimensional modeling and intelligent algorithms. SUMMARY

[0005] The present application provides a pincer operation fatigue monitoring method and system based on Internet of Things sensing to solve the technical problems of low accuracy and insufficient precision of existing pincer fatigue monitoring.

[0006] The technical solutions of the present application to solve the above technical problems are as follows:

[0007] In a first aspect, the present application provides a pincer operation fatigue monitoring method based on Internet of Things sensing, comprising:

[0008] Synchronously collecting multi-angle jaw images and jaw point cloud data of the pincer, and simulating and constructing a jaw 3D model according to the multi-angle jaw images and the jaw point cloud data;

[0009] Extracting jaw use features based on the jaw 3D model, predicting the front clamping fatigue degree of the pincer according to the jaw use features, and outputting a first operation fatigue index;

[0010] The grip strength sequence of the pincers in the empty window time zone is acquired through pressure sensing monitoring, operation strength and operation frequency analysis are performed according to the grip strength sequence, front clamping fatigue degree prediction of the pincers is performed based on the operation strength feature and the operation frequency feature, and a second operation fatigue index is outputted;

[0011] The dynamic weight proportion is set based on the operation strength feature and the operation frequency feature, the first operation fatigue index and the second operation fatigue index are fused to obtain a predicted operation fatigue coefficient of the pincers, and the predicted operation fatigue coefficient is set as a fatigue degree monitoring result.

[0012] In a second aspect, the present application provides a pincers operation fatigue degree monitoring system based on Internet of Things sensing, comprising:

[0013] A model construction module is configured to synchronously acquire multi-angle jaw images and jaw point cloud data of the pincers, and simulate and construct a jaw 3D model according to the multi-angle jaw images and the jaw point cloud data.

[0014] A first fatigue degree prediction module is configured to extract jaw use features based on the jaw 3D model, and perform front clamping fatigue degree prediction of the pincers according to the jaw use features, and output a first operation fatigue index.

[0015] A second fatigue degree prediction module is configured to acquire a grip strength sequence of the pincers in an empty window time zone through pressure sensing monitoring, perform operation strength and operation frequency analysis according to the grip strength sequence, perform front clamping fatigue degree prediction of the pincers based on the operation strength feature and the operation frequency feature, and output a second operation fatigue index.

[0016] A fusion output module is configured to set a dynamic weight proportion based on the operation strength feature and the operation frequency feature, fuse the first operation fatigue index and the second operation fatigue index to obtain a predicted operation fatigue coefficient of the pincers, and set the predicted operation fatigue coefficient as a fatigue degree monitoring result.

[0017] The present application has the following beneficial effects:

[0018] Compared with the prior art, firstly, the multi-angle jaw image and the jaw point cloud data of the fish forceps are synchronously collected, the jaw 3D model is simulated and constructed according to the multi-angle jaw image and the jaw point cloud data, the accurate conversion from two-dimensional information to three-dimensional model is realized, and a digital analysis carrier with macrostructure accuracy and microstructure integrity is provided for subsequent extraction of fatigue features. Secondly, the jaw usage feature is extracted based on the jaw 3D model, the front clamping fatigue degree of the fish forceps is predicted according to the jaw usage feature, and a first operation fatigue index is output. Through deep learning, the physical loss feature of the jaw is converted into a quantifiable fatigue index, and the accurate evaluation of the fatigue state of the fish forceps itself is realized. Thirdly, the grip strength sequence of the fish forceps in the detection empty window time zone is obtained through pressure sensor monitoring, the operation intensity and operation frequency are analyzed according to the grip strength sequence, the front clamping fatigue degree of the fish forceps is predicted based on the operation intensity feature and the operation frequency feature, and a second operation fatigue index is output. The influence of personnel operation on the fish forceps is accurately quantitatively evaluated. Finally, the dynamic weight proportion is set based on the operation intensity feature and the operation frequency feature, the first operation fatigue index and the second operation fatigue index are fused to obtain the predicted operation fatigue coefficient of the fish forceps, which is set as the fatigue degree monitoring result, so that the final fatigue degree evaluation can adapt to different use intensity and stability scenes.

[0019] Through the above technical solution, the multi-angle jaw image and the point cloud data are synchronously collected and the jaw 3D model is constructed, the micro-damage and macro-structure deformation of the jaw can be completely restored, the problem that the hidden damage cannot be identified by artificial naked eye is solved; the fatigue is quantitatively evaluated from the physical loss of the fish forceps itself, the subjective deviation of artificial judgment is avoided; the grip strength sequence in the detection empty window time zone is captured in real time through the pressure sensor, the operation intensity and operation frequency features are further analyzed, the dynamic superimposed influence of the operation end on the tool fatigue is accurately quantified, the traditional monitoring only focuses on the tool itself and ignores the interactive fatigue acceleration effect of the tool, the dynamic weight proportion is set based on the operation intensity feature and the operation frequency feature, the evaluation emphasis can be adaptively adjusted according to different operation scenes, and it is ensured that the predicted operation fatigue coefficient can accurately match the actual fatigue state of the tool. In this way, the real-time and accurate monitoring of the fatigue degree of the fish forceps is realized, the fatigue failure risk can be warned in advance, the safety accidents such as part falling and hand cut are effectively avoided, the cost waste caused by excessive replacement of tools and the operation interruption caused by untimely replacement are avoided, data support is provided for tool whole life cycle management, and operation safety and production efficiency in industrial scenes are improved. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 A flowchart of a fish forceps operation fatigue degree monitoring method based on Internet of Things sensing provided by the present application is shown.

[0021] Figure 2A structure schematic diagram of a pincers operation fatigue monitoring system based on Internet of Things sensing provided by the present application is provided.

[0022] In the drawings, the components represented by various reference numerals are as follows:

[0023] The model construction module 11, the first fatigue degree prediction module 12, the second fatigue degree prediction module 13, and the fusion output module 14. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0025] In the description of the present application, the terms "first", "second" are used only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise explicitly specified.

[0026] In the description of the present application, the term "for example" is used to indicate "as an example, illustration or explanation". Any embodiment described as "for example" in the present application is not necessarily interpreted as more preferred or more advantageous than other embodiments. The following description is given in order to enable any person skilled in the art to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that those skilled in the art can realize the present application without using these specific details. In other examples, well-known structures and processes will not be described in detail in order to avoid unnecessary details making the description of the present application obscure. Therefore, the present application is not intended to be limited to the shown embodiments, but is consistent with the broadest scope of principles and features disclosed.

[0027] In one embodiment, as shown in the accompanying drawings, the present application provides a pincers operation fatigue monitoring method based on Internet of Things sensing, comprising: Figure 1

[0028] S10: synchronously collecting multi-angle pincers image and pincers point cloud data of the pincers, and simulating and constructing a pincers 3D model according to the multi-angle pincers image and pincers point cloud data.

[0029] ​Traditional pincers operation fatigue monitoring methods usually adopt manual visual inspection or empirical judgment, completely rely on manual subjective experience, lack of digital collection and quantitative analysis of three-dimensional morphological characteristics of the jaw, and thus it is difficult to accurately capture tool wear condition, thereby leading to problems of insufficient monitoring accuracy and large error.

[0030] In view of the above problems, the multi-angle jaw image and the jaw point cloud data of the pincers are synchronously collected, and a jaw 3D model is simulated and constructed according to the multi-angle jaw image and the jaw point cloud data.

[0031] For example, the multi-angle jaw image can be captured by means of a multi-camera array or a rotatable image collection module, 8-12 uniformly distributed collection angles (such as front, left and right 45° side, top, bottom, etc.) are arranged around the key regions of the jaw (such as the blade edge, the pivot connection, and the clamping surface), and a high-definition industrial camera is used to capture the image at each angle, and the micro-damage details on the surface of the jaw are recorded, such as the extension trajectory of the crack and the profile of the edge gap, etc., to provide basic data for subsequent extraction of surface wear characteristics.

[0032] For example, the jaw point cloud data can be realized by a laser three-dimensional scanner or a depth camera, a laser beam or infrared light is projected onto the surface of the jaw, and the three-dimensional space coordinates (X, Y, Z axis data) of a large number of points are calculated by receiving the reflected signal, and finally the jaw point cloud data composed of hundreds of thousands to millions of points is formed.

[0033] For example, the synchronous collection is realized by hardware clock synchronization and space calibration, such as connecting multiple devices to the same clock module, pre-calibrating the relative position of the camera and the scanner, establishing a unified coordinate system, etc., to ensure that the multi-angle image and the point cloud data correspond to the jaw at the same time and in the same state, and to avoid feature matching errors caused by collection time difference or spatial misalignment.

[0034] Specifically, step S10 in the method comprises:

[0035] The edge detection algorithm is used to extract the edge profile features of the multi-angle jaw image respectively, and the multi-angle edge profile is output.

[0036] The multi-angle edge profile is fitted according to the image collection agreement, and a three-dimensional jaw profile is obtained.

[0037] The point cloud fitting of the jaw point cloud data is performed based on the three-dimensional jaw profile, the best jaw point cloud fitting result is obtained, and the jaw 3D model is simulated and constructed based on the best jaw point cloud fitting result.

[0038] In the embodiments of the present application, first, an edge detection algorithm is used to extract edge contour features of the multi-angle jaw images respectively, and multi-angle edge contours are output. The edge detection algorithm is used to filter the background, light and shadow and other redundant information in the multi-angle jaw images, and only the contour lines of the jaw, such as the edge of the blade, the handle contour, the boundary of the pivot connection and the like, are reserved. Exemplarily, the Canny algorithm can be used to extract edge contour features of the collected multi-angle fish forceps jaw images, and a two-dimensional line set composed of continuous edge lines is output as the jaw edge contour of the corresponding angle by suppressing noise interference and detecting image gradient mutation regions. In this way, through multi-view information fusion, the overall contour, opening angle, handle and jaw connection structure and other macro morphological features of the fish forceps jaw can be accurately captured, thereby providing a geometric framework basis for subsequent 3D modeling.

[0039] Secondly, the multi-angle edge contours are fitted according to the image acquisition protocol to obtain a three-dimensional jaw contour. The image acquisition protocol refers to the spatial parameter information recorded in the image acquisition process, which includes not only the specific collection angles, such as 0° front, 90° side and 45° oblique, but also the positioning data such as the relative distance between the camera and the jaw and the lens focal length, and these information constitutes the spatial coordinate reference of the multi-angle edge contour.

[0040] Exemplarily, after obtaining the accurate spatial angle and collection position information of the multi-angle edge contour from the image acquisition protocol, the multi-angle edge contour is spatially spliced and fitted through a coordinate system conversion algorithm, for example, the horizontal axis of the jaw blade in the front profile and the vertical axis of the handle in the side profile are spatially associated according to the actual collection angle (such as 90° orthogonal between the front and the side), and at the same time, the distance parameter is calibrated to the proportion, and finally a three-dimensional jaw contour is constructed. The three-dimensional jaw contour can completely reflect the overall shape and key structural relationship of the jaw, such as the jaw head curvature, handle curvature, jaw opening angle, relative distance between the pivot and the handle end point, and the like, but the three-dimensional jaw contour is only a wireframe structure, which lacks the real surface texture, small recess and other detailed features of the jaw.

[0041] Finally, based on the three-dimensional jaw contour, the point cloud data of the jaw is fitted to obtain the best jaw point cloud fitting result, and a jaw 3D model is simulated and constructed based on the best jaw point cloud fitting result. The point cloud data of the jaw is obtained by an industrial three-dimensional scanning technology, such as laser triangulation scanning, structured light scanning, etc., and is a data set containing a large number of three-dimensional coordinate points. Each three-dimensional coordinate point corresponds to a physical position on the surface of the jaw, which can not only reflect the overall three-dimensional shape of the jaw, but also capture millimeter-level or even micron-level surface micro details, such as the extension track of a small crack, the depth and area of a surface pit, and the wear marks of a blade edge. It should be noted that the simulation technology involved in the present application uses mature technology methods in the art, such as polygon modeling and surface reconstruction functions in three-dimensional modeling software. The specific implementation mode is prior art, which will not be described here.

[0042] Exemplarily, based on the three-dimensional jaw contour as a spatial reference, the point cloud data is constrained to fit the macro geometric shape of the three-dimensional jaw contour by a point cloud registration algorithm, while the discrete point cloud data is smoothed, denoised and surface fitted to fill the surface vacancy of the three-dimensional jaw contour and restore the micro morphology of the jaw surface. Finally, the best jaw point cloud fitting result is obtained, and then a complete jaw 3D model is constructed based on the best jaw point cloud fitting result by simulation technology. The jaw 3D model not only retains the macro structural features of the three-dimensional jaw contour, such as the jaw opening angle, handle length, relative position of the jaw head and pivot, etc., but also accurately reproduces the micro defect details recorded by the point cloud data, such as surface cracks and local pits, etc. This provides a digital analysis carrier with both macro structural accuracy and micro detail integrity for subsequent extraction of fatigue features such as crack depth, pit volume, and pivot axis parallelism by algorithm.

[0043] Specifically, the "point cloud fitting of the jaw point cloud data based on the three-dimensional jaw contour as a reference to obtain the best jaw point cloud fitting result" comprises:

[0044] The adaptive fitting convergence number is set based on the image acquisition accuracy and the point cloud acquisition accuracy, wherein the adaptive fitting convergence number is negatively correlated with the image acquisition accuracy and the point cloud acquisition accuracy;

[0045] Based on the three-dimensional jaw contour as a reference, the jaw point cloud data is randomly fitted once to obtain a first jaw point cloud fitting result, and the ratio of the number of point clouds in the first jaw point cloud fitting result to the total number of point clouds in the jaw point cloud data is calculated as a first fitting accuracy;

[0046] The jaw point cloud data is randomly fitted again based on the three-dimensional jaw contour as a reference to obtain a second jaw point cloud fitting result and a second fitting accuracy;

[0047] The point cloud iterative fitting is continuously performed until the fitting is stopped when the adaptive fitting convergence number is reached, and a jaw point cloud fitting result with the maximum fitting accuracy in the point cloud fitting process is output as the best jaw point cloud fitting result.

[0048] In the embodiments of the present application, first, the adaptive fitting convergence number is set based on the image acquisition accuracy and the point cloud acquisition accuracy. The adaptive fitting convergence number refers to the maximum number of iterations of the point cloud fitting, which is negatively correlated with the image acquisition accuracy and the point cloud acquisition accuracy, that is, the clearer the collected image and the denser the point cloud, the fewer the number of iterations required. This is because high-precision data itself contains more effective information, and does not need to be iterated multiple times to fit accurately. On the contrary, low-precision data usually needs more attempts to achieve the expected convergence effect to make up for the lack of information.

[0049] Exemplarily, the image acquisition accuracy (such as image resolution) and the point cloud acquisition accuracy (such as point cloud density) can be obtained through device parameters. Based on the principle that the adaptive fitting convergence number is negatively correlated with the image acquisition accuracy and the point cloud acquisition accuracy, for example, for a high-definition image (such as 4k) + high-density point cloud, because the data contains sufficient effective features, 10 iterations can meet the fitting accuracy requirement; and for a blurred image (such as 720P resolution) + low-density point cloud, because the data has high noise and sparse features, it needs to be improved to 20 iterations to cover more effective point cloud combinations to ensure the reliability of the fitting result. Those skilled in the art can dynamically set the adaptive fitting convergence number according to the actual situation.

[0050] Secondly, based on the three-dimensional jaw contour, the jaw point cloud data is randomly fitted once to obtain a first jaw point cloud fitting result, and the ratio of the number of point clouds in the first jaw point cloud fitting result to the total number of point clouds in the jaw point cloud data is calculated as the first fitting accuracy. The calculation logic of the first fitting accuracy is: the ratio of the number of point clouds participating in the fitting (i.e. the point clouds meeting the three-dimensional jaw contour reference) in the first jaw point cloud fitting result to the total number of point clouds in the jaw point cloud data. The higher the ratio, the more effective point clouds covered in the fitting process, and the more consistent with the three-dimensional jaw contour, and the higher the first fitting accuracy.

[0051] Exemplarily, when the first random fitting is performed based on the three-dimensional jaw contour as a spatial reference, the point clouds meeting the contour geometric constraints are selected from the jaw point cloud data. For example, if the total number of point clouds in the jaw point cloud data is 1000, the number of point clouds participating in the fitting in the first jaw point cloud fitting result is 800, and the remaining 200 are judged as noise points or invalid points because they deviate from the contour reference, then the first fitting accuracy = 800 / 1000 = 80%, which directly reflects the coverage breadth and fitting quality of the effective point clouds in this fitting. The higher the first fitting accuracy, the higher the proportion of effective point clouds meeting the three-dimensional jaw contour reference included in the fitting in the fitting process.

[0052] Again, according to the same method, continue to randomly fit the jaw point cloud data based on the three-dimensional jaw contour, obtain a second jaw point cloud fitting result, and calculate the ratio of the number of point clouds in the second jaw point cloud fitting result to the total number of point clouds in the jaw point cloud data as a second fitting accuracy.

[0053] Finally, according to the same method, continue to perform point cloud iterative fitting until the fitting is stopped when the adaptive fitting convergence number is reached, and output the jaw point cloud fitting result with the maximum fitting accuracy in the point cloud fitting process as the best jaw point cloud fitting result. The purpose of multiple random fitting is to avoid the contingency of single fitting and to improve the probability of finding the optimal result by covering more possible point cloud combinations. The jaw point cloud fitting result with the maximum fitting accuracy can cover the effective point cloud data to the greatest extent and ensure that the surface details of the jaw 3D model are consistent with the actual jaw state, providing a reliable digital basis for subsequent fatigue feature extraction.

[0054] In summary, compared with the prior art, the present application synchronously collects multi-angle jaw images and jaw point cloud data of a carp jaw, and simulates and constructs a jaw 3D model according to the multi-angle jaw images and the jaw point cloud data. In this way, accurate conversion from two-dimensional information to a three-dimensional model is realized, providing a digital analysis carrier with both macrostructure accuracy and microstructure integrity for subsequent extraction of fatigue features.

[0055] S20: Extracting a jaw use feature based on the jaw 3D model, predicting a carp jaw front clamping fatigue degree according to the jaw use feature, and outputting a first work fatigue index.

[0056] The cracks, edge notches, surface pits and other features accumulated by the carp jaw in long-term front clamping work can reflect the ability of the carp jaw to withstand clamping load and maintain functional stability. Generally speaking, the more serious the damage of the jaw use feature and the more obvious the deformation, the higher the front clamping fatigue degree and the higher the risk of tool failure, so there is a certain positive correlation between the jaw use feature and the front clamping fatigue degree of the carp jaw.

[0057] To solve the above problems, the present application extracts a jaw use feature based on the jaw 3D model, predicts a carp jaw front clamping fatigue degree according to the jaw use feature, and outputs a first work fatigue index.

[0058] Specifically, step S20 in the method comprises:

[0059] extracting a pincer use feature from the pincer 3D model by using a convolutional neural network, and obtaining the pincer use feature, wherein the pincer use feature at least includes a crack feature, an edge notch feature, a surface pit feature, a pivot axis parallelism, an included angle change feature, and an overall bending feature;

[0060] Based on the historical use log of the same type of fish forceps, a sample pincer use feature set is collected, and the fish forceps front clamping fatigue degree under different sample pincer use features is labeled to obtain a sample work fatigue index label set;

[0061] The sample pincer use feature set is used as input, and the sample work fatigue index label set is used as supervision, and a deep learning model is trained to convergence to obtain a first clamping fatigue degree predictor;

[0062] The first clamping fatigue degree predictor is used to predict the front clamping fatigue degree of the fish forceps according to the pincer use feature, and a first work fatigue index is output.

[0063] In the embodiments of the present application, first, a convolutional neural network is used to extract a pincer use feature from a pincer 3D model to obtain a pincer use feature. The pincer use feature at least includes a crack feature, an edge notch feature, a surface pit feature, a pivot axis parallelism, an included angle change feature, and an overall bending feature. Specifically, the crack feature, such as crack length, depth, and distribution density, directly reflects the microscopic damage degree of the pincer; the edge notch feature, such as notch length, width, and depth; the surface pit feature, such as pit number and pit depth; the pivot axis parallelism refers to whether the pivot axis of the two pincer arms of the fish forceps remains parallel, and the greater the pivot axis deviation angle, the more serious the structural deformation; the included angle change feature refers to the difference between the minimum included angle when the pincer is closed and the initial state, such as an initial closed included angle of 5°, which changes to 8° after use, and the included angle change feature is the difference of 3°; the overall bending feature refers to whether the pincer arm as a whole is bent, and the bending curvature or the distance from the straight line, such as a handle bending degree of 2mm / m; these data reflect the fatigue state of the pincer itself from different dimensions.

[0064] Exemplarily, for the three-dimensional characteristics of the jaw 3D model, it can be converted into a data format processable by a convolutional neural network (CNN) in two ways: one is a multi-view projection method, which projects the jaw 3D model from a plurality of preset angles (such as front, side, 45° oblique, etc.) to generate a 2D image sequence, which completely retains the morphological information of the jaw at each viewing angle; the second is voxelization processing, which discretizes the three-dimensional space where the jaw 3D model is located into a regular cubic grid, and represents whether the grid contains the jaw structure in numerical form, realizing digital mapping of three-dimensional information. Among them, the CNN realizes progressive feature learning through multiple convolution kernels: the shallow network identifies basic features such as edges and textures, such as the edge lines of the jaw blade and the texture differences of the surface pits; the deep network fuses, abstracts and reconstructs the shallow features, eliminates irrelevant interference information, and finally extracts high-order features that can accurately reflect the fatigue state of the jaw. After CNN processing, these features will be quantized into standardized digital vectors, such as [2mm (crack length), 0.5mm (notch depth), 0.8° (pivot axis parallelism), 3° (angle change feature), …], forming a structured jaw use feature. In this way, the implicit physical wear information in the jaw 3D model, such as micro-cracks and macro-deformations, is converted into explicit digital features that can be operated by CNN and analyzed by subsequent prediction models.

[0065] It should be noted that the convolutional neural network used in this application is based on the mature technical framework in the art, such as the improved version of VGG and ResNet optimized for three-dimensional features. The specific implementation methods of network layer design, convolution kernel size selection, and activation function configuration are all existing technologies, and do not require additional innovation. Therefore, they will not be described here.

[0066] Secondly, based on the historical use log of similar pincers, the sample pincer use feature set is collected, and the fish pincer front clamping fatigue degree under different sample pincer use features is labeled to obtain a sample operation fatigue index label set. Preferably, the fish pincer front clamping fatigue degree label can be quantified by mechanical experiments, expert evaluation, etc.: the mechanical experiment tests the maximum clamping force decay rate and fatigue fracture threshold of the pincers in different loss states through front clamping load test, providing an objective benchmark for fatigue degree labeling; expert evaluation quantitatively labels the loss condition by combining industrial scene experience.

[0067] Exemplarily, from the historical use logs of the same type of forceps, samples covering different degrees of damage are collected according to the whole life cycle stage: new tools (without any surface damage), light use (slight wear), moderate use (obvious cracks), near scrap (severe deformation), for each sample, the same method as the foregoing steps is used to extract its jaw use features, and the sample jaw use feature set is formed. According to the severity of the feature sample jaw use feature, more than three experts mark the front clamping fatigue degree of the forceps, for example, slight cracks correspond to 0.2, and severe deformation corresponds to 0.8. Finally, the data set corresponding one-to-one between the sample jaw use feature and the front clamping fatigue degree of the forceps is formed as the sample operation fatigue index label set.

[0068] Again, the sample jaw use feature set is used as input, and the sample operation fatigue index label set is used as supervision to train the deep learning model to convergence, and the first clamping fatigue degree predictor is obtained. Exemplarily, the first clamping fatigue degree predictor can be obtained by the following technical path: 1. Data preparation: the sample jaw use feature set and the sample operation fatigue index label set are divided into a training set, a validation set, and a test set according to a ratio of 7:1.5:1.5. The training set is used for model parameter learning, the validation set is used for real-time adjustment of hyperparameters to avoid overfitting, and the test set is used for final evaluation of the model generalization ability to ensure the representativeness of the data distribution. 2. Model construction: since the sample jaw use feature is a structured one-dimensional feature vector, a fully connected neural network architecture can be used to construct the first clamping fatigue degree predictor, which mainly consists of an input layer, a hidden layer, and an output layer. The number of neurons in the input layer is consistent with the dimension of the sample jaw use feature, which is responsible for receiving the standardized feature vector. The hidden layer is set to 2-3 layers, the number of neurons in each layer is set to 64-128, the activation function is ReLU, and a Dropout layer (dropout rate=0.2) is added between adjacent hidden layers to randomly shield some neurons to avoid overfitting. The output layer is set to one neuron, the activation function is Sigmoid, the output value is compressed to the 0-1 interval, and the predicted fatigue index is directly output. 3. Model training: the sample jaw use feature in the training set is used as input, and the corresponding sample operation fatigue index label is used as the supervision label. The Adam optimizer is used, the learning rate is initially set to 0.001, and is adjusted adaptively with iterations. The mean square error (MSE) is used as the loss function. The predicted value is calculated by forward propagation, the loss value is obtained by comparing the true label, the network weights and biases are updated by back propagation to minimize the loss, and the above iteration process is repeatedly performed until the validation set loss does not decrease (fluctuation amplitude <0.001) for 5-10 consecutive iterations, and the prediction accuracy of the test set is stable at more than 90%. It is considered to be converged, and the first clamping fatigue degree predictor with qualified generalization ability and fitting precision is obtained.

[0069] Finally, the first clamping fatigue predictor is used to predict the front clamping fatigue degree of the fish tongs according to the jaw use characteristics, and output a first operation fatigue index, wherein the first operation fatigue index reflects the fatigue degree of the fish tongs caused by physical loss (such as cracks, deformation, etc.) of the fish tongs itself, and the value range is 0-1, 0 represents no fatigue, and 1 represents complete fatigue. For example, a pre-trained first clamping fatigue predictor is called, and the jaw use characteristics [2mm (crack length), 0.5mm (gap depth), 0.8° (pivot axis parallelism), 3° (jaw angle change characteristics), …] are input, and the first operation fatigue index 0.4 is predicted and output.

[0070] In summary, compared with the prior art, the present application extracts the jaw use characteristics based on the jaw 3D model, predicts the front clamping fatigue degree of the fish tongs according to the jaw use characteristics, and outputs a first operation fatigue index. In this way, the physical loss characteristics of the jaw are converted into a quantifiable fatigue index through deep learning, and the fatigue state of the fish tongs itself is accurately evaluated.

[0071] S30: Obtain a grip strength sequence of the fish tongs in the detection empty window time zone through pressure sensor monitoring, analyze the operation intensity and operation frequency according to the grip strength sequence, predict the front clamping fatigue degree of the fish tongs based on the operation intensity characteristics and the operation frequency characteristics, and output a second operation fatigue index.

[0072] During the operation of the fish tongs, unreasonable states of grip strength and operation frequency of the operator can cause fatigue of the fish tongs. High grip strength of the operator can cause the jaw, pivot and other components of the fish tongs to bear concentrated load for a long time, accelerate plastic deformation of the metal material, and increase the friction frequency of the moving parts of the tool and the uneven force impact caused by high-frequency operation or rhythm disorder, which can cause rapid loss of lubricating oil, accelerated wear of surface plating, and further induce fatigue damage such as cracks and increased gaps in the tool.

[0073] To solve the above problems, the present application obtains a grip strength sequence of the fish tongs in the detection empty window time zone through pressure sensor monitoring, analyzes the operation intensity and operation frequency according to the grip strength sequence, predicts the front clamping fatigue degree of the fish tongs based on the operation intensity characteristics and the operation frequency characteristics, and outputs a second operation fatigue index.

[0074] Specifically, step S30 in the method comprises:

[0075] A pressure sensor is deployed on the fish tongs, and a grip strength sequence of the fish tongs in a detection empty window time zone is obtained through periodic pressure sensor monitoring, wherein the detection empty window time zone is a time interval from a latest detection time point to a current time point.

[0076] According to the grip strength sequence, operation intensity analysis is performed, and the maximum grip strength, grip strength mean and grip strength fluctuation coefficient are calculated as operation intensity characteristics;

[0077] According to the grip strength sequence, operation frequency analysis is performed, and the operation frequency mean and operation frequency fluctuation coefficient are calculated as operation frequency characteristics.

[0078] In the embodiment of the application, a pressure sensor is first deployed on the upper part of the pliers, and the grip strength sequence in the detection window time zone is periodically monitored and acquired by the pressure sensor. The detection window time zone is the time interval from the current time point to the nearest detection time point, i.e., the time interval between two consecutive monitoring times, such as 10 seconds, 30 seconds, etc. The skilled in the art can set it dynamically according to the work intensity, for example, if the current detection time is 10:00:00 and the detection window time zone is 10 seconds, the next detection time is 10:00:10. The purpose of setting the detection window time zone is to balance the real-time data and the computing power consumption. High-frequency detection is more accurate but consumes more energy, while low-frequency detection consumes less energy but may miss critical operations.

[0079] For example, a high-sensitivity pressure sensor is deployed in the handle holding area of the pliers, such as a piezoelectric film sensor that can capture 0.1N level grip changes, to ensure that the grip strength during clamping work can be sensed in real time. The pressure sensor periodically collects grip data in each detection window time zone, such as every 0.5 seconds, and sorts the collected time into a grip strength sequence, such as [50N, 52N, 48N, 55N, …]. The length of the grip strength sequence is determined by the detection window time zone and the sampling frequency, for example, if the detection window time zone is 10 seconds and the sampling frequency is 0.5 seconds, the grip strength sequence contains 20 data points.

[0080] Secondly, according to the grip strength sequence, operation intensity analysis is performed, and the maximum grip strength, grip strength mean and grip strength fluctuation coefficient are calculated as operation intensity characteristics. The maximum grip strength refers to the maximum value in the grip strength sequence, reflecting the maximum load applied in a single clamping operation. The greater the maximum grip strength, the more concentrated the force on the jaws, and the faster the fatigue accumulation. The grip strength mean refers to the arithmetic mean of the grip strength sequence, reflecting the average force level in the detection window time zone. The higher the grip strength mean, the greater the overall operation intensity. The grip strength fluctuation coefficient = standard deviation of the grip strength sequence / grip strength mean, reflecting the stability of the grip strength. The greater the grip fluctuation, the greater the grip strength fluctuation coefficient, the more uneven the force on the jaws, and the more serious the local fatigue.

[0081] Exemplarily, the maximum grip strength, such as 55N, is extracted from the grip strength sequence [50N, 52N, 48N, 55N,...], the arithmetic mean of all grip strengths, such as 51.25N, is calculated as the grip strength mean, and the standard deviation of the grip strength sequence, such as 1.82, is calculated to obtain the grip strength fluctuation coefficient = 1.82 / 51.25 = 3.55%. In this way, three-dimensional data are extracted from the grip strength sequence to quantify the size and stability of the force applied by the operator.

[0082] Finally, the operation frequency analysis is performed according to the grip strength sequence to calculate the operation frequency mean and the operation frequency fluctuation coefficient as the operation frequency characteristics.

[0083] The operation frequency mean refers to the average number of opening and closing operations of the fish forceps per standard unit of time (such as per minute or per hour), such as 8 times per minute, which can be determined according to the dynamic change law of the grip strength sequence: by setting a grip strength threshold, such as setting grip strength > 5N as operation start and < 5N as operation end, the peak-valley change period in the grip strength sequence is identified, and when the grip strength rises from below the threshold (close to 0N) to the peak (the maximum force application point of the operation), and then falls below the threshold, it is determined as one complete opening and closing operation. Then, the total number of complete opening and closing operations in the idle window period that meet the above criteria is counted, and the operation frequency mean is obtained by converting the total number of times to the unit time frequency according to the formula: total number of times / idle window period duration x standard unit time. The higher the operation frequency mean, the more frequent the friction between the pivot shaft and the jaw arm per unit time, and the more impact the pivot shaft needs to withstand during each opening and closing process, which will accelerate the consumption of lubricating oil and the wear of metal parts at the pivot shaft, and thus speed up the fatigue loss of the moving parts.

[0084] The operation frequency fluctuation coefficient refers to the ratio of the standard deviation of the time interval between two consecutive complete opening and closing operations to the arithmetic mean of all operation intervals, which is used to quantify the stability of the operation rhythm. For example, if 4 complete opening and closing operations are identified from the grip strength sequence, the operation intervals (the time difference between the start of adjacent two operations) are 5 seconds, 6 seconds, 5 seconds, and 6 seconds, respectively, then the operation interval mean = (5+6+5+6) / 4 = 5.5 seconds / time, and the standard deviation of the operation interval is 0.5 seconds, and finally the operation frequency fluctuation coefficient = 0.5 / 5.5 = 9% is obtained. The larger the operation frequency fluctuation coefficient, the higher the dispersion degree of the operation interval, which means that the operation rhythm of the operator is more chaotic, and the jaw needs to be frequently switched between the stressed working state and the idle standby state. This unstable working mode will cause the load on the key components such as the jaw pivot shaft and the blade edge to lack regularity, and cannot form a uniform fatigue accumulation process, which is prone to stress concentration at the load mutation and accelerates local fatigue damage.

[0085] Further, the "carp forceps front clamping fatigue degree prediction based on the operation intensity feature and the operation frequency feature, and outputting a second work fatigue index" comprises:

[0086] Pre-training a second clamping fatigue degree predictor, wherein the second clamping fatigue degree predictor comprises Q second prediction plug-ins, Q is not less than 10;

[0087] Based on the grip strength fluctuation coefficient and the operation frequency fluctuation coefficient, a use fluctuation coefficient is evaluated, which is set as a second prediction complexity coefficient. The product of the second prediction complexity coefficient and an initial plug-in selection number is rounded to obtain an optimal plug-in selection number K, wherein K is greater than or equal to 3 and less than or equal to Q, the initial plug-in selection number is 5, and the second prediction complexity coefficient is positively correlated with the grip strength fluctuation coefficient and the operation frequency fluctuation coefficient;

[0088] Among the Q second prediction plug-ins, K prediction plug-ins are randomly selected, the operation intensity feature and the operation frequency feature are used to predict the carp forceps front clamping fatigue degree, and after the K prediction results are averaged, a second work fatigue index is obtained.

[0089] In the embodiments of the present application, a second clamping fatigue degree predictor is first pre-trained, wherein the second clamping fatigue degree predictor comprises Q second prediction plug-ins, Q is not less than 10, and each second prediction plug-in is a lightweight machine learning model, which can be used to predict the carp forceps front clamping fatigue degree based on the operation intensity feature and the operation frequency feature. Exemplarily, the second clamping fatigue degree predictor can be obtained through the following technical path: 1. Data preparation: collect historical operation data of the same type of carp forceps, including complete operation intensity features (including maximum grip strength, grip strength average, grip strength fluctuation coefficient) and operation frequency features (including operation frequency average, operation frequency fluctuation coefficient), and aggregate to form a standardized sample operation feature set. At the same time, a sample tool fatigue degree label set is labeled by artificial labeling, and then a Q-fold cross-validation method is used to randomly divide the sample operation feature set and the sample tool fatigue degree label set into Q mutually exclusive subsets (Q≥10). Each time, one subset is selected as a test benchmark, and the remaining Q-1 subsets are combined as a sample training set, and finally Q independent sample training sets are obtained. For each sample training set, further divide it into a sub-training set (used for plug-in parameter learning), a sub-validation set (used for adjusting hyperparameters), and a sub-test set (used for verifying plug-in accuracy) in the ratio of 7:1.5:1.5.

[0090] 2. Model construction: based on the random forest architecture, a second prediction plug-in is constructed, each second prediction plug-in can be set to 50-100 decision trees, the maximum depth is set to 8-12 to avoid overfitting caused by too deep, each tree randomly selects 3-4 features (total feature number is 5) for splitting to enhance the diversity between trees, and mean square error (MSE) is used as the splitting basis.

[0091] 3. Model training: for each sample training set, the operating intensity features and operating frequency features in the sub-training set are used as joint input, and the corresponding sample tool fatigue degree label is used as the supervision signal to train each second prediction plug-in. During the training process, the splitting nodes and branch rules of the tree are iteratively optimized through the sub-training set to make each tree form independent learning of the mapping relationship between features and fatigue degree. The MSE of the sub-validation set is used as the evaluation index. When the MSE of the sub-validation set decreases by less than 0.001 in 5 consecutive iterations, and the prediction error of the sub-test set is greater than or equal to 85%, it is determined that the plug-in converges. Q sample training sets are trained in the same way to finally obtain Q second prediction plug-ins with independent parameters and different generalization abilities.

[0092] 4. Model combination: the Q second prediction plug-ins trained are packaged into a unified second clamping fatigue degree predictor, the interfaces between the plug-ins remain independent, forming a multi-model collaborative prediction architecture to improve the adaptability to complex operation scenarios.

[0093] Secondly, the use fluctuation coefficient is evaluated based on the grip strength fluctuation coefficient and the operation frequency fluctuation coefficient, and is set as the second prediction complexity coefficient. The optimal plug-in selection number K is obtained by rounding the product of the second prediction complexity coefficient and the initial plug-in selection number. Wherein, K is greater than or equal to 3 and less than or equal to Q, and the initial plug-in selection number is 5. Exemplarily, the second prediction complexity coefficient can be obtained based on the comparison between the standard fluctuation benchmark and the actual fluctuation data. First, the preset standard grip strength fluctuation coefficient and the standard operation frequency fluctuation coefficient are determined based on the industry operation specification and the historical optimal operation data of similar tools, such as the preset standard grip strength fluctuation coefficient being 5% and the standard operation frequency fluctuation coefficient being 10%. When the grip strength fluctuation coefficient is 3.55% and the operation frequency fluctuation coefficient is 9%, the ratio of the grip strength fluctuation coefficient to the standard grip strength fluctuation coefficient is 3.55 / 5=0.71, and the ratio of the operation frequency fluctuation coefficient to the standard operation frequency fluctuation coefficient is 9 / 10=0.9. Then the average value is calculated as (0.71+0.9) / 2=0.805, and the use fluctuation coefficient is obtained as the second prediction complexity coefficient. The second prediction complexity coefficient is positively correlated with the grip strength fluctuation coefficient and the operation frequency fluctuation coefficient. The higher the second prediction complexity coefficient, the more unstable the operation intensity and frequency, and more plug-ins are needed to participate in prediction to cover complex modes.

[0094] Exemplarily, if the second prediction complexity coefficient is 0.805 and the initial plug selection quantity is 5, the product of the second prediction complexity coefficient and the initial plug selection quantity is calculated and rounded to obtain the optimal plug selection quantity K = 4. In this way, when the operation intensity and frequency are unstable, more plugs are selected to ensure the accuracy and robustness of the fatigue degree prediction; when the operation intensity and frequency are stable, fewer plugs are selected to reduce the model calculation amount, reduce the system resource consumption, and ensure the real-time and efficiency of the prediction; the dynamic matching of the plug quantity is realized, and the balance between the prediction accuracy and the prediction efficiency is considered.

[0095] Finally, K prediction plugs are randomly selected from the Q second prediction plugs, and the operation intensity feature and the operation frequency feature are input into the K prediction plugs respectively to perform the fatigue degree prediction of the front clamping of the fish tongs. Each prediction plug outputs a prediction result, and finally, the second job fatigue index is obtained by averaging the K prediction results. For example, when K = 4, 4 prediction plugs are randomly selected from the Q second prediction plugs, and 4 prediction results are output respectively: 0.4, 0.45, 0.35, and 0.4. Then, the average value is calculated as (0.4+0.45+0.35+0.4) / 4=0.4, which is taken as the second job fatigue index. The second job fatigue index can reflect the influence degree of the artificial operation process on the fatigue of the fish tongs.

[0096] In summary, compared with the prior art, the present application acquires the grip strength sequence of the fish tongs in the empty window time zone through pressure sensing monitoring, analyzes the operation intensity and the operation frequency according to the grip strength sequence, performs the fatigue degree prediction of the front clamping of the fish tongs based on the operation intensity feature and the operation frequency feature, and outputs the second job fatigue index. In this way, the influence of personnel operation on the fish tongs is accurately quantitatively evaluated.

[0097] S40: Based on the operation intensity feature and the operation frequency feature, a dynamic weight ratio is set, the first job fatigue index and the second job fatigue index are fused to obtain a predicted job fatigue coefficient of the fish tongs, which is taken as a fatigue monitoring result.

[0098] Due to the significant differences in the actual use scenarios of the fish tongs, the operation intensity feature and the operation frequency feature in different scenarios have completely different superposition effects and dominant degrees on the first job fatigue index and the second job fatigue index. For example, in a high-intensity high-frequency scenario, the acceleration effect of the operation feature on fatigue is much greater than the tool static loss, and at this time, the influence of the operation end is the main factor of fatigue. In a low-intensity low-frequency scenario, the physical loss accumulated by the fish tongs for a long time becomes the dominant factor of fatigue, and the influence of the operation end is relatively weak.

[0099] If a fixed weight evaluation is used, it will cause scene adaptation deviation, so the first and second adaptation weight proportions must be dynamically adjusted to make the final prediction work fatigue coefficient accurately match the fatigue causes in different scenes, ensure that the evaluation results are highly consistent with the actual fatigue state of the tool, and avoid misjudgment caused by fixed weight.

[0100] To solve the above problems, the application sets a dynamic weight proportion based on the operation intensity feature and the operation frequency feature, fuses the first work fatigue index and the second work fatigue index to obtain a prediction work fatigue coefficient of the fish pliers, and sets the prediction work fatigue coefficient as a fatigue degree monitoring result.

[0101] Specifically, step S40 in the method comprises:

[0102] Based on the maximum grip strength, the grip strength mean value and the operation frequency mean value, the use intensity of the fish pliers is evaluated to obtain an initial use intensity index;

[0103] According to the use fluctuation coefficient, the initial use intensity index is compensated to obtain a compensated use intensity index;

[0104] The ratio of the compensated use intensity index to the use intensity mean value of the fish pliers in a historical time range is set as a second weight adjustment coefficient, the second initial weight is compensated to obtain a second adaptation weight, wherein the second initial weight is 0.5, and the second adaptation weight is greater than or equal to 0.2 and less than or equal to 0.7;

[0105] The first adaptation weight is obtained by subtracting the second adaptation weight from 1, and the first adaptation weight and the second adaptation weight are used as a dynamic weight proportion.

[0106] In the embodiments of the present application, first, the use intensity of the fish forceps is evaluated based on the maximum grip strength, the average grip strength and the average operation frequency to obtain an initial use intensity index. For example, first, the maximum grip strength, the average grip strength and the average operation frequency are normalized to eliminate dimensional differences. For example, the maximum safe grip strength of the fish forceps can be set to 100 N, and if it exceeds, it is easy to accelerate fatigue. When the actual maximum grip strength is 55 N, the standardized value is 55 / 100=0.55. Similarly, if the safe grip strength is 60 N, the standardized value of the actual average grip strength is 51.25 N, which is 51.25 / 60=0.85. If the safe operation frequency is 15 times per minute, the standardized value of the actual average operation frequency is 8 times per minute, which is 8 / 15=0.53. Then, the weighted sum is obtained to obtain the initial use intensity index. For example, according to the influence weight of the maximum grip strength, the average grip strength and the operation frequency on the use intensity, the weights are respectively 0.4, 0.3 and 0.3, and then the initial use intensity index is 0.4*0.55+0.3*0.85+0.3*0.53=0.64.

[0107] Secondly, since the initial use intensity index does not consider the influence of operation stability on fatigue, the initial use intensity index is compensated according to the use fluctuation coefficient to obtain a compensated use intensity index. The compensated use intensity index=use fluctuation coefficient*initial use intensity index, for example, if the use fluctuation coefficient is 0.805 and the initial use intensity index is 0.64, then the compensated use intensity index=0.805*0.64=0.52, the greater the fluctuation, the higher the compensated use intensity index under the same intensity.

[0108] Again, the ratio of the compensation use intensity index to the average use intensity index of the fish forceps in the historical time range is set as the second weight adjustment coefficient, and the second initial weight is compensated to obtain the second adaptive weight. The higher the compensation use intensity index, the more significant the influence of operation stability on the fatigue of the fish forceps, the larger the second weight adjustment coefficient, and the larger the second adaptive weight obtained by compensation according to the second weight adjustment coefficient. The average use intensity index of the fish forceps in the historical time range refers to the average value of the compensation use intensity index in the past period of time (such as the last 7 days, historical data under the same working condition), which reflects the historical regular level. The second initial weight is 0.5, and the second adaptive weight is greater than or equal to 0.2 and less than or equal to 0.7. The upper and lower limits of the second adaptive weight are set to avoid the weight being too high or too low to ignore a certain dimension. For example, if the compensation use intensity index is 0.52 and the average use intensity index of the fish forceps in the historical time range is 0.5, the second weight adjustment coefficient = 0.52 / 0.5 = 1.04, and the second adaptive weight = 1.04 x 0.5 = 0.52. The second adaptive weight is the dynamic proportion of the second operation fatigue index of the fish forceps (the index of the dynamic superimposed influence of the human operation process on the fatigue of the fish forceps).

[0109] Finally, the first adaptive weight is obtained by subtracting the second adaptive weight from 1, and the first adaptive weight and the second adaptive weight are used as the dynamic weight proportion to ensure that the evaluation can adapt to the weight balance of the static loss and the dynamic influence of the operation of the fish forceps under different use intensity scenarios. The first adaptive weight = 1-second adaptive weight, for example, if the second adaptive weight is 0.52, the first adaptive weight = 1-0.52 = 0.48, and the first adaptive weight is the dynamic proportion of the first operation fatigue index of the fish forceps (the tool itself loss).

[0110] Further, the first adaptive weight and the second adaptive weight jointly constitute the dynamic weight proportion, and the predicted operation fatigue coefficient of the fish forceps = the first adaptive weight x the first operation fatigue index + the second adaptive weight x the second operation fatigue index. After weighted summation, the two types of fatigue influences of the static loss and the dynamic influence of the operation of the fish forceps can be integrated into a single quantitative index. The predicted operation fatigue coefficient of the fish forceps is set as the fatigue monitoring result, which has the same value range as the first and second operation fatigue indexes. The closer the value is to 0, the lower the current fatigue degree of the fish forceps, and the safety use margin is sufficient. The closer the value is to 1, the higher the fatigue degree, and it is close to the failure threshold, so the tool needs to be stopped for maintenance or replaced in time, which provides intuitive and accurate quantitative basis for the safety use and maintenance decision of the fish forceps.

[0111] In summary, compared with the prior art, the application sets a dynamic weight ratio based on the operation intensity feature and the operation frequency feature, fuses the first operation fatigue index and the second operation fatigue index to obtain a predicted operation fatigue coefficient of the fish forceps, and sets it as a fatigue monitoring result. In this way, the first operation fatigue index (tool self-loss) and the second operation fatigue index (operation influence) are set with a dynamic weight ratio, so that the final fatigue evaluation can adapt to different use intensity and stability scenarios.

[0112] In summary, the embodiments of the application have at least the following technical effects:

[0113] Compared with the prior art, the application first synchronously collects multi-angle jaw images and jaw point cloud data of the fish forceps, and simulates and constructs a jaw 3D model according to the multi-angle jaw images and the jaw point cloud data. In this way, accurate conversion from two-dimensional information to a three-dimensional model is realized, providing a digital analysis carrier with macrostructure accuracy and microstructure integrity for subsequent extraction of fatigue features.

[0114] Secondly, the application extracts jaw use features based on the jaw 3D model, predicts the front clamping fatigue degree of the fish forceps according to the jaw use features, and outputs a first operation fatigue index. In this way, the physical loss features of the jaw are converted into quantifiable fatigue indexes through deep learning, realizing accurate evaluation of the fatigue state of the fish forceps itself.

[0115] Thirdly, the application acquires a grip strength sequence of the fish forceps in the empty window time zone through pressure sensing monitoring, analyzes the operation intensity and the operation frequency according to the grip strength sequence, predicts the front clamping fatigue degree of the fish forceps based on the operation intensity feature and the operation frequency feature, and outputs a second operation fatigue index. In this way, the influence of personnel operation on the fish forceps is accurately quantitatively evaluated.

[0116] Finally, the application sets a dynamic weight ratio based on the operation intensity feature and the operation frequency feature, fuses the first operation fatigue index and the second operation fatigue index to obtain a predicted operation fatigue coefficient of the fish forceps, and sets it as a fatigue monitoring result. In this way, the first operation fatigue index (tool self-loss) and the second operation fatigue index (operation influence) are set with a dynamic weight ratio, so that the final fatigue evaluation can adapt to different use intensity and stability scenarios.

[0117] Through the above technical solution, this application simultaneously acquires multi-angle jaw images and point cloud data to construct a 3D model of the jaws, which can completely restore the microscopic damage and macroscopic structural deformation of the jaws, solving the problem that hidden damage cannot be identified by the naked eye. It quantifies fatigue assessment based on the physical wear of the pliers themselves, avoiding the subjective bias of human judgment. By using pressure sensors to capture the grip strength sequence within the detection window area in real time, it further analyzes the characteristics of operational intensity and frequency, accurately quantifying the dynamic superposition effect of the operating end on tool fatigue, filling the gap in traditional monitoring that only focuses on the tool itself and ignores the accelerated fatigue effect of human-tool interaction. Based on the characteristics of operational intensity and frequency, a dynamic weight ratio is set, which can adaptively adjust the assessment focus according to different work scenarios, ensuring that the predicted work fatigue coefficient accurately matches the actual fatigue state of the tool. Thus, real-time and accurate monitoring of pliers fatigue is achieved, enabling early warning of fatigue failure risks, effectively avoiding safety accidents such as falling parts and hand injuries, and avoiding cost waste caused by excessive tool replacement and work interruptions caused by untimely replacement. It also provides data support for the full life cycle management of tools, improving work safety and production efficiency in industrial scenarios.

[0118] Example 2, as Figure 2 As shown, based on the same inventive concept as the carp pliers operation fatigue monitoring method based on IoT sensing provided in Embodiment 1, this embodiment of the invention also provides a carp pliers operation fatigue monitoring system based on IoT sensing, including:

[0119] Model building module 11 is used to synchronously collect multi-angle images of the carp's jaws and point cloud data of the jaws, and to simulate and build a 3D model of the jaws based on the multi-angle images of the jaws and point cloud data of the jaws.

[0120] The first fatigue prediction module 12 is used to extract the jaw usage features based on the jaw 3D model, predict the fatigue of front clamping of the carp pliers according to the jaw usage features, and output the first work fatigue index.

[0121] The second fatigue prediction module 13 is used to obtain the grip strength sequence of the carp pliers in the detection window time zone through pressure sensor monitoring, perform operation intensity and operation frequency analysis based on the grip strength sequence, predict the fatigue of front clamping of the carp pliers based on the operation intensity characteristics and operation frequency characteristics, and output the second operation fatigue index.

[0122] The fusion output module 14 is used to set a dynamic weight ratio based on the operation intensity characteristics and operation frequency characteristics, fuse the first operation fatigue index and the second operation fatigue index to obtain the predicted operation fatigue coefficient of the carp pliers, and set it as the fatigue monitoring result.

[0123] Specifically, the model building module 11 is used for:

[0124] An edge detection algorithm is used to extract edge contour features from the multi-angle jaw images, and output multi-angle edge contours;

[0125] The multi-angle edge contours are fitted according to the image acquisition requirements, and a three-dimensional jaw contour is obtained;

[0126] The three-dimensional jaw contour is used as a reference to perform point cloud fitting on the jaw point cloud data, and the best jaw point cloud fitting result is obtained, and a jaw 3D model is simulated and constructed based on the best jaw point cloud fitting result.

[0127] Further, the output "using the three-dimensional jaw contour as a reference to perform point cloud fitting on the jaw point cloud data to obtain the best jaw point cloud fitting result" includes:

[0128] The adaptive fitting convergence number is set based on the image acquisition accuracy and the point cloud acquisition accuracy, wherein the adaptive fitting convergence number is negatively related to the image acquisition accuracy and the point cloud acquisition accuracy;

[0129] The three-dimensional jaw contour is used as a reference to perform point cloud fitting on the jaw point cloud data, and the first jaw point cloud fitting result is obtained, and the ratio of the number of point clouds in the first jaw point cloud fitting result to the total number of point clouds in the jaw point cloud data is calculated as the first fitting accuracy;

[0130] The three-dimensional jaw contour is used as a reference to perform point cloud fitting on the jaw point cloud data, and the second jaw point cloud fitting result is obtained, and the second fitting accuracy is calculated;

[0131] The point cloud iterative fitting is continued until the adaptive fitting convergence number is reached, and the jaw point cloud fitting result with the maximum fitting accuracy in the point cloud fitting process is output as the best jaw point cloud fitting result.

[0132] The first fatigue degree prediction module 12 is specifically configured to:

[0133] The jaw 3D model is used to extract jaw usage features using a convolutional neural network, and the jaw usage features are obtained, wherein the jaw usage features at least include crack features, edge notch features, surface pit features, pivot axis parallelism, angle change features and overall bending features;

[0134] Based on the historical usage log of similar pincers, a sample pincer usage feature set is collected, and the fish pincer front clamping fatigue degree under different sample pincer usage features is labeled to obtain a sample operation fatigue index label set;

[0135] Adopting the sample jaw use feature set as input, adopting the sample operation fatigue index label set as supervision, training a deep learning model to convergence, obtaining a first clamping fatigue degree predictor;

[0136] Using the first clamping fatigue degree predictor, predicting the pincer front clamping fatigue degree according to the jaw use features, and outputting a first operation fatigue index.

[0137] Among them, the second fatigue degree prediction module 13 is specifically used for:

[0138] Deploying a pressure sensor on the pincer, periodically monitoring and acquiring a grip strength sequence of the pincer in a detection empty window time zone, wherein the detection empty window time zone is a time interval from the current time point to the nearest detection time point;

[0139] According to the grip strength sequence, operation intensity analysis is performed, and the maximum grip strength, grip strength mean and grip strength fluctuation coefficient are calculated as operation intensity features;

[0140] According to the grip strength sequence, operation frequency analysis is performed, and the operation frequency mean and operation frequency fluctuation coefficient are calculated as operation frequency features.

[0141] Further, the "pincer front clamping fatigue degree prediction based on operation intensity features and operation frequency features, and outputting a second operation fatigue index" includes:

[0142] Pre-training a second clamping fatigue degree predictor, wherein the second clamping fatigue degree predictor includes Q second prediction plugins, and Q is not less than 10;

[0143] Based on the grip strength fluctuation coefficient and the operation frequency fluctuation coefficient, a use fluctuation coefficient is evaluated and set as a second prediction complexity coefficient. The second prediction complexity coefficient is multiplied by the initial plugin selection number to obtain an optimal plugin selection number K, wherein K is greater than or equal to 3 and less than or equal to Q, the initial plugin selection number is 5, and the second prediction complexity coefficient is positively correlated with the grip strength fluctuation coefficient and the operation frequency fluctuation coefficient.

[0144] Randomly selecting K prediction plugins from the Q second prediction plugins, predicting the pincer front clamping fatigue degree according to the operation intensity features and the operation frequency features, and calculating the mean of the K prediction results to obtain a second operation fatigue index.

[0145] Among them, the fusion output module 14 is specifically used for:

[0146] Based on the maximum grip strength, grip strength mean and operation frequency mean, the use intensity of the pincer is evaluated to obtain an initial use intensity index;

[0147] According to the use of the fluctuation coefficient, the initial use intensity index is compensated to obtain a compensated use intensity index;

[0148] The ratio of the compensated use intensity index to the average use intensity of the forceps in a historical time range is set as a second weight adjustment coefficient, and a second initial weight is compensated to obtain a second adaptive weight, wherein the second initial weight is 0.5, and the second adaptive weight is greater than or equal to 0.2 and less than or equal to 0.7;

[0149] A first adaptive weight is obtained by subtracting the second adaptive weight from 1, and the first adaptive weight and the second adaptive weight are used as dynamic weight proportions.

[0150] In summary, the embodiments of the present application have at least the following technical effects:

[0151] Compared with the prior art, the present application first synchronously collects multi-angle jaw images and jaw point cloud data of the forceps through the model construction module, simulates and constructs a jaw 3D model according to the multi-angle jaw images and the jaw point cloud data, realizes accurate conversion from two-dimensional information to a three-dimensional model, and provides a digital analysis carrier with macro-structure accuracy and micro-detail integrity for subsequent extraction of fatigue features. Secondly, through the first fatigue degree prediction module, the jaw use feature is extracted based on the jaw 3D model, the front clamping fatigue degree of the forceps is predicted according to the jaw use feature, and a first operation fatigue index is output. The physical loss feature of the jaw is converted into a quantifiable fatigue index through deep learning, and the fatigue state of the forceps itself is accurately evaluated. Thirdly, through the second fatigue degree prediction module, the grip strength sequence of the forceps in the detection empty window time zone is obtained through pressure sensing monitoring, the operation intensity and operation frequency are analyzed according to the grip strength sequence, the front clamping fatigue degree of the forceps is predicted based on the operation intensity feature and the operation frequency feature, and a second operation fatigue index is output. The influence of personnel operation on the forceps is accurately quantitatively evaluated. Finally, through the fusion output module, a dynamic weight proportion is set based on the operation intensity feature and the operation frequency feature, a predicted operation fatigue coefficient of the forceps is obtained by fusing the first operation fatigue index and the second operation fatigue index, and is set as a fatigue degree monitoring result, so that the final fatigue degree evaluation can adapt to different use intensity and stability scenarios. In this way, real-time and accurate monitoring of the fatigue degree of the forceps is realized, fatigue failure risks can be warned in advance, safety accidents such as part falling and hand injury can be effectively avoided, cost waste caused by excessive replacement of tools and operation interruption caused by untimely replacement can be avoided, data support is provided for tool full life cycle management, and operation safety and production efficiency in industrial scenarios are improved.

[0152] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, since various changes and modifications within the spirit of the application will become apparent to those skilled in the art from this detailed description.

[0153] A person skilled in the art would understand that embodiments of the present application can be provided as methods, systems, or computer program products. Thus, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0154] The present application is described with reference to the accompanying drawings, which show example embodiments of the present application. The drawings incorporated in and constitute a part of this specification, illustrate embodiments of the application and, together with the specification, serve to explain the principles of the application. In the drawings: Figure 1 one or more processes and / or blocks Figure 1 means for performing the function specified by the block or blocks.

[0155] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 one or more processes and / or blocks Figure 1 function specified by the block or blocks.

[0156] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the Figure 1 one or more processes and / or blocks Figure 1 steps of the function specified by the block or blocks.

[0157] Although preferred embodiments of the application have been described, a person of ordinary skill in the art can make additional changes and modifications, once armed with the present disclosure.

[0158] It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application cover modifications and variations of this application provided they come within the scope of the application and their equivalent technology.

Claims

1. A method for monitoring fatigue in carp tongs operation based on Internet of Things (IoT) sensing, characterized in that, The methods include: Simultaneously acquire multi-angle images and point cloud data of the carp's jaws, and simulate and construct a 3D model of the jaws based on the multi-angle images and point cloud data. Based on the 3D model of the jaws, jaw usage features are extracted, and fatigue of front clamping of carp pliers is predicted according to the jaw usage features, and the first operational fatigue index is output. The grip strength sequence of the carp pliers within the detection window time zone is obtained by pressure sensor monitoring. The operation intensity and operation frequency are analyzed based on the grip strength sequence. The fatigue of front clamping of the carp pliers is predicted based on the operation intensity characteristics and operation frequency characteristics, and a second operation fatigue index is output. The detection window time zone is the time interval between the current time point and the most recent detection time point. Based on the operational intensity characteristics and operational frequency characteristics, a dynamic weight ratio is set, and the first operational fatigue index and the second operational fatigue index are fused to obtain the predicted operational fatigue coefficient of the carp pliers, which is set as the fatigue monitoring result. The analysis of operational intensity and frequency based on the grip strength sequence includes: Based on the grip strength sequence, operational intensity analysis is performed to calculate the maximum grip strength, the average grip strength, and the grip strength fluctuation coefficient, which are used as operational intensity characteristics. Based on the grip strength sequence, the operation frequency analysis is performed to calculate the mean operation frequency and the operation frequency fluctuation coefficient, which are used as operation frequency characteristics. Among them, fatigue of front-facing clamping of carp pliers is predicted based on operation intensity characteristics and operation frequency characteristics, and a second operation fatigue index is output, including: A pre-trained second clamping fatigue predictor is provided, wherein the second clamping fatigue predictor includes Q second prediction plug-ins, where Q is not less than 10. The usage fluctuation coefficient is evaluated based on the grip strength fluctuation coefficient and the operation frequency fluctuation coefficient, and is set as the second prediction complexity coefficient. The product of the second prediction complexity coefficient and the initial number of plug-ins selected is rounded to obtain the optimal number of plug-ins selected, K, where K is greater than or equal to 3 and less than or equal to Q. The initial number of plug-ins selected is 5. The second prediction complexity coefficient is positively correlated with the grip strength fluctuation coefficient and the operation frequency fluctuation coefficient. K prediction plugins are randomly selected from the Q second prediction plugins. Based on the operation intensity characteristics and operation frequency characteristics, the fatigue of front clamping of the carp pliers is predicted. After averaging the K prediction results, the second operation fatigue index is obtained.

2. The method for monitoring fatigue in carp tongs operation based on Internet of Things sensing according to claim 1, characterized in that, Based on the multi-angle jaw images and jaw point cloud data, a 3D model of the jaws is constructed through simulation, including: The edge contour features of the multi-angle clamp images are extracted using an edge detection algorithm, and the multi-angle edge contours are output. Based on the image acquisition instructions, the multi-angle edge contours are fitted to obtain the three-dimensional jaw contours. Using the three-dimensional jaw contour as a reference, the jaw point cloud data is fitted to obtain the optimal jaw point cloud fitting result, and a 3D model of the jaw is simulated and constructed based on the optimal jaw point cloud fitting result.

3. The method for monitoring fatigue in carp tongs operation based on Internet of Things sensing according to claim 2, characterized in that, Using the three-dimensional jaw contour as a reference, point cloud fitting is performed on the jaw point cloud data to obtain the optimal jaw point cloud fitting result, including: The number of fitting convergence times is set based on the image acquisition accuracy and the point cloud acquisition accuracy, wherein the number of fitting convergence times is negatively correlated with the image acquisition accuracy and the point cloud acquisition accuracy; Using the three-dimensional jaw contour as a reference, the jaw point cloud data is randomly fitted once to obtain the first jaw point cloud fitting result, and the ratio of the number of points in the first jaw point cloud fitting result to the total number of points in the jaw point cloud data is calculated as the first fitting accuracy. Using the three-dimensional jaw contour as a reference again, the jaw point cloud data is randomly fitted to obtain a second jaw point cloud fitting result and a second fitting accuracy. Continue iterative point cloud fitting until the required number of fitting convergences is reached, then stop fitting and output the jaw point cloud fitting result with the highest fitting accuracy during the point cloud fitting process as the best jaw point cloud fitting result.

4. The method for monitoring fatigue in carp tongs operation based on Internet of Things sensing according to claim 1, characterized in that, Based on the 3D model of the jaws, jaw usage features are extracted. Based on these features, fatigue of frontal clamping with the carp pliers is predicted, and a first operational fatigue index is output, including: The jaw usage features are extracted from the 3D model of the jaws using a convolutional neural network. The jaw usage features include at least crack features, edge notch features, surface pit features, pivot axis parallelism, angle variation features, and overall bending features. Based on the historical usage logs of similar carp pliers, a set of sample plier jaw usage features was collected, and the fatigue of the pliers' front clamping under different sample plier jaw usage features was marked to obtain a set of sample operation fatigue index labels. Using the sample clamp feature set as input and the sample job fatigue index label set as supervision, a deep learning model is trained until convergence to obtain the first clamping fatigue predictor. Using the first clamping fatigue predictor, the fatigue of front clamping of the carp clamp is predicted based on the clamping characteristics of the jaws, and a first operational fatigue index is output.

5. The method for monitoring the fatigue of carp tongs operation based on Internet of Things sensing according to claim 1, characterized in that, Based on the aforementioned operation intensity characteristics and operation frequency characteristics, a dynamic weighting ratio is set, including: The use intensity of the carp pliers is evaluated based on the maximum grip strength, the average grip strength, and the average operating frequency to obtain the initial use intensity index; The initial usage intensity index is compensated based on the usage fluctuation coefficient to obtain the compensated usage intensity index; The ratio of the compensation intensity index to the average carp pliers usage intensity over a historical time range is set as the second weight adjustment coefficient. The second initial weight is then compensated to obtain the second adaptation weight, wherein the second initial weight is 0.5 and the second adaptation weight is greater than or equal to 0.2 and less than or equal to 0.

7. The first adaptation weight is obtained by subtracting the second adaptation weight from 1, and the first adaptation weight and the second adaptation weight are used as the dynamic weight ratio.

6. A fatigue monitoring system for carp tongs based on Internet of Things (IoT) sensing, characterized in that, For performing the method according to any one of claims 1-5, comprising: The model building module is used to synchronously collect multi-angle images and point cloud data of the carp claw, and to simulate and build a 3D model of the claw based on the multi-angle images and point cloud data of the claw. The first fatigue prediction module is used to extract the jaw usage features based on the jaw 3D model, predict the fatigue of front clamping of the carp pliers according to the jaw usage features, and output the first work fatigue index. The second fatigue prediction module is used to obtain the grip strength sequence of the carp pliers within the detection window time zone through pressure sensor monitoring, perform operation intensity and operation frequency analysis based on the grip strength sequence, predict the fatigue of front clamping of the carp pliers based on the operation intensity characteristics and operation frequency characteristics, and output the second operation fatigue index. The fusion output module is used to set a dynamic weight ratio based on the operation intensity characteristics and operation frequency characteristics, fuse the first operation fatigue index and the second operation fatigue index to obtain the predicted operation fatigue coefficient of the carp pliers, and set it as the fatigue monitoring result.

Images