Sound barrier nut state detection method applied to nut dismounting robot

Abstract

The invention discloses a sound barrier nut state detection method applied to a nut dismounting robot, and relates to the field of data processing. The method comprises the steps of obtaining point cloud data of an area where a sound barrier nut is located; based on the point cloud data, constructing a nut three-dimensional model, and performing comparison calculation on the nut three-dimensional model and a standard nut model to obtain a distance field deviation between the nut three-dimensional model and the standard nut model; an abnormal deviation value is determined from the deviation values, the state of the sound barrier nut is determined according to the abnormal deviation value, and the state comprises a normal state and an abnormal state; if the sound barrier nut is determined to be in the abnormal state, dividing the target point cloud corresponding to the abnormal deviation value into a plurality of target point cloud clusters; extracting target features of each target point cloud cluster; according to the target features, abnormal state types corresponding to the abnormal states are recognized, and the abnormal state types comprise nut surface dirt, nut abrasion and nut corrosion. By implementing the technical scheme provided by the invention, the accuracy of nut state detection is improved.

Description

Technical Field

[0001] This application relates to the field of data processing, and particularly to a method for detecting the state of a sound barrier nut applied to a nut disassembly robot. Background Art

[0002] With the rapid development and extensive construction of high-speed railways and other rail transit, the sound barriers along the line have become key facilities to ensure the noise reduction effect and the comfort of residents. These sound barriers usually include a variety of materials such as complex steel structures, sound barrier plates, rubber materials, and bolts, and need to be maintained regularly to maintain their performance and safety.

[0003] Currently, traditional nut state detection technologies mainly rely on visual detection or simple physical measurement. These methods have certain limitations in actual operation. For example, visual detection is easily affected by environmental light and the shadow of the robot itself, which easily leads to inaccurate detection results. Physical measurement methods may not meet the high-precision requirements due to the limited accuracy of robot operation. Therefore, there is a problem of low accuracy in nut state detection in related technologies.

[0004] Therefore, there is an urgent need for a method for detecting the state of a sound barrier nut applied to a nut disassembly robot. Summary of the Invention

[0005] This application provides a method for detecting the state of a sound barrier nut applied to a nut disassembly robot, which improves the accuracy of nut state detection.

[0006] In the first aspect of this application, a method for detecting the state of a sound barrier nut applied to a nut disassembly robot is provided. The method includes: obtaining point cloud data of the area where the sound barrier nut is located, where the point cloud data includes three-dimensional coordinate information of the sound barrier nut; based on the point cloud data, constructing a three-dimensional nut model, and performing a comparison calculation between the three-dimensional nut model and a standard nut model to obtain a distance field deviation between the three-dimensional nut model and the standard nut model, where the distance field deviation includes deviation values of each area on the surface of the sound barrier nut; determining abnormal deviation values from the deviation values, and determining the state of the sound barrier nut according to the abnormal deviation values, where the state includes a normal state and an abnormal state; if it is determined that the sound barrier nut is in an abnormal state, dividing the target point cloud corresponding to the abnormal deviation value into multiple target point cloud clusters; extracting target features of each of the target point cloud clusters, where the target features include shape features, size features, and position features; and identifying the abnormal state type corresponding to the abnormal state according to the target features, where the abnormal state type includes dirt on the nut surface, nut wear, and nut corrosion.

[0007] By adopting the above technical solution, by obtaining the point cloud data of the area where the sound barrier nut is located, constructing a three-dimensional nut model, comparing it with the standard nut model, and calculating the distance field deviation, the differences in each area of the nut surface can be accurately captured. Determine the abnormal deviation value according to the deviation value, and judge whether the nut state is abnormal, realizing the automatic detection and abnormal identification of the nut state. If the nut is in an abnormal state, divide the target point cloud corresponding to the abnormal deviation value into multiple point cloud clusters, extract the shape, size and position characteristics of each point cloud cluster, and identify the specific abnormal state type according to these characteristics, such as dirt, wear or rust on the nut surface, etc. This method realizes the refined detection and diagnosis of the sound barrier nut state through point cloud data analysis and feature extraction, and can accurately detect and locate various defects and abnormal conditions on the nut surface.

[0008] Optionally, based on the point cloud data, constructing a three-dimensional nut model, and comparing and calculating the three-dimensional nut model with the standard nut model to obtain the distance field deviation between the three-dimensional nut model and the standard nut model, specifically includes: performing triangular mesh division on the point cloud data to construct the three-dimensional nut model, and the three-dimensional nut model is composed of multiple triangular mesh patches; registering the three-dimensional nut model with the standard nut model so that the three-dimensional nut model and the standard nut model are located in the same coordinate system; calculating the shortest distance from each grid point on the three-dimensional nut model to the surface of the standard nut model to generate the distance field deviation.

[0009] By adopting the above technical solution, by performing triangular mesh division on the point cloud data to construct a three-dimensional nut model composed of triangular mesh patches, while retaining the detailed information of the point cloud, the data volume can be greatly reduced, and the efficiency of subsequent processing can be improved. Registering the three-dimensional nut model with the standard nut model so that they are located in the same coordinate system provides a basis for subsequent comparison and calculation. By calculating the shortest distance from each grid point on the three-dimensional nut model to the surface of the standard nut model to generate the distance field deviation, the deviation degree between each part of the nut surface and the standard nut model can be accurately characterized. Since the geometric structure of the triangular mesh model is more regular and is easy to perform subsequent feature extraction and analysis, it lays a foundation for realizing the refined detection of the nut state.

[0010] Optionally, determining the abnormal deviation value from the deviation values and determining the state of the sound barrier nut according to the abnormal deviation value specifically includes: calculating the mean and variance of the distance field deviation, and determining the distance field deviation threshold according to the mean and the variance; determining the deviation values greater than or equal to the distance field deviation threshold in the distance field deviation as abnormal deviation values; counting the number of the abnormal deviation values; if the number of the abnormal deviation values is greater than or equal to a preset abnormal threshold, determining the state of the sound barrier nut as an abnormal state; if the number of the abnormal deviation values is less than the preset abnormal threshold, determining the state of the sound barrier nut as a normal state.

[0011] By adopting the above technical solution, by calculating the mean and variance of the distance field deviation and determining the distance field deviation threshold based on the mean and variance, the range of normal deviation and abnormal deviation can be defined adaptively. Determining the deviation values greater than or equal to the preset abnormal threshold as abnormal deviation values and counting the number of abnormal deviation values can quantitatively evaluate the distribution of the abnormal area on the nut surface. By comparing the number of abnormal deviation values with the preset abnormal threshold, it can be automatically determined whether the overall state of the nut is abnormal. This method fully considers the statistical characteristics of the distance field deviation, depicts the overall trend and dispersion degree of the deviation distribution through the mean and variance, and adaptively determines the abnormal determination criterion.

[0012] Optionally, dividing the target point cloud corresponding to the abnormal deviation value into multiple target point cloud clusters specifically includes: randomly determining a preset number of target point clouds from the multiple target point clouds as initial clustering centers; calculating the first distance from each target point cloud to the first clustering center and the second distance from each target point cloud to the second clustering center, where the first clustering center and the second clustering center are any two initial clustering centers among the multiple initial clustering centers; if it is determined that the first distance is less than or equal to the second distance, dividing the target point cloud into the point cloud cluster where the first clustering center is located; for each point cloud cluster, calculating the geometric center of the data points included in each point cloud cluster; determining the geometric center as a new clustering center and performing iteration according to the new clustering center until the geometric centers of each point cloud cluster no longer change, and obtaining the preset number of target point cloud clusters.

[0013] By adopting the above technical solution, by clustering and dividing the target point cloud corresponding to the abnormal deviation value and splitting it into multiple point cloud clusters, the abnormal areas can be automatically separated, providing a basis for subsequent feature extraction and abnormal classification. This method automatically determines the ownership relationship of the point cloud through iterative optimization. By randomly selecting the initial clustering center, calculating the distance from each target point cloud to the clustering center, and determining the point cloud cluster it belongs to according to the distance size until the clustering center is stable, the abnormal point cloud can be adaptively divided into a preset number of point cloud clusters. This unsupervised learning method can automatically discover the internal structure and distribution pattern of the abnormal area, does not require pre-labeled data, and is applicable to complex scenarios such as abnormal detection on the nut surface. At the same time, by controlling the number of clustering categories, the subdivision granularity of the abnormal area can be flexibly adjusted to meet different application requirements.

[0014] Optionally, extracting the target features of each of the target point cloud clusters specifically includes: calculating the three-dimensional bounding box of each of the target point cloud clusters, and extracting the length, width, and height of the three-dimensional bounding box as the size features of each of the target point cloud clusters; calculating the second moment invariant of each of the target point cloud clusters, and extracting the eigenvalue of the second moment invariant as the shape feature of each of the target point cloud clusters; calculating the geometric center coordinates of each of the target point cloud clusters, and extracting the geometric center coordinates as the position features of each of the target point cloud clusters.

[0015] By adopting the above technical solution, by calculating the three-dimensional bounding box of the target point cloud cluster and extracting the length, width, and height of the bounding box as the size features of the point cloud cluster, the spatial span of the abnormal area in three directions can be characterized. This method can effectively quantify the size information of the abnormal area. By calculating the second moment invariant of the target point cloud cluster and extracting its eigenvalue as the shape feature of the point cloud cluster, the geometric shape features of the abnormal area can be characterized. The second moment invariant is invariant to transformations such as translation and rotation, can accurately describe the shape attributes of the point cloud, and is of great significance for the discrimination of abnormal types. By calculating the geometric center coordinates of the target point cloud cluster and taking them as the position features of the point cloud cluster, the approximate position of the abnormal area on the nut surface can be determined. Combining these three types of features can comprehensively characterize the shape, size, and position attributes of the abnormal area, providing rich criteria for subsequent abnormal state recognition.

[0016] Optionally, identifying the abnormal state type corresponding to the abnormal state according to the target feature specifically includes: constructing an abnormal state feature sample library, where the abnormal state feature sample library includes multiple feature samples with labeled abnormal state types, and each feature sample includes shape features, size features, and position features corresponding to the abnormal state type; calculating the feature similarity between the target feature of the target point cloud cluster and each of the feature samples; if it is determined that the target feature similarity is greater than or equal to a preset similarity threshold, determining that the abnormal state type corresponding to the target point cloud cluster is the abnormal state type corresponding to the target feature sample, where the target feature similarity is the feature similarity between the target point cloud cluster and the target feature sample, and the target feature sample is any one of the multiple feature samples.

[0017] By adopting the above technical solution, by constructing an abnormal state feature sample library, the shape features, size features, and position features of the abnormal state samples of known types are labeled and stored, providing a reference standard for abnormal state recognition. During actual recognition, by calculating the similarity between the target feature of the target point cloud cluster and each feature sample, and taking the abnormal type corresponding to the sample with a similarity greater than or equal to the preset similarity threshold as the recognition result, automatic classification of unknown abnormalities is achieved. This recognition method based on feature similarity can effectively map unknown abnormalities to known abnormal types by matching features with the labeled samples. At the same time, by setting a preset similarity threshold, the confidence level of recognition can be controlled, improving the reliability of classification. This method makes full use of prior knowledge, constructs a feature sample library, and transforms expert experience and historical data into a reference model available for reasoning, greatly improving the efficiency and accuracy of abnormal state recognition. At the same time, as the sample library is continuously improved and updated, the recognition performance will also continue to improve, having good scalability.

[0018] Optionally, calculating the feature similarity between the target feature of the target point cloud cluster and each of the feature samples specifically includes: respectively calculating the Euclidean distances between the shape feature, size feature, and position feature of the target point cloud cluster and the shape feature, size feature, and position feature of each of the feature samples to obtain the shape distance, size distance, and position distance under each abnormal state type; performing weighted summation on the shape distance, size distance, and position distance under each abnormal state type to obtain the feature similarity between the target feature of the target point cloud cluster and each feature sample.

[0019] By adopting the above technical solutions, by respectively calculating the Euclidean distances between the shape features, size features, and position features of the target point cloud cluster and the feature samples, the shape distances, size distances, and position distances in different abnormal state types can be obtained, which can respectively measure the proximity degrees of the target point cloud cluster and each sample in different feature dimensions. By performing a weighted sum of these three types of distances, a comprehensive feature similarity can be obtained, and based on considering the importance of different features, the overall similarity degree between the target point cloud cluster and each sample can be evaluated. This similarity measurement method based on weighted Euclidean distance is simple and intuitive, with high calculation efficiency, and can quickly compare the similarity relationships between the target point cloud cluster and a large number of samples. By adjusting the weights of different features, the contributions of each feature to the similarity calculation can be flexibly balanced to adapt to different application requirements.

[0020] In the second aspect of the present application, a sound barrier nut state detection device applied to a nut disassembly robot is provided. The device is the nut disassembly robot, and the nut disassembly robot includes an acquisition module and a processing module, where: the acquisition module is used to acquire point cloud data of the area where the sound barrier nut is located, and the point cloud data includes three-dimensional coordinate information of the sound barrier nut; the processing module is used to construct a three-dimensional nut model based on the point cloud data, and perform a comparison calculation between the three-dimensional nut model and a standard nut model to obtain a distance field deviation between the three-dimensional nut model and the standard nut model, and the distance field deviation includes deviation values of each area on the surface of the sound barrier nut; the processing module is further used to determine abnormal deviation values from the deviation values, and determine the state of the sound barrier nut according to the abnormal deviation values, and the state includes a normal state and an abnormal state; the processing module is further used to divide the target point cloud corresponding to the abnormal deviation value into multiple target point cloud clusters if it is determined that the sound barrier nut is in an abnormal state; the processing module is further used to extract target features of each of the target point cloud clusters, and the target features include shape features, size features, and position features; the processing module is further used to identify the abnormal state type corresponding to the abnormal state according to the target features, and the abnormal state type includes dirt on the nut surface, nut wear, and nut corrosion.

[0021] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, both the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method described in any one of the above.

[0022] In the fourth aspect of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores instructions, and when the instructions are executed, the method described in any one of the above is executed.

[0023] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. By obtaining the point cloud data of the area where the sound barrier nut is located, constructing a three-dimensional model of the nut, comparing it with the standard nut model, and calculating the distance field deviation, the differences in each area of the nut surface can be accurately captured. Determine the abnormal deviation value according to the deviation value, and judge whether the nut state is abnormal, realizing the automatic detection and abnormal recognition of the nut state. If the nut is in an abnormal state, divide the target point cloud corresponding to the abnormal deviation value into multiple point cloud clusters, extract the shape, size, and position characteristics of each point cloud cluster, and identify the specific abnormal state type according to these characteristics, such as dirt, wear, or rust on the nut surface. This method realizes the refined detection and diagnosis of the sound barrier nut state through point cloud data analysis and feature extraction, and can accurately discover and locate various defects and abnormal conditions on the nut surface. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a schematic flowchart of a method for detecting the state of a sound barrier nut applied to a nut disassembly robot disclosed in an embodiment of the present application; Figure 2 is a schematic block diagram of a device for detecting the state of a sound barrier nut applied to a nut disassembly robot disclosed in an embodiment of the present application; Figure 3 is a schematic structural diagram of an electronic device disclosed in an embodiment of the present application.

[0025] Description of the reference numerals: 201, acquisition module; 202, processing module; 300, electronic device; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments.

[0027] In the description of the embodiments of the present application, words such as "for example" or "for instance" are used to represent examples, illustrations, or explanations. Any embodiment or design solution described as "for example" or "for instance" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, using words such as "for example" or "for instance" is intended to present relevant concepts in a specific manner.

[0028] In the description of the embodiments of the present application, the term "a plurality" means two or more. For example, a plurality of systems means two or more systems, and a plurality of screen terminals means two or more screen terminals. In addition, the terms "first" and "second" are used only for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. The terms "comprise", "include", "have" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0029] The present application provides a method for detecting the state of a sound barrier nut applied to a nut disassembly robot. Refer to Figure 1 , Figure 1 FIG. is a schematic flowchart of a method for detecting the state of a sound barrier nut applied to a nut disassembly robot provided by an embodiment of the present application. This method is applied to a nut disassembly robot, which is used to execute a sound barrier nut state detection program. The nut disassembly robot can communicate with a user device through a wired or wireless network. This method includes steps S101 to S106, and the above steps are as follows: Step S101: Obtain point cloud data of the area where the sound barrier nut is located. The point cloud data includes three-dimensional coordinate information of the sound barrier nut.

[0030] In step S101, the nut disassembly robot obtains the point cloud data of the area where the sound barrier nut is located through a three-dimensional scanning device equipped on itself. The three-dimensional scanning device can be a device such as a laser scanner, a structured light scanner, or a depth camera, and is used to collect three-dimensional space information on the surface of the sound barrier nut.

[0031] Specifically, the nut disassembly robot first controls the robotic arm to move above the area where the sound barrier nut is located according to the approximate position information of the sound barrier nut input in advance. Then, the three-dimensional scanning device at the end of the robotic arm scans the sound barrier nut and its surrounding area. During the scanning process, the three-dimensional scanning device emits a laser beam or structured light, and calculates the three-dimensional coordinate information of each position point on the surface of the sound barrier nut by measuring the time or pattern change of the laser beam or structured light reflected back to the device, so as to obtain the original point cloud data of the area where the sound barrier nut is located.

[0032] In order to obtain complete three-dimensional information of the sound barrier nut, the nut disassembly robot can scan the sound barrier nut multiple times from different angles. For example, the nut disassembly robot can control the robotic arm to rotate around the sound barrier nut and perform a scan every certain angle until all surfaces of the sound barrier nut are covered.

[0033] In addition, since the original point cloud data may contain abnormal data such as noise points and outliers, which affect the accuracy of subsequent processing. Therefore, the nut disassembly robot also needs to preprocess the original point cloud data to remove the abnormal point cloud data therein and improve the point cloud quality. Point cloud preprocessing methods include statistical filtering, voxel filtering, radius filtering, etc., and appropriate methods can be selected according to the actual situation.

[0034] After the above scanning and preprocessing steps, the nut disassembly robot finally obtains the point cloud data containing the three-dimensional coordinate information of the sound barrier nut, providing a data basis for subsequent nut state analysis.

[0035] Step S102: Based on the point cloud data, construct a three-dimensional model of the nut, and compare and calculate the three-dimensional model of the nut with the standard nut model to obtain the distance field deviation between the three-dimensional model of the nut and the standard nut model. The distance field deviation includes the deviation values of each region on the surface of the sound barrier nut.

[0036] In step S102, based on the point cloud data, construct a three-dimensional model of the nut, and compare and calculate the three-dimensional model of the nut with the standard nut model to obtain the distance field deviation between the three-dimensional model of the nut and the standard nut model, specifically including: performing triangular mesh division on the point cloud data to construct a three-dimensional model of the nut, and the three-dimensional model of the nut is composed of multiple triangular mesh patches; registering the three-dimensional model of the nut with the standard nut model so that the three-dimensional model of the nut and the standard nut model are located in the same coordinate system; calculating the shortest distance from each grid point on the three-dimensional model of the nut to the surface of the standard nut model to generate the distance field deviation.

[0037] Specifically, the nut disassembly robot performs triangular mesh division on the point cloud data to convert the discrete point cloud into a continuous three-dimensional surface model. Triangular mesh division algorithms include Delaunay triangulation, moving least squares method, etc. Through triangular mesh division, the point cloud data is connected into a mesh model composed of multiple triangular patches, and each patch is defined by three vertices and the corresponding normal vector. The triangular mesh model can better represent the surface shape and topological structure of the sound barrier nut, facilitating subsequent analysis.

[0038] Next, the nut disassembly robot loads the CAD model corresponding to the current type of sound barrier nut from the standard nut model library as the reference standard for comparison. To perform an accurate comparison, the three-dimensional nut model constructed needs to be registered with the standard nut model so that they are in the same coordinate system. The registration process can be achieved through rigid body transformation (translation + rotation), and the registration algorithms include the ICP (Iterative Closest Point) algorithm and the NDT (Normal Distributions Transform) algorithm, etc. Through iterative optimization, an optimal transformation matrix is found to make the three-dimensional nut model coincide with the standard nut model as much as possible.

[0039] After registration, the nut disassembly robot calculates the shortest distance from each grid point on the three-dimensional nut model to the surface of the standard nut model to generate a distance field deviation. The distance field deviation reflects the deformation degree of the three-dimensional nut model relative to the standard model. The larger the deviation, the more likely there are defects or abnormalities in the local area of the nut surface. To quantify the deviation, the distance values can be mapped to a color scale to generate an intuitive visualization result. For example, a red-yellow-green color mapping scheme can be used, where red represents areas with larger deviations and green represents areas with smaller deviations.

[0040] It should be noted that due to errors in the actual production and use processes, there will always be a certain deviation between the three-dimensional nut model and the standard model. Therefore, a reasonable deviation threshold needs to be set, and the areas exceeding this threshold are regarded as abnormal and require key attention. The selection of the deviation threshold needs to comprehensively consider factors such as the material of the nut, precision requirements, and operating conditions, and usually can be determined by empirical values or statistical analysis methods.

[0041] Step S103: Determine the abnormal deviation values from the deviation values and determine the status of the sound barrier nut according to the abnormal deviation values. The status includes the normal status and the abnormal status.

[0042] In step S103, determining the abnormal deviation values from the deviation values and determining the status of the sound barrier nut specifically includes: calculating the mean and variance of the distance field deviation, and determining the distance field deviation threshold according to the mean and variance; determining the deviation values greater than or equal to the distance field deviation threshold in the distance field deviation as the abnormal deviation values; counting the number of abnormal deviation values; if the number of abnormal deviation values is greater than or equal to the preset abnormal threshold, then determine the status of the sound barrier nut as the abnormal status; if the number of abnormal deviation values is less than the preset abnormal threshold, then determine the status of the sound barrier nut as the normal status.

[0043] Specifically, the nut removal robot performs statistical analysis on the distance field deviation and calculates its mean and variance. The mean reflects the average level of the overall nut deviation, and the variance reflects the degree of dispersion of the deviation value. Based on the mean and variance, a distance field deviation threshold can be determined as a criterion for distinguishing normal deviation from abnormal deviation. Generally speaking, the distance field deviation threshold can be set in the form of "mean + k times standard deviation", where k is an empirical coefficient that can be adjusted according to actual needs. For example, when k=2, the deviation threshold is the mean plus 2 times the standard deviation.

[0044] After determining the distance field deviation threshold, the nut removal robot marks the deviation values in the distance field deviation that are greater than or equal to the threshold as abnormal deviation values. These abnormal deviation values correspond to local areas on the nut surface, which may have defects, damage or other problems and need to be paid attention to. The nut removal robot will count the number of abnormal deviation values for subsequent status determination.

[0045] Next, the nut removal robot compares the number of abnormal deviation values with the preset abnormal threshold to determine the final state of the sound barrier nut. The preset abnormal threshold indicates the maximum number of abnormal deviations allowed. If the threshold is exceeded, the nut state is considered abnormal. The setting of the preset abnormal threshold requires a balance between the sensitivity and false alarm rate of detection. Usually, a reasonable value can be determined based on historical data and expert experience. For example, the preset abnormal threshold can be set to 1% of the total number of grid points on the nut surface, that is, when the number of abnormal deviation values exceeds 1% of the total number of grid points, the nut state is considered abnormal.

[0046] If the number of abnormal deviation values is greater than or equal to the preset abnormal threshold, the nut removal robot will determine the state of the sound barrier nut as abnormal. This means that there are many defects or abnormal areas on the surface of the nut, which may not meet the use requirements and require further inspection and processing. On the contrary, if the number of abnormal deviation values is less than the preset abnormal threshold, the overall state of the nut is considered to be good and is determined to be normal.

[0047] Step S104: If it is determined that the sound barrier nut is in an abnormal state, the target point cloud corresponding to the abnormal deviation value is divided into multiple target point cloud clusters.

[0048] In step S104, the target point cloud corresponding to the abnormal deviation value is divided into multiple target point cloud clusters, which specifically includes: randomly determining a preset number of target point clouds from multiple target point clouds as the initial clustering centers; calculating the first distance from each target point cloud to the first clustering center and the second distance from each target point cloud to the second clustering center, where the first clustering center and the second clustering center are any two initial clustering centers among the multiple initial clustering centers; if it is determined that the first distance is less than or equal to the second distance, then dividing the target point cloud into the point cloud cluster where the first clustering center is located; for each point cloud cluster, calculating the geometric center of the data points included in each point cloud cluster; determining the geometric center as the new clustering center and performing iteration based on the new clustering center until the geometric centers of each point cloud cluster no longer change, obtaining a preset number of target point cloud clusters.

[0049] Specifically, when the nut disassembly robot determines that the sound barrier nut is in an abnormal state, it will perform clustering segmentation on the target point cloud corresponding to the abnormal deviation value and divide it into multiple target point cloud clusters. First, the nut disassembly robot randomly selects a preset number (usually k) of point clouds from the target point cloud corresponding to the abnormal deviation value as the initial clustering centers. These k initial clustering centers represent the initial positions of k point cloud clusters to be generated, and subsequent point clouds will be classified based on this. The selection of the initial clustering centers will affect the clustering effect, so the method of randomly selecting multiple times and comparing the results can be used to obtain a better initial value.

[0050] Next, the nut disassembly robot calculates the Euclidean distance from each target point cloud to each initial clustering center and determines the point cloud cluster to which it belongs according to the distance. Specifically, for any two initial clustering centers, the distance from the target point cloud to them is calculated respectively, and the magnitudes of these two distances are compared. If the first distance from the target point cloud to the first clustering center is less than or equal to the second distance to the second clustering center, then the target point cloud is divided into the point cloud cluster where the first clustering center is located. Repeat this process until all target point clouds are assigned to the corresponding point cloud clusters.

[0051] After completing one round of point cloud assignment, the nut disassembly robot calculates the geometric center of each point cloud cluster. The geometric center represents the average position of all data points in the point cloud cluster and can be approximated by the arithmetic mean of the coordinates of the internal data points of the point cloud cluster. The calculated geometric center is regarded as the new clustering center, representing the central position of the point cloud cluster.

[0052] Then, the nut disassembly robot repeats the above process using the updated cluster centers, calculates the distance from each target point cloud to the new cluster centers again, and reassigns the ownership relationship of the point clouds according to the distances. Through continuous iteration, the cluster centers of the point cloud clusters gradually approach the true center of the data distribution, and the result of the point cloud division becomes more accurate. The iteration process continues until the geometric centers of the point cloud clusters no longer change significantly or the preset maximum number of iterations is reached.

[0053] Finally, the nut disassembly robot obtains the target point cloud clusters with the preset number k, and each point cloud cluster represents an abnormal area on the surface of the sound barrier nut. Through clustering segmentation, the abnormal point clouds are automatically classified into different clusters, facilitating subsequent feature extraction and defect identification.

[0054] Step S105: Extract the target features of each target point cloud cluster, where the target features include shape features, size features, and position features.

[0055] In step S105, extracting the target features of each target point cloud cluster specifically includes: calculating the three-dimensional bounding box of each target point cloud cluster, and extracting the length, width, and height of the three-dimensional bounding box as the size features of each target point cloud cluster; calculating the second-order moment invariant of each target point cloud cluster, and extracting the eigenvalue of the second-order moment invariant as the shape feature of each target point cloud cluster; calculating the geometric center coordinates of each target point cloud cluster, and extracting the geometric center coordinates as the position feature of each target point cloud cluster.

[0056] Specifically, the nut disassembly robot calculates the three-dimensional bounding box of each target point cloud cluster. The three-dimensional bounding box is the smallest cuboid that can completely enclose the point cloud cluster, and its edges are parallel to the coordinate axes. By finding the maximum and minimum values of the point cloud cluster on the three coordinate axes, the vertex coordinates of the bounding box can be determined, and then the length, width, and height of the bounding box can be calculated. These three dimensional parameters serve as the size features of the point cloud cluster, reflecting the spatial span of the abnormal area in the three directions. For example, a bounding box with a length of 5 mm, a width of 3 mm, and a height of 2 mm indicates that the abnormal area corresponding to the point cloud cluster has a larger span in the length direction and a smaller span in the height direction.

[0057] Next, the nut removal robot calculates the second-order moment invariant of each target point cloud cluster. The second-order moment invariant is a characteristic quantity that describes the shape of the point cloud. It remains invariant to transformations such as translation, rotation, and scaling, so it can effectively characterize the geometric characteristics of the point cloud. To calculate the second-order moment invariant, it is necessary to first construct the covariance matrix of the point cloud, which reflects the distribution of the point cloud on the three coordinate axes. Then the covariance matrix is decomposed by eigenvalue to obtain three eigenvalues and corresponding eigenvectors. The combination of these three eigenvalues can be used as the shape feature of the point cloud cluster, where a larger eigenvalue indicates that the point cloud has a larger degree of extension in the direction of the eigenvector, while a smaller eigenvalue indicates a smaller degree of extension. By comparing the size relationship of the three eigenvalues, the shape type of the point cloud cluster can be determined, such as spherical, flat, slender, etc.

[0058] Finally, the nut removal robot calculates the geometric center coordinates of each target point cloud cluster. The geometric center is the average coordinate of all points in the point cloud cluster, representing the position of the point cloud cluster in three-dimensional space. The three-dimensional coordinates of the geometric center can be obtained by calculating the arithmetic mean of the X, Y, and Z coordinates of all points inside the point cloud cluster. Using this coordinate as the position feature of the point cloud cluster, the approximate position of the abnormal area on the nut surface can be determined, such as the top, side, or bottom of the nut. Combining position features with other features, the type and cause of the anomaly can be more accurately inferred.

[0059] For example, suppose that the nut removal robot extracts the following features of a target point cloud cluster: the bounding box size is (8mm, 6mm, 3mm), the shape feature is (0.8, 0.15, 0.05), and the position feature is (0, 20mm, 50mm). This indicates that the abnormal area has a large span in length and width, but a small height, is flat, and is located in the middle and upper part of the nut. Combining these features, it can be preliminarily determined that the abnormality may be wear or depression on the surface of the nut. By comparing the feature library of known abnormal types, the final abnormal type recognition result can be given.

[0060] Step S106: Identify the abnormal state type corresponding to the abnormal state according to the target feature, and the abnormal state type includes nut surface dirt, nut wear and nut rust.

[0061] In step S106, according to the target feature, identify the abnormal state type corresponding to the abnormal state, specifically including: constructing an abnormal state feature sample library, which includes multiple feature samples with labeled abnormal state types, and each feature sample includes shape features, size features, and position features corresponding to the abnormal state type; calculating the feature similarity between the target feature of the target point cloud cluster and each feature sample; if it is determined that the target feature similarity is greater than or equal to the preset similarity threshold, then determine that the abnormal state type corresponding to the target point cloud cluster is the abnormal state type corresponding to the target feature sample, the target feature similarity is the feature similarity between the target point cloud cluster and the target feature sample, and the target feature sample is any one of the multiple feature samples.

[0062] Specifically, the nut disassembly robot uses the extracted target feature to identify the specific type of abnormal state, such as dirt on the nut surface, nut wear, or nut corrosion, by performing similarity matching with the samples in the abnormal state feature sample library. This process utilizes the supervised learning idea in machine learning to achieve the classification and identification of unknown abnormalities through the pre-constructed abnormal state feature sample library.

[0063] First, the nut disassembly robot needs to construct an abnormal state feature sample library. This sample library contains a large number of abnormal state samples of known types, and each sample has been manually labeled to determine its corresponding abnormal state type, such as dirt on the nut surface, nut wear, or nut corrosion. At the same time, the same features as in step S105, namely shape features, size features, and position features, are extracted from each sample. These features, together with the abnormal type labels, form complete feature samples. The more samples in the sample library and the more comprehensive the covered abnormal types, the more accurate the subsequent abnormal identification will be.

[0064] During actual identification, the nut disassembly robot calculates the similarity between the target feature of the target point cloud cluster extracted in step S105 and each feature sample in the abnormal state feature sample library. The similarity calculation can use common measurement methods such as Euclidean distance and cosine similarity to measure the closeness between two sets of feature vectors. During the calculation process, the shape features, size features, and position features of the target point cloud cluster are respectively compared with the corresponding features of the sample to obtain similarity scores in three aspects. Then, these three scores are weighted and averaged according to a certain weight to obtain the comprehensive feature similarity. The setting of the weight can be determined according to the contribution of different features to the judgment of the abnormal type. Usually, the weights of the shape features and size features are relatively high, while the weight of the position feature is relatively low.

[0065] Next, the nut disassembly robot compares the similarity between the calculated target point cloud cluster and each feature sample with a preset similarity threshold. The preset similarity threshold is an empirical value representing the minimum similarity degree at which two sets of features are considered to be of the same type of anomaly. If the similarity between the target point cloud cluster and a certain feature sample is greater than or equal to this threshold, it is considered that the type of the abnormal state corresponding to this point cloud cluster is the same as the type of this sample. In other words, the target point cloud cluster is classified into the abnormal category that is closest to its features.

[0066] Finally, after the nut disassembly robot completes the identification of the abnormal state types of all target point cloud clusters, it can give a complete description of the abnormal conditions of the sound barrier nuts. For example, the identification result shows that there are 3 abnormal areas on the nut surface, among which 2 are dirt on the nut surface and 1 is nut wear, and the worn area is located at the top of the nut. Such an identification result can provide a decision-making basis for subsequent nut repair and replacement.

[0067] For example, assume there are the following three feature samples in the sample library: Sample A (dirt on the nut surface): The shape feature is (0.2, 0.7, 0.1), the size feature is (5mm, 4mm, 1mm), and the position feature is (10mm, 10mm, 60mm); Sample B (nut wear): The shape feature is (0.6, 0.3, 0.1), the size feature is (8mm, 6mm, 2mm), and the position feature is (0, 20mm, 50mm); Sample C (nut rust): The shape feature is (0.4, 0.4, 0.2), the size feature is (10mm, 8mm, 4mm), and the position feature is (15mm, 15mm, 30mm). Given a target point cloud cluster with the following features: The shape feature is (0.7, 0.2, 0.1), the size feature is (6mm, 5mm, 2mm), and the position feature is (5mm, 15mm, 55mm). The calculated similarities are: the similarity with Sample A is 0.85, the similarity with Sample B is 0.92, and the similarity with Sample C is 0.78. Assume the preset similarity threshold is set to 0.9, then this target point cloud cluster will be identified as nut wear because its similarity with Sample B is the highest and exceeds the preset similarity threshold. Through this method of matching based on the sample library, the nut disassembly robot can automatically identify the type of the unknown abnormal state and achieve intelligent abnormal detection and classification.

[0068] In a possible implementation, calculating the feature similarity between the target feature of the target point cloud cluster and each feature sample specifically includes: respectively calculating the Euclidean distances between the shape feature, size feature, and position feature of the target point cloud cluster and the shape feature, size feature, and position feature of each feature sample to obtain the shape distance, size distance, and position distance under each abnormal state type; performing a weighted sum on the shape distance, size distance, and position distance under each abnormal state type to obtain the feature similarity between the target feature of the target point cloud cluster and each feature sample.

[0069] Specifically, the nut disassembly robot first calculates the Euclidean distances between the shape feature, size feature, and position feature of the target point cloud cluster and the corresponding features of each feature sample in the sample library. The Euclidean distance is used to measure the difference between two feature vectors. For the shape feature, the shape feature vector of the target point cloud cluster can be subtracted from the shape feature vector of the sample, and then the norm of the difference vector is calculated to obtain the shape distance. Similarly, for the size feature and position feature, the Euclidean distances between them and the sample features are also calculated respectively to obtain the size distance and position distance. In this way, for each feature sample, three distance values can be obtained, respectively representing the degrees of difference between the target point cloud cluster and the sample in terms of shape, size, and position.

[0070] It should be noted that since the dimensions and numerical ranges of the shape, size, and position features may be different, directly calculating the Euclidean distance may cause the influence of a certain feature to be amplified or reduced. To balance the contributions of different features, the feature vectors can be normalized before calculating the distance to map them to the same scale. The normalization methods include min-max normalization, zero-mean unit-variance normalization, etc.

[0071] Next, the nut disassembly robot performs a weighted sum on the three distance values obtained for each sample to obtain the comprehensive similarity between the target point cloud cluster and the sample. The formula for the weighted sum can be expressed as: Comprehensive similarity = w1 * shape distance + w2 * size distance + w3 * position distance Among them, w1, w2, and w3 are the weight coefficients of the shape distance, size distance, and position distance respectively, which are used to control the importance of different features in the similarity calculation. These weight coefficients can be set according to experience or the results of data analysis, or can be automatically optimized through machine learning methods. Generally speaking, the shape and size features have a greater impact on the judgment of abnormal types, so their weight coefficients usually take larger values, while the influence of the position feature is relatively small, and the weight coefficient can take a smaller value. The value range of the weight coefficients is usually between 0 and 1, and the sum of the three weight coefficients is 1. This application does not limit the specific values of the weight coefficients.

[0072] Through weighted summation, the nut disassembly robot obtains the comprehensive similarity between the target point cloud cluster and each feature sample. The greater the similarity, the closer the features of the target point cloud cluster are to those of the sample, and the greater the possibility of the corresponding abnormal type. Finally, the feature sample with the largest similarity is found, and the corresponding abnormal state type is considered to be the abnormal type of the target point cloud cluster.

[0073] Referring to Figure 2 , this application also provides a sound barrier nut state detection device applied to a nut disassembly robot. The device is a nut disassembly robot, which includes an acquisition module 201 and a processing module 202, where: The acquisition module 201 is used to acquire point cloud data in the area where the sound barrier nut is located, and the point cloud data includes the three-dimensional coordinate information of the sound barrier nut; The processing module 202 is used to construct a three-dimensional nut model based on the point cloud data, and perform a comparison calculation between the three-dimensional nut model and the standard nut model to obtain the distance field deviation between the three-dimensional nut model and the standard nut model. The distance field deviation includes the deviation values of each area on the surface of the sound barrier nut; The processing module 202 is further used to determine the abnormal deviation value from the deviation values, and determine the state of the sound barrier nut according to the abnormal deviation value. The state includes a normal state and an abnormal state; The processing module 202 is further used to divide the target point cloud corresponding to the abnormal deviation value into multiple target point cloud clusters if it is determined that the sound barrier nut is in an abnormal state; The processing module 202 is further used to extract the target features of each target point cloud cluster. The target features include shape features, size features, and position features; The processing module 202 is further used to identify the abnormal state type corresponding to the abnormal state according to the target features. The abnormal state types include dirt on the nut surface, nut wear, and nut rust.

[0074] In a possible implementation manner, the processing module 202 constructs a three-dimensional nut model based on the point cloud data, and performs a comparison calculation between the three-dimensional nut model and the standard nut model to obtain the distance field deviation between the three-dimensional nut model and the standard nut model, specifically including: The processing module 202 performs triangular mesh division on the point cloud data to construct a three-dimensional nut model, and the three-dimensional nut model is composed of multiple triangular mesh patches; The processing module 202 registers the three-dimensional nut model with the standard nut model so that the three-dimensional nut model and the standard nut model are located in the same coordinate system; The processing module 202 calculates the shortest distance from each grid point on the three-dimensional nut model to the surface of the standard nut model to generate the distance field deviation.

[0075] In a possible implementation, the processing module 202 determines abnormal deviation values from the deviation values and determines the state of the sound barrier nuts according to the abnormal deviation values, specifically including: the processing module 202 calculates the mean and variance of the distance field deviation, and determines the distance field deviation threshold according to the mean and variance; the processing module 202 determines the deviation values greater than or equal to the distance field deviation threshold in the distance field deviation as abnormal deviation values; the processing module 202 counts the number of abnormal deviation values; if the number of abnormal deviation values is greater than or equal to the preset abnormal threshold, then the processing module 202 determines that the state of the sound barrier nuts is an abnormal state; if the number of abnormal deviation values is less than the preset abnormal threshold, then the processing module 202 determines that the state of the sound barrier nuts is a normal state.

[0076] In a possible implementation, the processing module 202 divides the target point clouds corresponding to the abnormal deviation values into multiple target point cloud clusters, specifically including: the processing module 202 randomly determines a preset number of target point clouds from the multiple target point clouds as the initial clustering centers; the processing module 202 calculates the first distance between each target point cloud and the first clustering center and the second distance between each target point cloud and the second clustering center, where the first clustering center and the second clustering center are any two initial clustering centers among the multiple initial clustering centers; if it is determined that the first distance is less than or equal to the second distance, then the processing module 202 divides the target point cloud into the point cloud cluster where the first clustering center is located; the processing module 202 calculates the geometric center of the data points included in each point cloud cluster for each point cloud cluster; the processing module 202 determines the geometric center as the new clustering center and performs iteration according to the new clustering center until the geometric centers of each point cloud cluster no longer change, obtaining a preset number of target point cloud clusters.

[0077] In a possible implementation, the processing module 202 extracts the target features of each target point cloud cluster, specifically including: the processing module 202 calculates the three-dimensional bounding box of each target point cloud cluster and extracts the length, width, and height of the three-dimensional bounding box as the size features of each target point cloud cluster; the processing module 202 calculates the second-order moment invariant of each target point cloud cluster and extracts the eigenvalue of the second-order moment invariant as the shape feature of each target point cloud cluster; the processing module 202 calculates the geometric center coordinates of each target point cloud cluster and extracts the geometric center coordinates as the position feature of each target point cloud cluster.

[0078] In a possible implementation, the processing module 202 identifies the abnormal state type corresponding to the abnormal state according to the target feature, specifically including: the processing module 202 constructs an abnormal state feature sample library, which includes multiple feature samples with labeled abnormal state types, and each feature sample includes shape features, size features, and position features corresponding to the abnormal state type; the processing module 202 calculates the feature similarity between the target feature of the target point cloud cluster and each feature sample; if the processing module 202 determines that the target feature similarity is greater than or equal to the preset similarity threshold, it determines that the abnormal state type corresponding to the target point cloud cluster is the abnormal state type corresponding to the target feature sample, the target feature similarity is the feature similarity between the target point cloud cluster and the target feature sample, and the target feature sample is any one of the multiple feature samples.

[0079] In a possible implementation, the processing module 202 calculates the feature similarity between the target feature of the target point cloud cluster and each feature sample, specifically including: the processing module 202 calculates the Euclidean distances between the shape feature, size feature, and position feature of the target point cloud cluster and the shape feature, size feature, and position feature of each feature sample respectively, to obtain the shape distance, size distance, and position distance under each abnormal state type; the processing module 202 performs weighted summation on the shape distance, size distance, and position distance under each abnormal state type to obtain the feature similarity between the target feature of the target point cloud cluster and each feature sample.

[0080] It should be noted that: when the device provided in the above embodiment realizes its functions, only the above-mentioned division of each functional module is used for illustration. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0081] This application also provides an electronic device. Refer to Figure 3 , Figure 3 is a schematic structural diagram of an electronic device provided in an embodiment of this application. The electronic device 300 may include: at least one processor 301, at least one network interface 304, a user interface 303, a memory 305, and at least one communication bus 302.

[0082] Among them, the communication bus 302 is used to realize the connection and communication between these components.

[0083] Among them, the user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may further include a standard wired interface and a wireless interface.

[0084] Among them, the network interface 304 may optionally include a standard wired interface, a wireless interface (such as a Wi-Fi interface).

[0085] Among them, the processor 301 may include one or more processing cores. The processor 301 connects various parts within the entire server through various interfaces and lines. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 305, and by calling the data stored in the memory 305, it executes various functions of the server and processes data. Optionally, the processor 301 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 301 may integrate a combination of one or several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301 and may be implemented separately by a single chip.

[0086] Among them, the memory 305 may include random access memory (RAM) and may also include read-only memory. Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area may store the data involved in the above-mentioned method embodiments. The memory 305 is optionally also at least one storage device located far from the aforementioned processor 301. Refer to Figure 3In the memory 305, which is a computer storage medium, an operating system, a network communication module, a user interface module, and an application program for the sound barrier nut state detection method applied to the nut disassembling robot can be included.

[0087] In Figure 3 In the electronic device 300 shown, the user interface 303 is mainly used to provide an interface for the user to obtain the data input by the user; and the processor 301 can be used to call the application program stored in the memory 305 for the sound barrier nut state detection method applied to the nut disassembling robot. When executed by one or more processors 301, the electronic device 300 is caused to execute one or more of the methods as described in the above embodiments. It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0088] The present application also provides a computer-readable storage medium storing instructions. When executed by one or more processors 301, the electronic device 300 is caused to execute one or more of the methods as described in the above embodiments.

[0089] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0090] In several implementation manners provided by the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some service interfaces. The indirect couplings or communication connections of the devices or units can be in an electrical or other form.

[0091] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0092] In addition, in each embodiment of the present application, the functional units may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0093] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable memory. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product. The computer software product is stored in a memory and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in the various embodiments of the present application. The aforementioned memory includes: various media such as USB flash drives, mobile hard disks, magnetic disks, or optical discs that can store program codes.

[0094] The above are only exemplary embodiments of the present disclosure and should not be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure still fall within the scope covered by the present disclosure. After considering the specification and the practice of the present disclosure, those skilled in the art will readily think of other embodiments of the present disclosure.

[0095] The present application aims to cover any variations, uses, or adaptive changes of the present disclosure that follow the general principles of the present disclosure and include common general knowledge or conventional technical means in the technical field not recorded in the present disclosure. The specification and the embodiments are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for detecting the state of a sound barrier nut applied to a nut disassembly robot, characterized in that, The method is applied to a nut disassembly robot, and the method includes: Obtain point cloud data of the area where the sound barrier nut is located, and the point cloud data includes three-dimensional coordinate information of the sound barrier nut; Based on the point cloud data, construct a three-dimensional nut model, and perform comparison calculations between the three-dimensional nut model and a standard nut model to obtain a distance field deviation between the three-dimensional nut model and the standard nut model, where the distance field deviation includes deviation values of each area on the surface of the sound barrier nut; Determine abnormal deviation values from the deviation values, and determine the state of the sound barrier nut according to the abnormal deviation values, where the state includes a normal state and an abnormal state; If it is determined that the sound barrier nut is in an abnormal state, divide the target point cloud corresponding to the abnormal deviation value into multiple target point cloud clusters; Extract target features of each of the target point cloud clusters, where the target features include shape features, size features, and position features; According to the target features, identify the abnormal state type corresponding to the abnormal state, where the abnormal state type includes dirt on the nut surface, nut wear, and nut rust; 2. The method according to claim 1, wherein The constructing a three-dimensional nut model based on the point cloud data, and performing comparison calculations between the three-dimensional nut model and a standard nut model to obtain a distance field deviation between the three-dimensional nut model and the standard nut model specifically includes: Perform triangular mesh division on the point cloud data to construct the three-dimensional nut model, and the three-dimensional nut model is composed of multiple triangular mesh patches; Register the three-dimensional nut model with the standard nut model so that the three-dimensional nut model and the standard nut model are in the same coordinate system; Calculate the shortest distance from each grid point on the three-dimensional nut model to the surface of the standard nut model to generate a distance field deviation; 3. The method according to claim 1, wherein The determining abnormal deviation values from the deviation values, and determining the state of the sound barrier nut according to the abnormal deviation values specifically includes: Calculate the mean and variance of the distance field deviation, and determine a distance field deviation threshold according to the mean and the variance; Determine the deviation values greater than or equal to the distance field deviation threshold in the distance field deviation as abnormal deviation values; Count the number of the abnormal deviation values; If the number of the abnormal deviation values is greater than or equal to a preset abnormal threshold, determine that the state of the sound barrier nut is an abnormal state; If the number of the abnormal deviation values is less than the preset abnormal threshold, determine that the state of the sound barrier nut is a normal state; 4. The method according to claim 1, wherein The dividing the target point cloud corresponding to the abnormal deviation value into multiple target point cloud clusters specifically includes: Randomly determine a preset number of target point clouds from multiple target point clouds as initial clustering centers; Calculate a first distance from each target point cloud to a first clustering center and a second distance from each target point cloud to a second clustering center, where the first clustering center and the second clustering center are any two initial clustering centers among the multiple initial clustering centers; If it is determined that the first distance is less than or equal to the second distance, divide the target point cloud into the point cloud cluster where the first clustering center is located; For each of the point cloud clusters, calculate the geometric center of the data points included in each of the point cloud clusters; Determine the geometric center as a new clustering center, and perform iteration based on the new clustering center until the geometric centers of all the point cloud clusters no longer change, to obtain the target point cloud clusters of the preset quantity.

5. The method according to claim 1, wherein The extracting of the target features of each of the target point cloud clusters specifically includes: Calculate the three-dimensional bounding box of each of the target point cloud clusters, and extract the length, width, and height of the three-dimensional bounding box as the size features of each of the target point cloud clusters; Calculate the second-order moment invariant of each of the target point cloud clusters, and extract the eigenvalue of the second-order moment invariant as the shape feature of each of the target point cloud clusters; Calculate the geometric center coordinates of each of the target point cloud clusters, and extract the geometric center coordinates as the position features of each of the target point cloud clusters.

6. The method according to claim 1, wherein The identifying of the abnormal state type corresponding to the abnormal state according to the target features specifically includes: Construct an abnormal state feature sample library, where the abnormal state feature sample library includes a plurality of feature samples with labeled abnormal state types, and each feature sample includes shape features, size features, and position features corresponding to the abnormal state type; Calculate the feature similarity between the target features of the target point cloud cluster and each of the feature samples; If it is determined that the target feature similarity is greater than or equal to a preset similarity threshold, determine that the abnormal state type corresponding to the target point cloud cluster is the abnormal state type corresponding to the target feature sample, where the target feature similarity is the feature similarity between the target point cloud cluster and the target feature sample, and the target feature sample is any one of the plurality of feature samples.

7. The method according to claim 6, characterized in that, The calculating of the feature similarity between the target features of the target point cloud cluster and each of the feature samples specifically includes: Calculate the Euclidean distances between the shape feature, size feature, and position feature of the target point cloud cluster and the shape feature, size feature, and position feature of each of the feature samples respectively, to obtain the shape distance, size distance, and position distance under each abnormal state type; Perform weighted summation on the shape distance, size distance, and position distance under each abnormal state type to obtain the feature similarity between the target features of the target point cloud cluster and each of the feature samples.

8. An acoustic barrier nut state detection device applied to a nut disassembly robot, characterized in that, The device is a nut disassembly robot, and the nut disassembly robot includes an acquisition module (201) and a processing module (202), where: The acquisition module (201) is configured to acquire point cloud data of the area where the sound barrier nut is located, and the point cloud data includes the three-dimensional coordinate information of the sound barrier nut; The processing module (202) is configured to construct a nut three-dimensional model based on the point cloud data, and perform comparison calculation between the nut three-dimensional model and a standard nut model to obtain a distance field deviation between the nut three-dimensional model and the standard nut model, and the distance field deviation includes the deviation values of each area on the surface of the sound barrier nut; The processing module (202) is further configured to determine an abnormal deviation value from the deviation values, and determine the state of the sound barrier nut according to the abnormal deviation value, where the state includes a normal state and an abnormal state; The processing module (202) is further configured to, if it is determined that the sound barrier nut is in an abnormal state, divide the target point cloud corresponding to the abnormal deviation value into a plurality of target point cloud clusters; The processing module (202) is further configured to extract target features of each of the target point cloud clusters, where the target features include shape features, size features, and position features; The processing module (202) is further configured to identify the abnormal state type corresponding to the abnormal state according to the target features, where the abnormal state type includes dirt on the nut surface, nut wear, and nut corrosion.

9. An electronic device, characterized in that, It includes a processor (301), a memory (305), a user interface (303), and a network interface (304). The memory (305) is used to store instructions. The user interface (303) and the network interface (304) are used to communicate with other devices. The processor (301) is used to execute the instructions stored in the memory (305) so that the electronic device (300) executes the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1-7 is executed.

Images