Automatic identification method and system for mechanical watch case defects based on deep learning

Abstract

The present invention relates to the technical field of automated defect detection, and discloses a method and system for automatic identification of mechanical watch casing defects based on deep learning. The method comprises acquiring original image data and original point cloud data; performing feature extraction to obtain image contour features and point cloud feature values; inputting into a preset initial defect detection model to obtain preliminary defect detection results; performing geometric feature extraction to obtain key geometric features and minimum circumscribed rectangle features; correcting, standardizing and screening the chord length in the key geometric features to obtain a screened feature matrix; comparing, updating and feedback-adjusting the elements in the screened feature matrix with a preset threshold value to obtain the final classification result of the defect area. The standardized feature matrix is ​​input into a preset fusion geometric feature model to obtain a preliminary classification result corresponding to the position of the defect area. The method can improve the classification effect of automatic defect identification detection and improve the detection accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of automated defect detection, and in particular to a method and system for automatically identifying defects in a mechanical watch casing based on deep learning. Background Art

[0002] Mechanical watches are popular among consumers for their high mechanical quality, accurate timekeeping, long wearing time and easy maintenance. However, during the production process, the casing of mechanical watches is prone to defects such as wear, scratches, cracks and holes due to the influence of processing precision, material properties and external environment. These defects not only affect the appearance of the watch, but also may reduce the service life and durability of the watch. In particular, some minor defects are difficult to detect with the naked eye and require high-precision automated detection methods.

[0003] In the prior art, a defect detection method based on image processing technology obtains the image data of the dial through an image acquisition module, combines the defect contour curvature extraction algorithm to quickly locate the defect area, and identifies and alarms the defects through a classification model.

[0004] However, this technology has obvious shortcomings in practical applications: due to the diversity of shapes and sizes of defects in mechanical watch cases, relying solely on curvature extraction cannot fully describe the characteristics of the defective area, resulting in poor classification results and low detection accuracy. Summary of the invention

[0005] The present invention provides a method and system for automatically identifying defects in a mechanical watch casing based on deep learning, which can improve the classification effect of automatic defect identification and detection and enhance detection accuracy.

[0006] In the first aspect, in order to solve the above technical problems, the present invention provides a method for automatically identifying defects of a mechanical watch casing based on deep learning, comprising:

[0007] Acquire original image data and original point cloud data, wherein the original image data includes image data of the mechanical watch case before and during processing, and the original point cloud data includes point cloud data of the mechanical watch case before and during processing;

[0008] Extract features from the original image data and the original point cloud data using an image feature model and a point cloud feature model, respectively, to obtain image contour features and point cloud feature values;

[0009] Inputting the image contour feature and the point cloud feature value into a preset initial defect detection model to obtain a preliminary defect detection result, wherein the preliminary defect detection result includes defect area location and defect contour information;

[0010] Extracting geometric features from the preliminary defect detection results to obtain key geometric features and minimum circumscribed rectangle features; wherein the key geometric features include angle, side length, chord length and curvature, and the minimum circumscribed rectangle features include minimum circumscribed rectangle length, minimum circumscribed rectangle width and minimum circumscribed rectangle angle;

[0011] The chord length in the key geometric feature is corrected in combination with the chord length in the ideal model to obtain an updated key geometric feature, and an original eigenvalue matrix is ​​formed according to the updated key geometric feature and the minimum circumscribed rectangle feature;

[0012] Standardizing the original eigenvalue matrix to obtain a standardized eigenvalue matrix;

[0013] Inputting the standardized feature matrix into a preset fusion geometric feature model to obtain a preliminary classification result corresponding to the defect area position;

[0014] Selecting geometric features that meet the preliminary classification results from the standardized feature matrix to obtain a selected feature matrix;

[0015] Comparing the elements in the filtered feature matrix with a preset threshold, optimizing and updating the elements in the filtered feature matrix according to the comparison result to obtain an optimized feature matrix;

[0016] Combined with the optimized feature matrix, feedback adjustment is performed on the model parameters of the fused geometric feature model to obtain the final classification result of the defect area.

[0017] Preferably, the image contour feature and the point cloud feature value are input into a preset initial defect detection model to obtain a preliminary defect detection result, wherein the preliminary defect detection result includes defect area location and defect contour information, including:

[0018] Performing feature alignment on the image contour features and the point cloud feature values ​​respectively to obtain aligned image features and aligned point cloud features;

[0019] Performing multimodal fusion on the aligned image features and the aligned point cloud features to obtain fused joint features;

[0020] The fused joint features are input into a deep learning network to obtain a preliminary defect detection result, which includes defect area location and defect contour information.

[0021] Preferably, the extracting geometric features from the preliminary defect detection results to obtain key geometric features and minimum circumscribed rectangle features includes:

[0022] The angle is calculated using the following formula:

[0023]

[0024] in, is the angle between adjacent edges in the defect area, and are the vectors on the two adjacent sides, For vector The model, For vector Model;

[0025] The side length is calculated using the following formula:

[0026]

[0027] in, is the length of the edge between adjacent vertices, and are the coordinates of two adjacent vertices;

[0028] The chord length is calculated using the following formula:

[0029]

[0030] in, is the length of the chord between any two points, and Coordinates of any two points;

[0031] The radius of gyration of a triangle is calculated using the following formula:

[0032]

[0033] in, is the radius of gyration of the triangle, a, b, c represent the lengths of the three sides of the triangle respectively, is the area of ​​the triangle;

[0034] The curvature is calculated by the following formula:

[0035]

[0036] in, is the curvature of the defect area contour, Represents the interior angles of a triangle;

[0037] The minimum enclosing rectangle length is calculated using the following formula:

[0038]

[0039] in, For the The minimum enclosing rectangle length of the defect area, that is, the span of the minimum enclosing rectangle in the x direction, is the maximum x-coordinate of the midpoint of the defect area contour, is the minimum x-coordinate of the midpoint of the defect area contour;

[0040] The minimum bounding rectangle width is calculated using the following formula:

[0041]

[0042] in, For the The minimum bounding rectangle width of the defect area, that is, the span of the minimum bounding rectangle in the y direction, is the maximum y coordinate of the midpoint of the defect area contour, is the minimum y coordinate of the midpoint of the defect area contour;

[0043] The minimum enclosing rectangle angle is calculated using the following formula:

[0044]

[0045] in, For the The minimum circumscribed rectangle angle of the defect area is the angle formed by the vertices of the minimum circumscribed rectangle. is the x-coordinate difference between two adjacent vertices, is the y-coordinate difference between two adjacent vertices.

[0046] Preferably, the chord length in the key geometric feature is corrected by combining the chord length in the ideal model to obtain an updated key geometric feature, and an original eigenvalue matrix is ​​formed according to the updated key geometric feature and the minimum circumscribed rectangle feature, including:

[0047] Combined with the chord length in the ideal model, the chord length in the key geometric features is corrected by the following formula:

[0048]

[0049] in, is the corrected chord length, is the length of the chord between any two points, is the chord length in the ideal model;

[0050] Replacing the chord length in the key geometric feature according to the corrected chord length to obtain an updated key geometric feature;

[0051] According to the updated key geometric features and the minimum circumscribed rectangle features, the original eigenvalue matrix is ​​constructed by the following formula:

[0052]

[0053] in, is the original eigenvalue matrix, Each row in represents the geometric features of a defect area, and each column represents a specific geometric feature; , Indicates the angle, , represents the side length, , is the corrected chord length, , is the curvature, , is the minimum circumscribed rectangle length, , is the minimum bounding rectangle width, , is the minimum circumscribed rectangle angle.

[0054] Preferably, the step of normalizing the original eigenvalue matrix to obtain a standardized eigenvalue matrix comprises:

[0055] The original eigenvalue matrix is ​​normalized by the normalization formula to obtain a standardized eigenvalue matrix:

[0056]

[0057] in, represents the standardized feature matrix, is the original eigenvalue matrix Middle Row, No. The elements of the column, For the The minimum value of the column, For the Maximum value of the column.

[0058] Preferably, the step of inputting the standardized feature matrix into a preset fusion geometric feature model to obtain a preliminary classification result corresponding to the defect region position includes:

[0059] The mapping relationship between the model input layer and output layer of the fused geometric features is calculated by the following formula:

[0060]

[0061] in, is a preliminary classification result corresponding to the position of the defect area, indicating the initial classification probability of the defect area; is the normalized feature matrix of the input, is the weight matrix, is the deviation vector, Represents the activation function.

[0062] Preferably, comparing the elements in the filtered feature matrix with a preset threshold, and optimizing and updating the elements in the filtered feature matrix according to the comparison result to obtain an optimized feature matrix, comprises:

[0063] Each element in the filtered feature matrix is ​​judged according to a preset element minimum value and element maximum value, and each element is optimized according to the judgment result to obtain an optimized element;

[0064] Obtaining an optimized feature matrix according to the optimized elements;

[0065] Among them, judging each element includes:

[0066]

[0067] in, is the element in the original eigenvalue matrix, is the optimized element, and They are the preset minimum and maximum values ​​of the elements respectively.

[0068] Preferably, the step of combining the optimized feature matrix and performing feedback adjustment on the model parameters of the fused geometric feature model to obtain the final classification result of the defect area includes:

[0069] According to the optimized feature matrix, the mapping relationship between the model input layer and the output layer of the fused geometric features is recalculated to obtain a new model prediction probability value;

[0070] The error between the new model prediction probability value and the initial classification probability is calculated by the following formula:

[0071]

[0072] in, The error expressed, Predict the probability value for the new model, represents the initial classification probability;

[0073] According to the error, the weight matrix and bias vector are adjusted by the following formula:

[0074]

[0075]

[0076] in, and are the optimized weight matrix and the optimized bias vector respectively, is the learning rate, and are the gradients of the error with respect to the weight matrix and the bias vector, respectively;

[0077] Based on the optimized weight matrix and the optimized deviation vector, construct an optimized fusion geometric feature model;

[0078] The defective area is classified by the optimized fusion geometric feature model to obtain a final classification result of the defective area, and the final classification result includes the probability distribution of dents, cracks and holes.

[0079] In a second aspect, the present invention provides a mechanical watch case defect automatic identification system based on deep learning, comprising:

[0080] A data acquisition module, used to acquire original image data and original point cloud data, wherein the original image data includes image data of the mechanical watch housing before and during processing, and the original point cloud data includes point cloud data of the mechanical watch housing before and during processing;

[0081] A feature extraction module is used to extract features from the original image data and the original point cloud data respectively through an image feature model and a point cloud feature model to obtain image contour features and point cloud feature values;

[0082] A preliminary detection module, used for inputting the image contour features and the point cloud feature values ​​into a preset initial defect detection model to obtain a preliminary defect detection result, wherein the preliminary defect detection result includes defect area location and defect contour information;

[0083] A geometric feature extraction module, used to extract geometric features from the preliminary defect detection results to obtain key geometric features and minimum circumscribed rectangle features; wherein the key geometric features include angle, side length, chord length and curvature, and the minimum circumscribed rectangle features include minimum circumscribed rectangle length, minimum circumscribed rectangle width and minimum circumscribed rectangle angle;

[0084] A matrix construction module, used to correct the chord length in the key geometric feature in combination with the chord length in the ideal model to obtain an updated key geometric feature, and to construct an original eigenvalue matrix according to the updated key geometric feature and the minimum circumscribed rectangle feature;

[0085] A matrix standardization module is used to standardize the original eigenvalue matrix to obtain a standardized eigenvalue matrix;

[0086] A preliminary classification module, used for inputting the standardized feature matrix into a preset fusion geometric feature model to obtain a preliminary classification result corresponding to the position of the defect area;

[0087] A feature screening module, used to screen geometric features that meet the preliminary classification results from the standardized feature matrix to obtain a screened feature matrix;

[0088] An optimization and updating module, used for comparing the elements in the filtered feature matrix with a preset threshold value, and optimizing and updating the elements in the filtered feature matrix according to the comparison result to obtain an optimized feature matrix;

[0089] The final classification module is used to perform feedback adjustment on the model parameters of the fused geometric feature model in combination with the optimized feature matrix to obtain the final classification result of the defect area.

[0090] In a third aspect, the present invention also provides an electronic device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements any one of the above-mentioned methods for automatically identifying defects in mechanical watch cases based on deep learning.

[0091] In a fourth aspect, the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned methods for automatically identifying defects in mechanical watch cases based on deep learning.

[0092] Compared with the prior art, the present invention has the following beneficial effects: the embodiment of the present invention provides a method and system for automatic identification of mechanical watch case defects based on deep learning. The method includes: acquiring original image data and original point cloud data, the original image data including image data of the mechanical watch case before and during processing, the original point cloud data including point cloud data of the mechanical watch case before and during processing; performing feature extraction on the original image data and the original point cloud data respectively through an image feature model and a point cloud feature model to obtain image contour features and point cloud feature values; inputting the image contour features and the point cloud feature values ​​into a preset initial defect detection model to obtain preliminary defect detection results, the preliminary defect detection results including defect area location and defect contour information; performing geometric feature extraction on the preliminary defect detection results to obtain key geometric features and minimum bounding rectangle features; wherein the key geometric features include angle, side length, chord length and curvature, and the minimum bounding rectangle features include minimum bounding rectangle length, minimum bounding rectangle width and minimum bounding rectangle width. Rectangle angle; correct the chord length in the key geometric feature in combination with the chord length in the ideal model to obtain an updated key geometric feature, and form an original eigenvalue matrix based on the updated key geometric feature and the minimum circumscribed rectangle feature; standardize the original eigenvalue matrix to obtain a standardized feature matrix; input the standardized feature matrix into a preset fusion geometric feature model to obtain a preliminary classification result corresponding to the position of the defect area; filter the geometric features that meet the preliminary classification results from the standardized feature matrix to obtain a filtered feature matrix; compare the elements in the filtered feature matrix with a preset threshold, and optimize and update the elements in the filtered feature matrix according to the comparison result to obtain an optimized feature matrix; feedback adjust the model parameters of the fusion geometric feature model in combination with the optimized feature matrix to obtain the final classification result of the defect area.

[0093] In the present invention, the method obtains the original image and point cloud data of the mechanical watch case, extracts its image contour features and point cloud geometric features respectively, and aligns the two features and performs multimodal fusion to generate a more comprehensive joint feature representation; uses a deep learning model to process these features to identify the defect area and its contour; then, by calculating the geometric features (including angle, side length, chord length and curvature) and the minimum circumscribed rectangle features (including length, width and angle), the chord length is corrected in combination with the ideal model to construct a standardized feature matrix; the matrix is ​​further input into the fused geometric feature model to generate a preliminary classification result; then, through the feature screening and optimization update mechanism, the classification model parameters are adjusted to achieve accurate classification of the defect area. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] Figure 1It is a flowchart of a method for automatically identifying defects of a mechanical watch casing based on deep learning provided by a first embodiment of the present invention;

[0095] Figure 2 It is a structural diagram of a mechanical watch case defect automatic identification system based on deep learning provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0096] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0097] Reference Figure 1 The first embodiment of the present invention provides a method for automatically identifying defects of a mechanical watch casing based on deep learning, comprising the following steps:

[0098] S11, acquiring original image data and original point cloud data, wherein the original image data includes image data of the mechanical watch case before and during processing, and the original point cloud data includes point cloud data of the mechanical watch case before and during processing.

[0099] S12, performing feature extraction on the original image data and the original point cloud data respectively through an image feature model and a point cloud feature model to obtain image contour features and point cloud feature values.

[0100] S13, inputting the image contour features and the point cloud feature values ​​into a preset initial defect detection model to obtain a preliminary defect detection result, wherein the preliminary defect detection result includes defect area location and defect contour information.

[0101] S14, extracting geometric features from the preliminary defect detection results to obtain key geometric features and minimum circumscribed rectangle features; wherein the key geometric features include angles, side lengths, chord lengths and curvatures, and the minimum circumscribed rectangle features include minimum circumscribed rectangle length, minimum circumscribed rectangle width and minimum circumscribed rectangle angle.

[0102] S15, correcting the chord length in the key geometric feature in combination with the chord length in the ideal model to obtain an updated key geometric feature, and constructing an original eigenvalue matrix according to the updated key geometric feature and the minimum circumscribed rectangle feature.

[0103] S16, normalizing the original eigenvalue matrix to obtain a standardized eigenvalue matrix.

[0104] S17, inputting the standardized feature matrix into a preset fusion geometric feature model to obtain a preliminary classification result corresponding to the position of the defect area.

[0105] S18, selecting geometric features that meet the preliminary classification results from the standardized feature matrix to obtain a selected feature matrix.

[0106] S19, comparing the elements in the filtered feature matrix with a preset threshold, optimizing and updating the elements in the filtered feature matrix according to the comparison result, to obtain an optimized feature matrix.

[0107] S20, combining the optimized feature matrix, performing feedback adjustment on the model parameters of the fused geometric feature model to obtain a final classification result of the defect area.

[0108] It is worth mentioning that mechanical watches are popular among consumers for their high mechanical quality, accurate timekeeping, long wearing time and easy maintenance. However, during the production process, the casing of mechanical watches is prone to wear, scratches, cracks, holes and other defects due to the influence of processing precision, material properties and external environment. These defects not only affect the appearance of the watch, but also may reduce the service life and durability of the watch. In particular, some minor defects are difficult to detect with the naked eye and require high-precision automated detection methods.

[0109] In order to facilitate the understanding of the present invention, some preferred embodiments of the present invention are further described below.

[0110] In step S11, original image data and original point cloud data are acquired.

[0111] Preferably, the original image data includes image data of the mechanical watch case before and during processing, and the original point cloud data includes point cloud data of the mechanical watch case before and during processing.

[0112] Specifically, the process of obtaining raw image data and raw point cloud data usually relies on high-precision imaging and scanning technology, combined with automated equipment and data processing software. For mechanical watch casings, the process of obtaining raw data can be divided into the following steps:

[0113] First, optionally, the mechanical watch case is photographed by a high-resolution camera mounted on an industrial camera or visual sensor. During the processing, the camera needs to capture images of the case from multiple angles to ensure that its details from different perspectives are captured. The original image data includes the overall appearance of the case in the unprocessed state, as well as the images of the case after different processes (such as cutting, polishing, etc.) during the processing. The unprocessed image usually reflects the initial form of the case, and there may be obvious mold marks or processing marks on the surface; the images taken during the processing show changes in details, such as removed materials and smooth surfaces. These images not only provide basic data for subsequent analysis, but also provide a reference for precision calibration and quality control.

[0114] Secondly, optionally, the acquisition of point cloud data can be achieved through laser scanning, structured light or stereo vision technology. The laser scanner forms a set of precise spatial coordinate points by emitting a laser beam to the casing of a mechanical watch and receiving the signal reflected back. These coordinate points constitute the point cloud data, which represents the three-dimensional spatial position of each point on the surface of the casing. Similar to image data, point cloud data also needs to be collected at different processing stages. In the unprocessed state, the point cloud data shows the approximate outline and size of the casing surface, while the point cloud data during processing reflects the changes on the casing surface, including the gaps and shape adjustments after material removal. Through these point cloud data, the three-dimensional model of the casing can be accurately reconstructed for subsequent precision analysis and processing adjustments.

[0115] It is worth noting that in actual operation, image data and point cloud data are often collected synchronously to ensure that each frame of image and the corresponding point cloud data correspond to the shell state at the same time point. Optionally, this can be achieved through an integrated sensor system or data calibration method. Synchronous acquisition not only helps to improve the accuracy of the data, but also enables a more comprehensive evaluation and processing in the subsequent analysis process by combining the two-dimensional features of the image with the three-dimensional information of the point cloud.

[0116] Specifically, the original image data and point cloud data are stored and preliminarily processed by industrial computers or cloud servers. Image data needs to be compressed and format converted for efficient storage and transmission. Point cloud data needs to be denoised and coordinate system converted for subsequent geometric analysis or fusion with other data types. These processed data can be used as input for further analysis, modeling or control systems.

[0117] In step S12, feature extraction is performed on the original image data and the original point cloud data respectively through an image feature model and a point cloud feature model to obtain image contour features and point cloud feature values.

[0118] It is worth noting that the image feature model and the point cloud feature model are pre-stored and set feature extraction models. Specifically, the image feature model extracts descriptive and recognizable features from the image by modeling the geometry, color, and texture attributes of the image. The point cloud feature model constructs a mathematical model related to the geometric structure and shape of the object surface by analyzing the three-dimensional point information in the point cloud data. Point cloud data is composed of a large number of spatial coordinates (X, Y, Z) and is usually used to express the three-dimensional shape of an object. The point cloud feature model extracts the key features that describe the surface structure of the object by processing these spatial data.

[0119] Common image features include edge features, corner features, texture features, and color histograms. In an achievable way, image feature extraction can be divided into the following steps: preprocessing, including denoising, image enhancement and other operations to improve image quality. Commonly used techniques include Gaussian filtering and histogram equalization; edge detection, using edge detection algorithms (such as Canny edge detection) to identify significant edges in the image, which is crucial for extracting the contour of the mechanical watch case; feature point detection, using feature point extraction algorithms (such as SIFT, SURF, ORB, etc.) to identify unique key points in the image and calculate their descriptors; contour analysis, using contour extraction algorithms to extract the outer contour of the mechanical watch case, and generate image contour features based on the geometric characteristics of the contour (such as area, shape, curvature, etc.).

[0120] In one feasible way, the point cloud feature extraction process may include the following steps: data preprocessing, denoising, filtering, and sampling operations on the collected raw point cloud data. Commonly used techniques include uniform sampling and voxel grid filtering, the purpose of which is to reduce the amount of data and remove noise points to improve data quality; normal calculation and curvature analysis, calculating the normal vector and curvature of each point in the point cloud. The normal vector reflects the directionality of the point cloud surface, and the curvature can describe the degree of curvature of the surface near a certain point; feature point extraction, extracting feature points in the point cloud through methods such as the RANSAC (random sampling consistency) algorithm; point cloud segmentation, segmenting the point cloud data based on methods such as region growing and clustering, and dividing the point cloud data into multiple regions or objects; geometric feature calculation, performing geometric feature calculations on each segmented point cloud region, such as calculating the area, volume, and shape coefficient of the region.

[0121] In step S13, the image contour features and the point cloud feature values ​​are input into a preset initial defect detection model to obtain a preliminary defect detection result, which includes defect area location and defect contour information.

[0122] Preferably, the image contour feature and the point cloud feature value are input into a preset initial defect detection model to obtain a preliminary defect detection result, wherein the preliminary defect detection result includes defect area location and defect contour information, including:

[0123] Performing feature alignment on the image contour features and the point cloud feature values ​​respectively to obtain aligned image features and aligned point cloud features;

[0124] Performing multimodal fusion on the aligned image features and the aligned point cloud features to obtain fused joint features;

[0125] The fused joint features are input into a deep learning network to obtain a preliminary defect detection result, which includes defect area location and defect contour information.

[0126] Specifically, first, the image contour features and point cloud feature values ​​need to go through a feature alignment step. In this step, the image features and point cloud features are processed independently, and through alignment technology, the two data sources can be compared and fused in the same coordinate system or reference frame. Image features are extracted based on two-dimensional image data, including the edges, contours or texture features of the image, while point cloud features are extracted based on three-dimensional point cloud data, involving the geometry, normals, and curvature of the object surface. Since image data and point cloud data represent different spatial dimensions (two-dimensional and three-dimensional), it is necessary to align these two types of features, which ensures that these features can be effectively combined and complemented in further processing.

[0127] Specifically, the alignment process usually includes spatially aligning the image data with the point cloud data through perspective transformation, rotation, and scaling operations. For example, the coordinates of the image features are transformed so that they correspond to the three-dimensional point positions in the point cloud data, or the sensor calibration method is used to ensure that the coordinate systems of the image and point cloud data are consistent. Through feature alignment, it is possible to ensure that information from different data sources is comparable and consistent in subsequent analysis, providing reliable input for subsequent fusion and training of deep learning models.

[0128] Specifically, the aligned image features and the aligned point cloud features will enter the stage of multimodal fusion. In this step, the data from the image and point cloud are combined into a comprehensive feature vector, called a joint feature, through a specific fusion method. The purpose of multimodal fusion is to make up for the possible shortcomings of a single data source by combining the advantages of two data sources. For example, image features can provide detailed surface texture and shape information, while point cloud features can provide rich 3D geometry and surface structure information. After fusing these two types of information, a more comprehensive description can be obtained, which helps improve the accuracy and robustness of defect detection.

[0129] Specifically, the multimodal fusion technology can adopt weighted averaging, convolutional neural network (CNN) or fusion method based on attention mechanism. The weighted averaging method performs fusion by assigning weights to different modalities; the convolutional neural network can process image and point cloud data through a specific network structure, and then combine the feature information of the two; and the fusion method based on attention mechanism can adjust the feature fusion method according to the importance of different regions.

[0130] Finally, the fused joint features are input into a deep learning network, which can be a convolutional neural network (CNN)-based model or other neural networks suitable for processing multimodal data. By learning the defect patterns in a large amount of sample data, the deep learning network can identify the defect area in the input joint features and output preliminary defect detection results. These results include the regional location of the defect and the defect contour information. The defect area location refers to the spatial location or area where the defect is located in the image and point cloud data, presented in the form of coordinates; the defect contour information describes the shape, size, and distribution of the defect, helping to further identify the type of defect (such as cracks, dents, scratches).

[0131] In step S14, geometric features are extracted from the preliminary defect detection results to obtain key geometric features and minimum circumscribed rectangle features; wherein the key geometric features include angles, side lengths, chord lengths and curvatures, and the minimum circumscribed rectangle features include minimum circumscribed rectangle length, minimum circumscribed rectangle width and minimum circumscribed rectangle angle.

[0132] Specifically, the geometric feature extraction of the preliminary defect detection result to obtain key geometric features and minimum circumscribed rectangle features includes:

[0133] The angle is calculated using the following formula:

[0134]

[0135] in, is the angle between adjacent edges in the defect area, and are the vectors on the two adjacent sides, For vector The model, For vector Model;

[0136] The side length is calculated using the following formula:

[0137]

[0138] in, is the length of the edge between adjacent vertices, and are the coordinates of two adjacent vertices;

[0139] The chord length is calculated using the following formula:

[0140]

[0141] in, is the length of the chord between any two points, and Coordinates of any two points;

[0142] The radius of gyration of a triangle is calculated using the following formula:

[0143]

[0144] in, is the radius of gyration of the triangle, a, b, c represent the lengths of the three sides of the triangle respectively, is the area of ​​the triangle;

[0145] The curvature is calculated by the following formula:

[0146]

[0147] in, is the curvature of the defect area contour, Represents the interior angles of a triangle;

[0148] The minimum enclosing rectangle length is calculated using the following formula:

[0149]

[0150] in, For the The minimum enclosing rectangle length of the defect area, that is, the span of the minimum enclosing rectangle in the x direction, is the maximum x-coordinate of the midpoint of the defect area contour, is the minimum x-coordinate of the midpoint of the defect area contour;

[0151] The minimum bounding rectangle width is calculated using the following formula:

[0152]

[0153] in, For the The minimum bounding rectangle width of the defect area, that is, the span of the minimum bounding rectangle in the y direction, is the maximum y coordinate of the midpoint of the defect area contour, is the minimum y coordinate of the midpoint of the defect area contour;

[0154] The minimum enclosing rectangle angle is calculated using the following formula:

[0155]

[0156] in, For the The minimum circumscribed rectangle angle of the defect area is the angle formed by the vertices of the minimum circumscribed rectangle. is the x-coordinate difference between two adjacent vertices, is the y-coordinate difference between two adjacent vertices.

[0157] Specifically, the key geometric features include angles, side lengths, chord lengths, gyration radius, and curvature. The angle is obtained by calculating the angle between adjacent sides in the defect area. The angle can reflect the turning point or boundary curvature of the defect area. The side length measures the straight-line distance between two adjacent vertices in the defect area, which is used to describe the length or span of the defect. The chord length measures the straight-line distance between any two points, which helps evaluate the width or span of the defect area. The gyration radius is used to calculate the geometric shape of the defect area contour, which reflects the stability and shape characteristics of the defect, and is especially important in circular or nearly circular defects. Curvature is a feature that describes the degree of curvature of the defect area contour. A larger curvature value usually indicates a sharper curve in the defect area, which helps analyze curved or irregularly shaped defects.

[0158] Specifically, the minimum circumscribed rectangle feature is mainly used to represent the bounding box of the defect area, that is, the minimum rectangle surrounding the defect area. The length of the minimum circumscribed rectangle refers to the span of the rectangle on the horizontal axis, reflecting the lateral range of the defect area. The width refers to the span of the rectangle on the vertical axis, indicating the longitudinal range of the defect area. These two features can help determine the overall size and shape of the defect. The minimum circumscribed rectangle angle refers to the angle between the rectangle and the horizontal axis, which can reflect the inclination or rotation of the defect area, especially when the defect area has a certain angle offset, helping to accurately locate the direction of the defect.

[0159] In step S15, the chord length in the key geometric feature is corrected in combination with the chord length in the ideal model to obtain an updated key geometric feature, and an original eigenvalue matrix is ​​constructed based on the updated key geometric feature and the minimum circumscribed rectangle feature.

[0160] Preferably, the chord length in the key geometric feature is corrected by combining the chord length in the ideal model to obtain an updated key geometric feature, and an original eigenvalue matrix is ​​formed according to the updated key geometric feature and the minimum circumscribed rectangle feature, including:

[0161] Combined with the chord length in the ideal model, the chord length in the key geometric features is corrected by the following formula:

[0162]

[0163] in, is the corrected chord length, is the length of the chord between any two points, is the chord length in the ideal model;

[0164] Replacing the chord length in the key geometric feature according to the corrected chord length to obtain an updated key geometric feature;

[0165] According to the updated key geometric features and the minimum circumscribed rectangle features, the original eigenvalue matrix is ​​constructed by the following formula:

[0166]

[0167] in, is the original eigenvalue matrix, Each row in represents the geometric features of a defect area, and each column represents a specific geometric feature; , Indicates the angle, , represents the side length, , is the corrected chord length, , is the curvature, , is the minimum circumscribed rectangle length, , is the minimum bounding rectangle width, , is the minimum circumscribed rectangle angle.

[0168] Specifically, the corrected chord length is to adjust the original chord length according to the chord length in the ideal model. The chord length in the ideal model represents the chord length under ideal or standard conditions, while the chord length extracted in the actual test may deviate due to external factors or measurement errors. By comparing the actual chord length with the chord length in the ideal model, the actual chord length can be corrected to ensure that it more accurately reflects the actual situation of the defect area. This correction process helps to eliminate the inaccurate chord length caused by errors, thereby improving the accuracy of geometric analysis of defects.

[0169] Specifically, updating key geometric features means that after correcting the chord length, the corrected chord length is replaced with the original geometric features to obtain an updated set of key geometric features. This updated set of geometric features includes the corrected chord length and other geometric features, such as angle, side length, and curvature. This process ensures that all geometric features have high accuracy, especially when the chord length is a key factor affecting defect classification and location, the corrected data can more realistically reflect the morphology of the defect.

[0170] Specifically, the original eigenvalue matrix is ​​constructed by combining the updated key geometric features with the minimum enclosing rectangle features to form a unified feature matrix. The features of each defect area will be represented as a row of the matrix, where each row contains different geometric features of the defect area. For example, each row may contain the angle, side length, corrected chord length, curvature, length and width of the minimum enclosing rectangle, and angle information of the minimum enclosing rectangle. These features provide standardized input for subsequent analysis and processing, helping to improve the accuracy of automatic identification and classification of defects.

[0171] In step S16, the original eigenvalue matrix is ​​standardized to obtain a standardized eigenvalue matrix.

[0172] Preferably, the step of normalizing the original eigenvalue matrix to obtain a standardized eigenvalue matrix comprises:

[0173] The original eigenvalue matrix is ​​normalized by the normalization formula to obtain a standardized eigenvalue matrix:

[0174]

[0175] in, represents the standardized feature matrix, is the original eigenvalue matrix Middle Row, No. The elements of the column, For the The minimum value of the column, For the Maximum value of the column.

[0176] It is worth noting that the purpose of standardization is to convert the data in the original eigenvalue matrix into a unified scale for subsequent analysis and processing. Since different geometric features (such as angles, side lengths, chord lengths, etc.) usually have different value ranges and dimensions, directly comparing them or using them for model training may cause some features to have too much or too little impact on the final result. Standardization can eliminate this impact and make each feature's contribution to the model more balanced.

[0177] Specifically, the standardization process includes the following steps: First, it is necessary to determine each column of features (that is, the value range of each feature in all defect areas). For each column in the original eigenvalue matrix, calculate the minimum and maximum values ​​in the column. These two values ​​represent the minimum and maximum values ​​of the feature in all defect areas, respectively. Next, each eigenvalue is normalized, that is, the actual value of each feature is converted into a standardized value between 0 and 1. The normalization method is to subtract the minimum value of the feature column from each eigenvalue, and then divide it by the value range of the column (the difference between the maximum and minimum values). In this way, the value of each feature is compressed into a uniform range, avoiding some features dominating the training process of the model due to large or small values. After processing through the above steps, a new standardized feature matrix is ​​obtained, in which each eigenvalue is in the same scale range.

[0178] In step S17, the standardized feature matrix is ​​input into a preset fusion geometric feature model to obtain a preliminary classification result corresponding to the position of the defect area.

[0179] Preferably, the step of inputting the standardized feature matrix into a preset fusion geometric feature model to obtain a preliminary classification result corresponding to the defect region position includes:

[0180] The mapping relationship between the model input layer and output layer of the fused geometric features is calculated by the following formula:

[0181]

[0182] in, is a preliminary classification result corresponding to the position of the defect area, indicating the initial classification probability of the defect area; is the normalized feature matrix of the input, is the weight matrix, is the deviation vector, Represents the activation function.

[0183] Specifically, the standardized feature matrix is ​​input into the preset fusion geometric feature model to obtain the preliminary classification results of each defect area. The fusion geometric feature model is a model based on deep learning. Its task is to learn the relationship between the geometric features of each defect area through the input standardized feature matrix, so as to classify the defects. In this model, the input layer receives the standardized feature data, and the output layer gives the probability value of each defect area belonging to different categories.

[0184] Specifically, the standardized feature matrix is ​​input to the input layer of the model, which contains the geometric feature data of all defect areas. With this data, the model linearly transforms the features through the weight matrix and bias vector, and uses the activation function (Softmax) to perform nonlinear processing on the results. Specifically, the Softmax function converts the output of the model into a probability value, indicating the possibility that each defect area belongs to a different category.

[0185] It is worth noting that the role of the Softmax function is to convert the output value of the model (which may be any real number) into a set of non-negative numbers and ensure that the sum of the probabilities of all categories is 1. In this way, the model can output the probability that each defect area belongs to each possible category, which reflects the credibility or confidence that the defect area belongs to a certain category. The higher the output probability value, the greater the possibility that the defect area belongs to the corresponding category. The weight matrix (W) and the bias vector (b) are learned by the optimization algorithm during the model training process. The weight matrix controls the influence of the input features on the output, while the bias vector is used to adjust the output of the model to help the model better fit the data. Through this process, the standardized feature matrix is ​​converted into a preliminary defect classification result. The result represents the classification probability of each defect area in multiple categories.

[0186] In step S18, geometric features that meet the preliminary classification results are screened from the standardized feature matrix to obtain a screened feature matrix.

[0187] Specifically, in step S17, the preliminary classification results of each defect area are obtained by inputting the standardized feature matrix into the fusion geometric feature model. This result is that the geometric features (such as angle, side length, chord length, curvature, etc.) of each defect area are analyzed and classified by the model, and the probability of each defect area belonging to different categories is output. These probability values ​​are used to preliminarily determine which category the defect area belongs to. For example, the model may determine that a defect area is a crack, a dent, or a scratch. In step S18, the goal is to filter out geometric features that match the type of each defect area from the standardized feature matrix based on the preliminary classification results. For example, assuming that the preliminary classification results of the model indicate that some defect areas belong to the "crack" category, then in the screening process, the model will filter out relevant features based on the typical geometric features of the crack category (such as larger side length, specific angle and curvature, etc.). Optionally, this screening process can be based on the following principle: For each category (such as cracks, dents, surface scratches, etc.), a set of typical geometric feature value ranges or distributions will be determined during the training process. By combining with the preliminary classification results, data that meets these typical feature ranges is screened out.

[0188] After the screening process, the resulting filtered feature matrix contains geometric features related to the classification results. This filtered matrix will only contain features that are helpful for defect classification and further analysis. It reduces irrelevant or redundant features and focuses on key data that can accurately reflect the defect type and morphology.

[0189] In step S19, the elements in the filtered feature matrix are compared with a preset threshold value, and the elements in the filtered feature matrix are optimized and updated according to the comparison result to obtain an optimized feature matrix.

[0190] Preferably, comparing the elements in the filtered feature matrix with a preset threshold, and optimizing and updating the elements in the filtered feature matrix according to the comparison result to obtain an optimized feature matrix, comprises:

[0191] Each element in the filtered feature matrix is ​​judged according to a preset element minimum value and element maximum value, and each element is optimized according to the judgment result to obtain an optimized element;

[0192] Obtaining an optimized feature matrix according to the optimized elements;

[0193] Among them, judging each element includes:

[0194]

[0195] in, is the element in the original eigenvalue matrix, is the optimized element, and They are the preset minimum and maximum values ​​of the elements respectively.

[0196] Specifically, during the optimization process, the value of each element will be compared with the preset minimum and maximum values. The preset minimum and maximum values ​​represent the reasonable range of the feature value under a specific category or defect type. Through this step, it can be determined whether each element falls within the valid range.

[0197] Specifically, when the value of an element is between the minimum and maximum values, the element is considered valid and does not need to be modified, and its original value is maintained. When the value of an element is less than the minimum value, it means that the value of the element is lower than the expected range, so it needs to be adjusted to the minimum value to ensure that it meets a reasonable range. When the value of an element is greater than the maximum value, it means that the value of the element exceeds the expected upper limit, so it is adjusted to the maximum value to avoid the value being too large and having an adverse effect on subsequent analysis. After each element is compared with the preset minimum and maximum values, if its value exceeds the range, it will be limited to this range, thereby obtaining the optimized element. This optimization process helps to correct abnormal values ​​or noise data, making the data in the feature matrix more stable and reasonable.

[0198] In step S20, the model parameters of the fused geometric feature model are feedback-adjusted in combination with the optimized feature matrix to obtain a final classification result of the defect area.

[0199] Preferably, the step of combining the optimized feature matrix and performing feedback adjustment on the model parameters of the fused geometric feature model to obtain the final classification result of the defect area includes:

[0200] According to the optimized feature matrix, the mapping relationship between the model input layer and the output layer of the fused geometric features is recalculated to obtain a new model prediction probability value;

[0201] The error between the new model prediction probability value and the initial classification probability is calculated by the following formula:

[0202]

[0203] in, The error expressed, Predict the probability value for the new model, represents the initial classification probability;

[0204] According to the error, the weight matrix and bias vector are adjusted by the following formula:

[0205]

[0206]

[0207] in, and are the optimized weight matrix and the optimized bias vector respectively, is the learning rate, and are the gradients of the error with respect to the weight matrix and the bias vector, respectively;

[0208] Based on the optimized weight matrix and the optimized deviation vector, construct an optimized fusion geometric feature model;

[0209] The defective area is classified by the optimized fusion geometric feature model to obtain a final classification result of the defective area, and the final classification result includes the probability distribution of dents, cracks and holes.

[0210] Specifically, in step S20, the goal is to feedback-adjust the model parameters of the fused geometric feature model by combining the optimized feature matrix, so as to obtain the final classification result of the defect area.

[0211] Specifically, the optimized feature matrix is ​​used as input, and the output of the fused geometric feature model is recalculated according to a formula for calculating the mapping relationship between the input layer and the output layer of the model of the fused geometric features.

[0212] The model generates new predictions based on these optimized features. Specifically, the model calculates the probability distribution of each defect area belonging to different categories (such as dents, cracks, holes, etc.). Next, the new model prediction probability value is compared with the initial classification probability to calculate the error. The initial classification probability is the prediction result of the model in the previous step, and the new prediction probability is calculated by the optimized feature matrix. The size of the error reflects the gap between the classification result and the true classification of the model under the optimized feature input. The smaller the error, the more accurate the model's prediction and the more reliable the classification result. According to the calculated error, the parameters of the model are adjusted through the back propagation algorithm. Specifically, the weight matrix and bias vector of the model will be updated according to the size of the error. This process is adjusted by calculating the gradient of the error to the weight and bias, and then multiplying it by the learning rate. The learning rate controls the amplitude of the update to ensure that the model does not lose convergence during the optimization process due to too large or too small step size. Through this feedback adjustment, the parameters of the model will be gradually optimized, making its prediction results more accurate. After feedback adjustment, the model weights and biases are updated, and these optimized parameters constitute a new fused geometric feature model.

[0213] The model is now better able to classify defects based on the optimized features, providing more accurate prediction results. Finally, the defect areas are finally classified through the optimized fusion geometric feature model. The model classifies each defect area based on the new weight and bias parameters, and outputs the probability distribution of each area belonging to different types (such as dents, cracks, holes, etc.).

[0214] Specifically, the parameters of the model are adjusted through feedback, so that the fused geometric feature model can more accurately classify the defect area. In this process, by calculating the model error and adjusting the model weight and bias, the optimized model can make more accurate classification predictions based on the optimized feature data, and ultimately provide accurate classification results and probability distribution for the defect area.

[0215] In summary, the present invention provides a method for automatic identification of mechanical watch case defects based on deep learning, including: The embodiment of the present invention provides a method and system for automatic identification of mechanical watch case defects based on deep learning. The method includes: acquiring original image data and original point cloud data, the original image data including image data of the mechanical watch case before and during processing, the original point cloud data including point cloud data of the mechanical watch case before and during processing; performing feature extraction on the original image data and the original point cloud data respectively through an image feature model and a point cloud feature model to obtain image contour features and point cloud feature values; inputting the image contour features and the point cloud feature values ​​into a preset initial defect detection model to obtain preliminary defect detection results, the preliminary defect detection results including defect area location and defect contour information; performing geometric feature extraction on the preliminary defect detection results to obtain key geometric features and minimum circumscribed rectangle features; wherein the key geometric features include angles, side lengths, chord lengths and curvatures, and the minimum circumscribed rectangle features include minimum circumscribed rectangle length, minimum circumscribed rectangle width and minimum circumscribed rectangle width. Rectangle angle; correct the chord length in the key geometric feature in combination with the chord length in the ideal model to obtain an updated key geometric feature, and form an original eigenvalue matrix based on the updated key geometric feature and the minimum circumscribed rectangle feature; standardize the original eigenvalue matrix to obtain a standardized feature matrix; input the standardized feature matrix into a preset fusion geometric feature model to obtain a preliminary classification result corresponding to the position of the defect area; filter the geometric features that meet the preliminary classification results from the standardized feature matrix to obtain a filtered feature matrix; compare the elements in the filtered feature matrix with a preset threshold, and optimize and update the elements in the filtered feature matrix according to the comparison result to obtain an optimized feature matrix; feedback adjust the model parameters of the fusion geometric feature model in combination with the optimized feature matrix to obtain the final classification result of the defect area.

[0216] In the present invention, the method obtains the original image and point cloud data of the mechanical watch case, extracts its image contour features and point cloud geometric features respectively, and aligns the two features and performs multimodal fusion to generate a more comprehensive joint feature representation; uses a deep learning model to process these features to identify the defect area and its contour; then, by calculating the geometric features (including angle, side length, chord length and curvature) and the minimum circumscribed rectangle features (including length, width and angle), the chord length is corrected in combination with the ideal model to construct a standardized feature matrix; the matrix is ​​further input into the fused geometric feature model to generate a preliminary classification result; then, through the feature screening and optimization update mechanism, the classification model parameters are adjusted to achieve accurate classification of the defect area.

[0217] Reference Figure 2 The second embodiment of the present invention provides a mechanical watch case defect automatic identification system based on deep learning, comprising:

[0218] A data acquisition module, used to acquire original image data and original point cloud data, wherein the original image data includes image data of the mechanical watch housing before and during processing, and the original point cloud data includes point cloud data of the mechanical watch housing before and during processing;

[0219] A feature extraction module is used to extract features from the original image data and the original point cloud data respectively through an image feature model and a point cloud feature model to obtain image contour features and point cloud feature values;

[0220] A preliminary detection module, used for inputting the image contour features and the point cloud feature values ​​into a preset initial defect detection model to obtain a preliminary defect detection result, wherein the preliminary defect detection result includes defect area location and defect contour information;

[0221] A geometric feature extraction module, used to extract geometric features from the preliminary defect detection results to obtain key geometric features and minimum circumscribed rectangle features; wherein the key geometric features include angle, side length, chord length and curvature, and the minimum circumscribed rectangle features include minimum circumscribed rectangle length, minimum circumscribed rectangle width and minimum circumscribed rectangle angle;

[0222] A matrix construction module, used to correct the chord length in the key geometric feature in combination with the chord length in the ideal model to obtain an updated key geometric feature, and to construct an original eigenvalue matrix according to the updated key geometric feature and the minimum circumscribed rectangle feature;

[0223] A matrix standardization module is used to standardize the original eigenvalue matrix to obtain a standardized eigenvalue matrix;

[0224] A preliminary classification module, used for inputting the standardized feature matrix into a preset fusion geometric feature model to obtain a preliminary classification result corresponding to the position of the defect area;

[0225] A feature screening module, used to screen geometric features that meet the preliminary classification results from the standardized feature matrix to obtain a screened feature matrix;

[0226] An optimization and updating module, used for comparing the elements in the filtered feature matrix with a preset threshold value, and optimizing and updating the elements in the filtered feature matrix according to the comparison result to obtain an optimized feature matrix;

[0227] The final classification module is used to perform feedback adjustment on the model parameters of the fused geometric feature model in combination with the optimized feature matrix to obtain the final classification result of the defect area.

[0228] It should be noted that the automatic identification system for mechanical watch case defects based on deep learning provided in an embodiment of the present invention is used to execute all the process steps of the automatic identification method for mechanical watch case defects based on deep learning in the above-mentioned embodiment. The working principles and beneficial effects of the two correspond one to one, and therefore will not be repeated here.

[0229] An embodiment of the present invention further provides an electronic device. The electronic device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a program for a method for automatically identifying defects in a mechanical watch case based on deep learning. When the processor executes the computer program, the steps in the above-mentioned method for automatically identifying defects in a mechanical watch case based on deep learning are implemented, such as Figure 1 Alternatively, when the processor executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are realized, such as the final classification module.

[0230] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the electronic device.

[0231] The electronic device may be a computing device such as a desktop computer, a notebook, a PDA, and a smart tablet. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the above components are merely examples of electronic devices and do not constitute a limitation on the electronic device. The electronic device may include more or fewer components than the above components, or may combine certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0232] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device.

[0233] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the electronic device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0234] Wherein, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0235] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0236] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for automatic identification of mechanical watch casing defects based on deep learning, characterized in that: include: Acquire original image data and original point cloud data, wherein the original image data includes image data of the mechanical watch case before and during processing, and the original point cloud data includes point cloud data of the mechanical watch case before and during processing; Extract features from the original image data and the original point cloud data using an image feature model and a point cloud feature model, respectively, to obtain image contour features and point cloud feature values; Inputting the image contour feature and the point cloud feature value into a preset initial defect detection model to obtain a preliminary defect detection result, wherein the preliminary defect detection result includes defect area location and defect contour information; Extracting geometric features from the preliminary defect detection results to obtain key geometric features and minimum circumscribed rectangle features; wherein the key geometric features include angle, side length, chord length and curvature, and the minimum circumscribed rectangle features include minimum circumscribed rectangle length, minimum circumscribed rectangle width and minimum circumscribed rectangle angle; The chord length in the key geometric feature is corrected in combination with the chord length in the ideal model to obtain an updated key geometric feature, and an original eigenvalue matrix is ​​formed according to the updated key geometric feature and the minimum circumscribed rectangle feature; Standardizing the original eigenvalue matrix to obtain a standardized eigenvalue matrix; Inputting the standardized feature matrix into a preset fusion geometric feature model to obtain a preliminary classification result corresponding to the defect area position; Selecting geometric features that meet the preliminary classification results from the standardized feature matrix to obtain a selected feature matrix; Comparing the elements in the filtered feature matrix with a preset threshold, optimizing and updating the elements in the filtered feature matrix according to the comparison result to obtain an optimized feature matrix; Combined with the optimized feature matrix, feedback adjustment is performed on the model parameters of the fused geometric feature model to obtain the final classification result of the defect area.

2. The method for automatic identification of mechanical watch casing defects based on deep learning according to claim 1 is characterized in that: The image contour feature and the point cloud feature value are input into a preset initial defect detection model to obtain a preliminary defect detection result, wherein the preliminary defect detection result includes defect area location and defect contour information, including: Performing feature alignment on the image contour features and the point cloud feature values ​​respectively to obtain aligned image features and aligned point cloud features; Performing multimodal fusion on the aligned image features and the aligned point cloud features to obtain fused joint features; The fused joint features are input into a deep learning network to obtain a preliminary defect detection result, which includes defect area location and defect contour information.

3. The method for automatic identification of mechanical watch casing defects based on deep learning according to claim 1, characterized in that: The step of extracting geometric features from the preliminary defect detection results to obtain key geometric features and minimum circumscribed rectangle features includes: The angle is calculated using the following formula: in, is the angle between adjacent edges in the defect area, and are the vectors on the two adjacent sides, For vector The model, For vector Model; The side length is calculated using the following formula: in, is the length of the edge between adjacent vertices, and are the coordinates of two adjacent vertices; The chord length is calculated using the following formula: in, is the length of the chord between any two points, and Coordinates of any two points; The radius of gyration of a triangle is calculated using the following formula: in, is the radius of gyration of the triangle, a, b, c represent the lengths of the three sides of the triangle respectively, is the area of ​​the triangle; The curvature is calculated by the following formula: in, is the curvature of the defect area contour, Represents the interior angles of a triangle; The minimum enclosing rectangle length is calculated using the following formula: in, For the The minimum enclosing rectangle length of the defect area, that is, the span of the minimum enclosing rectangle in the x direction, is the maximum x-coordinate of the midpoint of the defect area contour, is the minimum x-coordinate of the midpoint of the defect area contour; The minimum bounding rectangle width is calculated using the following formula: in, For the The minimum bounding rectangle width of the defect area, that is, the span of the minimum bounding rectangle in the y direction, is the maximum y coordinate of the midpoint of the defect area contour, is the minimum y coordinate of the midpoint of the defect area contour; The minimum enclosing rectangle angle is calculated using the following formula: in, For the The minimum circumscribed rectangle angle of the defect area is the angle formed by the vertices of the minimum circumscribed rectangle. is the x-coordinate difference between two adjacent vertices, is the difference in y coordinates between two adjacent vertices.

4. The method for automatic identification of mechanical watch casing defects based on deep learning according to claim 1, characterized in that: The chord length in the key geometric feature is corrected by combining the chord length in the ideal model to obtain an updated key geometric feature, and an original eigenvalue matrix is ​​formed according to the updated key geometric feature and the minimum circumscribed rectangle feature, including: Combined with the chord length in the ideal model, the chord length in the key geometric features is corrected by the following formula: in, is the corrected chord length, is the length of the chord between any two points, is the chord length in the ideal model; Replacing the chord length in the key geometric feature according to the corrected chord length to obtain an updated key geometric feature; According to the updated key geometric features and the minimum circumscribed rectangle features, the original eigenvalue matrix is ​​constructed by the following formula: in, is the original eigenvalue matrix, Each row in represents the geometric features of a defect area, and each column represents a specific geometric feature; , Indicates the angle, , represents the side length, , is the corrected chord length, , is the curvature, , is the minimum circumscribed rectangle length, , is the minimum bounding rectangle width, , is the minimum circumscribed rectangle angle.

5. The method for automatic identification of mechanical watch casing defects based on deep learning according to claim 4 is characterized in that: The step of normalizing the original eigenvalue matrix to obtain a normalized eigenvalue matrix includes: The original eigenvalue matrix is ​​normalized by the normalization formula to obtain a standardized eigenvalue matrix: in, represents the standardized feature matrix, is the original eigenvalue matrix Middle Row, No. The elements of the column, For the The minimum value of the column, For the Maximum value of the column.

6. The method for automatic identification of mechanical watch casing defects based on deep learning according to claim 1, characterized in that: The step of inputting the standardized feature matrix into a preset fusion geometric feature model to obtain a preliminary classification result corresponding to the defect area position includes: The mapping relationship between the model input layer and output layer of the fused geometric features is calculated by the following formula: in, is a preliminary classification result corresponding to the position of the defect area, indicating the initial classification probability of the defect area; is the normalized feature matrix of the input, is the weight matrix, is the deviation vector, Represents the activation function.

7. The method for automatic identification of mechanical watch casing defects based on deep learning according to claim 1, characterized in that: The step of comparing the elements in the filtered feature matrix with a preset threshold, and optimizing and updating the elements in the filtered feature matrix according to the comparison result to obtain an optimized feature matrix includes: Each element in the filtered feature matrix is ​​judged according to a preset element minimum value and element maximum value, and each element is optimized according to the judgment result to obtain an optimized element; Obtaining an optimized feature matrix according to the optimized elements; Among them, judging each element includes: in, is the element in the original eigenvalue matrix, is the optimized element, and They are the preset minimum and maximum values ​​of the elements respectively.

8. The method for automatic identification of mechanical watch casing defects based on deep learning according to claim 6, characterized in that: The step of combining the optimized feature matrix and performing feedback adjustment on the model parameters of the fused geometric feature model to obtain the final classification result of the defect area includes: According to the optimized feature matrix, the mapping relationship between the model input layer and the output layer of the fused geometric features is recalculated to obtain a new model prediction probability value; The error between the new model prediction probability value and the initial classification probability is calculated by the following formula: in, The error expressed, Predict the probability value for the new model, represents the initial classification probability; According to the error, the weight matrix and bias vector are adjusted by the following formula: in, and are the optimized weight matrix and the optimized bias vector respectively, is the learning rate, and are the gradients of the error with respect to the weight matrix and the bias vector, respectively; Based on the optimized weight matrix and the optimized deviation vector, construct an optimized fusion geometric feature model; The defective area is classified by the optimized fusion geometric feature model to obtain a final classification result of the defective area, and the final classification result includes the probability distribution of dents, cracks and holes.

9. A mechanical watch case defect automatic identification system based on deep learning, characterized in that: include: A data acquisition module, used to acquire original image data and original point cloud data, wherein the original image data includes image data of the mechanical watch housing before and during processing, and the original point cloud data includes point cloud data of the mechanical watch housing before and during processing; A feature extraction module is used to extract features from the original image data and the original point cloud data respectively through an image feature model and a point cloud feature model to obtain image contour features and point cloud feature values; A preliminary detection module, used for inputting the image contour features and the point cloud feature values ​​into a preset initial defect detection model to obtain a preliminary defect detection result, wherein the preliminary defect detection result includes defect area location and defect contour information; A geometric feature extraction module, used to extract geometric features from the preliminary defect detection results to obtain key geometric features and minimum circumscribed rectangle features; wherein the key geometric features include angle, side length, chord length and curvature, and the minimum circumscribed rectangle features include minimum circumscribed rectangle length, minimum circumscribed rectangle width and minimum circumscribed rectangle angle; A matrix construction module, used to correct the chord length in the key geometric feature in combination with the chord length in the ideal model to obtain an updated key geometric feature, and to construct an original eigenvalue matrix according to the updated key geometric feature and the minimum circumscribed rectangle feature; A matrix standardization module is used to standardize the original eigenvalue matrix to obtain a standardized eigenvalue matrix; A preliminary classification module, used for inputting the standardized feature matrix into a preset fusion geometric feature model to obtain a preliminary classification result corresponding to the position of the defect area; A feature screening module, used to screen geometric features that meet the preliminary classification results from the standardized feature matrix to obtain a screened feature matrix; An optimization and updating module, used for comparing the elements in the filtered feature matrix with a preset threshold value, and optimizing and updating the elements in the filtered feature matrix according to the comparison result to obtain an optimized feature matrix; The final classification module is used to perform feedback adjustment on the model parameters of the fused geometric feature model in combination with the optimized feature matrix to obtain the final classification result of the defect area.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the method for automatically identifying defects of a mechanical watch case based on deep learning as described in any one of claims 1 to 8.

Images