Intelligent hardware grinding system and method based on machine vision

Abstract

The invention discloses an intelligent hardware grinding system based on machine vision, which comprises a machine vision module used for carrying out image acquisition on grinding of hardware and generating surface feature information and grinding state image feature information of the hardware; the robot grinding module is used for grinding the hardware according to the surface feature information and the grinding state image feature information; the control module is connected with the machine vision module and the robot grinding module and used for adjusting grinding equipment control instruction features and grinding parameters in real time according to the surface feature information and the grinding state image information; according to the method, the target hardware grinding information is subjected to image recognition, the image description set is mined, and accurate path planning is generated and covers shape, size, defect position and other information. According to the method, deviation judgment is carried out on the multiple grinding task events, so that the deviation correction execution strategy is determined, the hardware grinding precision is effectively improved, the hardware rejection rate is reduced, and the hardware production efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent grinding of hardware based on machine vision, and in particular to a system and method for intelligent grinding of hardware based on machine vision. Background Art

[0002] In the field of hardware processing, grinding is one of the key processes to achieve high precision and high quality on the workpiece surface. Traditional manual grinding methods have many problems, such as high technical requirements for operators, high labor intensity, harsh working environment, and unstable processing quality. With the rapid development of industrial automation and intelligent manufacturing, robotic automated grinding technology has gradually become mainstream. However, traditional robotic grinding systems rely on preset programs and fixed trajectories, which are difficult to adapt to the complex shapes and size changes of workpieces.

[0003] In recent years, machine vision technology has been increasingly used in industrial manufacturing, especially in the field of hardware inspection and processing. Through high-resolution imaging and intelligent image processing algorithms, machine vision systems can achieve high-precision detection of surface defects on workpieces and provide real-time feedback on the status of workpieces. Combined with robotics technology, machine vision can provide more accurate workpiece positioning and real-time monitoring functions for grinding systems, thereby improving grinding accuracy and efficiency.

[0004] However, the current machine vision grinding system still has some shortcomings. For example, grinding wheel wear detection usually relies on manual experience or qualitative methods, which makes it difficult to achieve accurate quantitative detection. In addition, for the grinding of complex curved surface hardware, the existing system still needs to be improved in terms of grinding path and grinding parameters. In addition, it is impossible to distinguish deviations for different hardware grinding task events. If a deviation problem is found, it is impossible to determine the automatic deviation execution strategy. Summary of the invention

[0005] The purpose of the present invention is to provide a hardware intelligent grinding system and method based on machine vision to solve the above-mentioned problems existing in the prior art.

[0006] The specific application is as follows:

[0007] A hardware intelligent grinding system based on machine vision, comprising:

[0008] The machine vision module is used to collect images of the hardware grinding and generate the surface feature information of the hardware and the image feature information of the grinding status;

[0009] A robot grinding module, used for grinding the hardware according to the surface feature information and the grinding state image feature information;

[0010] The control module is connected to the machine vision module and the robot grinding module, and is used to adjust the grinding equipment control instruction characteristics and grinding parameters in real time according to the surface feature information and the grinding state image feature information.

[0011] Furthermore, the machine vision module includes:

[0012] Visual sensor, used to collect multi-angle images of hardware. The collected information includes surface information and grinding status image information of hardware.

[0013] An image processing unit is used to filter and preprocess the collected images, extract and analyze the features after preprocessing, and generate surface feature information of the hardware and image feature information of the grinding state, wherein the surface feature information includes shape, size, defect location and surface roughness;

[0014] The specific implementation process of filtering preprocessing of the image includes:

[0015] The image acquired by the visual sensor, which includes the surface information of the hardware and the image information of the grinding state, is converted into digital form to obtain the three color channel information of RGB; the grayscale information is calculated according to the three color channel information of RGB to obtain the grayscale information of the image; the two-dimensional wavelet transform filtering is performed according to the grayscale information of the image to obtain the image data; wherein the process of the two-dimensional wavelet transform filtering includes calculating the two-dimensional wavelet transform function corresponding to the pixels on the image:

[0016] Where a is the scaling factor, x represents the horizontal coordinate value of the pixel in the two-dimensional coordinate system, y represents the vertical coordinate value of the pixel in the two-dimensional coordinate system, I(x,y) represents the wavelet transform result at the position (x,y), f represents the position parameter corresponding to x, g represents the position parameter corresponding to y, ψ * is the complex conjugate of the wavelet function, W(I,f,g) represents the wavelet transform filter function value at the (x,y) coordinate point; according to the wavelet transform filter function value, a two-dimensional convolution operation is performed with the image to obtain image filtering information; the two-dimensional convolution operation process is as follows:

[0017] Among them, g(x,y) is the image filtering information at the (x,y) coordinates, f(xi,yj) is the grayscale information at the (xi,yj) coordinates; k is half of the maximum value minus the minimum value of the horizontal coordinate of the coordinate system.

[0018] Furthermore, the control module includes:

[0019] A path planning unit, used to generate a grinding equipment control instruction feature according to the surface feature information combined with a target intelligent grinding decision algorithm, wherein the grinding equipment control instruction feature is used to characterize the equipment control instruction feature corresponding to the grinding path;

[0020] The adaptive adjustment unit is used to dynamically analyze the grinding state image feature information through the target deviation analysis algorithm according to the real-time feedback grinding state image feature information. If the grinding state is judged to be a deviation state, the grinding equipment control instruction characteristics and grinding parameters are dynamically adjusted according to the analysis results of the target deviation analysis algorithm.

[0021] Furthermore, the target intelligent grinding decision algorithm is debugged according to the following steps: obtaining a number of first visual sensing training data; the first visual sensing training data includes a first grinding task training label but does not include a second grinding task training label; obtaining a number of second visual sensing training data; the second visual sensing training data includes a second grinding task training label but does not include the first grinding task training label; debugging the initial intelligent grinding decision processing algorithm at least once according to the first visual sensing training data and the second visual sensing training data, until the first training error and the second training error meet the preset debugging requirements, thereby obtaining the target intelligent grinding decision algorithm; wherein the debugging process includes: The first visual sensing training data and a plurality of the second visual sensing training data are input into the initial intelligent grinding decision processing algorithm to obtain the first grinding equipment control training features of the corresponding first grinding tasks and the second grinding equipment control training features of the corresponding second grinding tasks; the first training error is determined according to the first grinding equipment control training features and the corresponding first grinding task training labels; the second training error is determined according to the second grinding equipment control training features and the corresponding second grinding task training labels; the algorithm weights of the initial intelligent grinding decision processing algorithm are optimized according to the first training error and the second training error, and the initial intelligent grinding decision processing algorithm after the optimized algorithm weights is used as the training data; The algorithm is used as the initial intelligent grinding decision processing algorithm before the next round of debugging processing; the obtaining of a plurality of first visual sensor training data includes: obtaining a plurality of initial visual sensor training data; wherein the initial visual sensor training data includes a first grinding task training label and a second grinding task training label; screening the data sets that do not conform to the preset rules of each initial visual sensor training data according to the set grinding process priority to obtain a plurality of corresponding first visual sensor training data; inputting the plurality of first visual sensor training data into the initial intelligent grinding decision processing algorithm to obtain the control training features of each first grinding equipment, including: for each first visual sensor training data, using the initial intelligent grinding decision processing algorithm to perform the control of the first visual sensor training data. The first visual sensor training data is subjected to multiple rounds of knowledge feature vector mining processing to obtain several first sensor monitoring training knowledge feature vector maps of feature pyramids; derivative knowledge feature vector mining is performed based on several first sensor monitoring training knowledge feature vector maps to obtain several second sensor monitoring training knowledge feature vector maps of feature pyramids; the first grinding equipment control training feature is determined based on the second sensor monitoring training knowledge feature vector map of the target feature pyramid; the first visual sensor training data and the second visual sensor training data are both taken from the surface feature information of the hardware, the first visual sensor training data represents the shallow surface features of the hardware, and the second visual sensor training data represents the deep surface features of the hardware.

[0022] Furthermore, the specific implementation process of the first visual sensing training data and the second visual sensing training data is:

[0023] Based on the trained feature dimensionality reduction mapping ResNeXt model, image feature extraction is performed on the hardware surface image obtained by the visual sensor to obtain the hardware surface image dimensionality reduction mapping features; wherein, the feature dimensionality reduction mapping ResNeXt model includes a shallow feature extraction unit and a deep feature extraction unit; the shallow feature extraction unit is used to perform shallow image feature extraction on the hardware surface image to generate first visual sensing training data; the deep feature extraction unit is used to perform deep image feature extraction on the hardware surface image based on the shallow image features obtained during the shallow image feature extraction to generate second visual sensing training data; the extraction accuracy of the deep image feature extraction is greater than the extraction accuracy of the shallow image feature extraction, and the feature dimensionality reduction mapping ResNeXt model is based on the ResNeXt network framework.

[0024] Furthermore, the specific implementation process of the target deviation analysis algorithm is as follows:

[0025] According to each grinding task event, grinding deviation discrimination operation is performed on the grinding state image feature information respectively to obtain the deviation discrimination suggestion feature corresponding to each grinding task event, and the grinding equipment control instruction feature and grinding parameter are dynamically adjusted according to the deviation discrimination suggestion feature, including: determining a first attention iteration branch from the target deviation analysis algorithm; the first attention iteration branch includes a spatiotemporal attention model corresponding to each grinding task event; through the spatiotemporal attention model corresponding to each grinding task event, a deviation discrimination operation is performed on the grinding state image description set to obtain the deviation discrimination suggestion feature corresponding to each grinding task event; a spatiotemporal attention model is used to determine the deviation discrimination suggestion feature corresponding to a grinding task event;

[0026] The grinding state image description set is obtained by performing an image description set mining operation on the grinding state image feature information by an image description mining module of a first attention iteration branch;

[0027] Obtain historical data of grinding task events in the database, and calculate the similarity value between the features of the grinding task events in the historical data and the features of the current grinding task events using the cosine similarity algorithm. If the similarity value meets the preset threshold, find the features of the corresponding grinding task events in the historical data, and convert the features of the corresponding grinding task events in the historical data into grinding equipment control instruction features and grinding parameters.

[0028] Furthermore, the robot grinding module comprises:

[0029] A force control sensor, used for real-time monitoring of the cutting force during the grinding process, and feeding back the cutting force information to the control module;

[0030] The robot actuator is used to perform grinding actions according to the instructions of the control module and adjust the grinding posture and feed speed in real time according to the cutting force information.

[0031] Furthermore, the control module further includes:

[0032] The grinding wheel wear detection unit is used to detect the wear status of the grinding wheel in real time according to the image acquisition information and the changes in the grinding parameters, and to issue an alarm or automatically adjust the grinding parameters when the grinding wheel wear exceeds a preset threshold. The grinding parameters include grinding speed, feed rate, cutting depth and grinding wheel speed.

[0033] Furthermore, the grinding wheel wear detection unit also includes:

[0034] Linear regression is used to predict the trend of grinding wheel wear;

[0035] Decision trees are used to identify conditions that may lead to grinding wheel wear problems;

[0036] The BR neural network is used to learn and predict complex grinding wheel wear patterns.

[0037] A method for intelligent grinding of hardware based on machine vision, the method is used to execute any one of the intelligent grinding systems for hardware based on machine vision, the method comprising the steps of:

[0038] S1. Collect images of the grinding of hardware parts through a machine vision module, and generate surface feature information of the hardware parts and image feature information of the grinding state;

[0039] S2, adjusting the control instruction characteristics and grinding parameters of the grinding equipment in real time through the control module according to the surface feature information and the grinding state image feature information;

[0040] S3, grinding the hardware by the robot grinding module according to the grinding equipment control instruction characteristics and grinding parameters of the control module;

[0041] S4. During the grinding process, the control module analyzes the grinding state image feature information according to the target deviation analysis algorithm, generates analysis results to dynamically adjust the grinding equipment control instruction features and grinding parameters, and performs wear detection on the grinding wheel.

[0042] Compared with the prior art, the embodiments of the present invention achieve the following beneficial effects:

[0043] The embodiment of the present invention provides a method for acquiring images including a machine vision module for hardware grinding, and generating surface feature information and grinding state image feature information of the hardware; a robot grinding module for grinding the hardware according to the surface feature information and the grinding state image feature information; a control module connected to the machine vision module and the robot grinding module, for adjusting the grinding equipment control instruction features and grinding parameters in real time according to the surface feature information and the grinding state image information; the present invention generates accurate path planning by performing image recognition on the target hardware grinding information and mining the image description set, covering multiple information such as shape, size, and defect location, and the path planning adjusts the grinding equipment control instruction features and grinding parameters in real time. For multiple grinding task events, deviation discrimination is performed respectively, so as to determine the deviation correction execution strategy, which effectively improves the grinding accuracy of hardware, reduces the scrap rate of hardware, and improves the production efficiency of hardware. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is an architecture diagram of a hardware intelligent grinding system based on machine vision provided by an embodiment of the present invention;

[0045] Figure 2 The present invention provides a flowchart of a method for intelligent grinding of hardware based on machine vision. DETAILED DESCRIPTION

[0046] The present invention will be described in detail below in conjunction with the accompanying drawings.

[0047] Example 1

[0048] The embodiment of the present invention provides a hardware intelligent grinding system based on machine vision, such as Figure 1 ,The system includes a machine vision module A1, a robot grinding module A3 and a control module A2;

[0049] Specifically, the machine vision module A1 of the system is used to collect images of the grinding of hardware parts and generate surface feature information and grinding state image feature information of the hardware parts; the robot grinding module A3 is used to grind the hardware parts according to the surface feature information and grinding state image feature information; the control module A2 is connected to the machine vision module A1 and the robot grinding module A3, and is used to adjust the grinding equipment control instruction characteristics and grinding parameters in real time according to the surface feature information and grinding state image feature information, and generate accurate path planning by performing image recognition on the target hardware grinding information and mining the image description set, covering various information such as shape, size, and defect location. The path planning adjusts the grinding equipment control instruction characteristics and grinding parameters in real time. For multiple grinding task events, deviation discrimination is performed separately to determine the execution strategy for correcting deviations. This effectively improves the grinding accuracy of hardware parts, reduces the scrap rate, and improves production efficiency.

[0050] In the above embodiment, specifically, the machine vision A1 module includes:

[0051] Visual sensor A4 is used to collect multi-angle images of hardware. The collected information includes the surface information and grinding status image information of the hardware.

[0052] It should be noted that the multi-angle image acquisition of the hardware is carried out by visual sensors, among which visual sensors are key components in modern industrial automation and intelligent manufacturing. They capture and analyze images through high-precision visual sensors to provide necessary visual information for the automation system. This module usually includes different types of cameras, such as CCD (charge-coupled device) cameras, CMOS (complementary metal oxide semiconductor) cameras and linear array cameras, each of which has unique advantages and application scenarios. CCD cameras are known for their high resolution, high sensitivity and excellent image quality, and are particularly suitable for applications that require high-quality image capture. CMOS cameras are favored for their fast imaging capabilities, low power consumption and cost-effectiveness, and are widely used in situations that require fast response. Linear array cameras are used for continuous object detection and measurement tasks with their expertise in high-speed scanning and precise measurement.

[0053] Exemplarily, in this embodiment, the visual sensor uses a CMOS high-definition infrared camera. The CMOS high-definition infrared camera images the hardware during multi-angle grinding and pre-processes the images. The obtained multi-angle grinding image data of the hardware contains detailed visual information of surface feature information and grinding status image information. This information can be used for further image processing, analysis and decision making. The CMOS high-definition infrared camera has advantages in processing low light intensity.

[0054] Image processing unit A5, used for filtering and preprocessing the collected images, extracting and analyzing the features after preprocessing, and generating surface feature information of the hardware and image feature information of the grinding state, wherein the surface feature information includes shape, size, defect location and surface roughness;

[0055] The specific implementation process of filtering preprocessing of the image includes:

[0056] The image acquired by the visual sensor, which includes the surface information of the hardware and the image information of the grinding state, is converted into digital form to obtain the three color channel information of RGB; the grayscale information is calculated according to the three color channel information of RGB to obtain the grayscale information of the image; the two-dimensional wavelet transform filtering is performed according to the grayscale information of the image to obtain the image data; wherein the process of the two-dimensional wavelet transform filtering includes calculating the two-dimensional wavelet transform function corresponding to the pixels on the image:

[0057] Where a is the scaling factor, x represents the horizontal coordinate value of the pixel in the two-dimensional coordinate system, y represents the vertical coordinate value of the pixel in the two-dimensional coordinate system, I(x,y) represents the wavelet transform result at the position (x,y), f represents the position parameter corresponding to x, g represents the position parameter corresponding to y, ψ * is the complex conjugate of the wavelet function, W(I,f,g) represents the wavelet transform filter function value at the (x,y) coordinate point; according to the wavelet transform filter function value, a two-dimensional convolution operation is performed with the image to obtain image filtering information; the two-dimensional convolution operation process is as follows:

[0058] Among them, g(x,y) is the image filtering information at the (x,y) coordinates, f(xi,yj) is the grayscale information at the (xi,yj) coordinates; k is half of the maximum value minus the minimum value of the horizontal coordinate of the coordinate system.

[0059] It should be noted that, according to the RGB three color channel information, the grayscale information is calculated to obtain the grayscale information of the image, and the grayscale information of the product is calculated by the following formula: Gra=0.298*R+0.584*G+0.117*B, where R represents the component value of the red channel, G represents the component value of the green channel, B represents the component value of the blue channel, and Gra represents the grayscale information of the image.

[0060] It should be noted that this embodiment uses two-dimensional wavelet transform filtering for image preprocessing, and bilateral filtering can also be used for image preprocessing. This method will not be described in detail here.

[0061] In the above embodiment, specifically, the control module A2 includes:

[0062] A path planning unit A6, used to generate a grinding equipment control instruction feature according to the surface feature information combined with a target intelligent grinding decision algorithm, wherein the grinding equipment control instruction feature is used to characterize the equipment control instruction feature corresponding to the grinding path;

[0063] The adaptive adjustment unit A7 is used to dynamically analyze the grinding state image feature information through the target deviation analysis algorithm according to the real-time feedback grinding state image feature information. If the grinding state is judged to be a deviation state, the grinding equipment control instruction characteristics and grinding parameters are dynamically adjusted according to the analysis results of the target deviation analysis algorithm.

[0064] In the above embodiment, specifically, the target intelligent grinding decision algorithm is debugged according to the following steps: obtaining a number of first visual sensing training data; the first visual sensing training data includes a first grinding task training label but does not include a second grinding task training label; obtaining a number of second visual sensing training data; the second visual sensing training data includes a second grinding task training label but does not include the first grinding task training label; debugging the initial intelligent grinding decision processing algorithm at least once according to the first visual sensing training data and the second visual sensing training data, until the first training error and the second training error meet the preset debugging requirements, and the target intelligent grinding decision algorithm is obtained; wherein the debugging process includes : Inputting a plurality of the first visual sensing training data and a plurality of the second visual sensing training data into the initial intelligent grinding decision processing algorithm, obtaining the first grinding equipment control training features of the corresponding first grinding tasks and the second grinding equipment control training features of the corresponding second grinding tasks; determining the first training error according to the first grinding equipment control training features and the corresponding first grinding task training labels; determining the second training error according to the second grinding equipment control training features and the corresponding second grinding task training labels; optimizing the algorithm weights of the initial intelligent grinding decision processing algorithm according to the first training error and the second training error, and using the initial intelligent grinding decision processing algorithm after optimizing the algorithm weights The method comprises the following steps: obtaining a plurality of first visual sensor training data; wherein the initial visual sensor training data comprises a first grinding task training label and a second grinding task training label; screening the data sets that do not conform to the preset rules of each initial visual sensor training data according to the set grinding process priority to obtain a plurality of corresponding first visual sensor training data; inputting the plurality of first visual sensor training data into the initial intelligent grinding decision processing algorithm to obtain the control training features of each first grinding equipment, comprising: for each first visual sensor training data, using the initial intelligent grinding decision processing algorithm to perform a first visual sensor training on the first visual sensor training data; The sensor training data is subjected to multiple rounds of knowledge feature vector mining processing to obtain a first sensor monitoring training knowledge feature vector map of several feature pyramids; derivative knowledge feature vector mining is performed based on the several first sensor monitoring training knowledge feature vector maps to obtain a second sensor monitoring training knowledge feature vector map of several feature pyramids; the first grinding equipment control training feature is determined based on the second sensor monitoring training knowledge feature vector map of the target feature pyramid; the first visual sensor training data and the second visual sensor training data are both taken from the surface feature information of the hardware, the first visual sensor training data represents the shallow surface features of the hardware, and the second visual sensor training data represents the deep surface features of the hardware.

[0065] It can be understood that the above debugging process of the target intelligent grinding decision algorithm includes the following two processes.

[0066] Process 1: Debugging basis of target intelligent grinding decision algorithm: data acquisition

[0067] 1) Acquisition of first visual sensor training data

[0068] Initial visual sensing training data: First, obtain some initial visual sensing training data, which are the basic data source for the entire debugging process. The initial visual sensing training data contains rich information, including the first grinding task training label and the second grinding task training label. For example, the initial visual sensing training data can be collected from a large number of hardware grinding historical records. For example, these historical records contain 10,000 sets of data points, each of which contains various visual sensing data in the grinding process, such as shape, size, defect location, surface roughness, etc., as well as the corresponding grinding task-related labels (first grinding task training label and second grinding task training label).

[0069] Screening to obtain the first visual sensor training data: According to the set grinding priority, the data sets that do not conform to the preset rules of each initial visual sensor training data are screened, and after being processed based on the trained feature dimension reduction mapping ResNeXt model, a number of corresponding first visual sensor training data are obtained. Setting the grinding priority is an indicator used to distinguish the importance or processing order of different grinding tasks. For example, the grinding priority is set to 0.6 (for example, the value range is 0 to 1). If the grinding priority of a grinding task in a certain initial visual sensor training data is less than this set value, it is screened out as the first visual sensor training data. For example, after screening, 6000 groups of first visual sensor training data are obtained, which only include the first grinding task training label and do not include the second grinding task training label. The purpose of this step is to separate the data related to a specific grinding task (the first grinding task) so that the intelligent grinding decision algorithm can be debugged for the first grinding task later.

[0070] 2) Acquisition of Second Vision Sensor Training Data

[0071] Corresponding to the acquisition of the first visual sensor training data, a number of second visual sensor training data are acquired. These data include the second grinding task training label but do not include the first grinding task training label. For example, from the remaining 4000 sets of initial visual sensor training data, 3000 sets of second visual sensor training data are obtained after being processed based on the trained feature dimension reduction mapping ResNeXt model. These data are specifically used to debug the intelligent grinding decision algorithm for the second grinding task.

[0072] Specifically, the specific implementation process of processing the first visual sensor training data and the second visual sensor training data based on the trained feature dimension reduction mapping ResNeXt model is:

[0073] Based on the trained feature dimensionality reduction mapping ResNeXt model, the initial visual sensing training data can be collected from a large number of hardware grinding historical records to perform image feature extraction, such as shape, size, defect location and surface roughness data, to obtain dimensionality reduction mapping features; wherein, the feature dimensionality reduction mapping ResNeXt model includes a shallow feature extraction unit and a deep feature extraction unit; the shallow feature extraction unit is used to perform shallow image feature extraction on the initial visual sensing training data to generate first visual sensing training data; the deep feature extraction unit is used to perform deep image feature extraction on the initial visual sensing training data based on the shallow image features obtained during the shallow image feature extraction to generate second visual sensing training data; the extraction accuracy of the deep image feature extraction is greater than the extraction accuracy of the shallow image feature extraction, and the feature dimensionality reduction mapping ResNeXt model is based on the ResNeXt network framework.

[0074] Process 2: Debugging process

[0075] 1) Input training data into the initial intelligent grinding decision algorithm

[0076] Input a plurality of first visual sensor training data and a plurality of second visual sensor training data into the initial intelligent grinding decision algorithm. The initial intelligent grinding decision algorithm can be an algorithm based on a neural network, such as a multi-layer perceptron (GMLP). For example, this GMLP has an input layer, a hidden layer, and an output layer. The number of nodes in the input layer is determined according to the number of features of the input sensor monitoring training data. For example, after the first visual sensor training data and the second visual sensor training data are preprocessed, each data point has 20 features, and then the input layer has 20 nodes. The hidden layer can be set to 2 layers, with 10 and 5 nodes in each layer respectively. The number of nodes in the output layer is determined according to the number of equipment control training features to be obtained, for example, 3 nodes (corresponding to different equipment control parameters).

[0077] When the first visual sensing training data is input, the first grinding equipment control training features of the corresponding first grinding tasks are obtained through GMLP calculation. Similarly, when the second visual sensing training data is input, the second grinding equipment control training features of the corresponding second grinding tasks are obtained. For example, for the first grinding task, the first grinding equipment control training features obtained may be the predicted values ​​of parameters such as the speed of the cutting device and the pressure of the drilling device; for the second grinding task, the second grinding equipment control training features obtained may be the predicted values ​​of parameters such as the rotation speed of the grinding device and the grinding pressure.

[0078] It should be noted that GMLP (Generative Model Learning Perceptron) is a neural network architecture based on a multi-layer perceptron (MLP), which aims to replace the traditional self-attention mechanism (such as the attention mechanism in Transformer) through gating mechanism and spatial projection to improve efficiency and performance.

[0079] 2) Determine the first training error and the second training error

[0080] Determination of the first training error: Determine the first training error based on each first grinding equipment control training feature and each corresponding first grinding task training label. Here, the mean square error (MSE) can be used as an error measurement indicator. Suppose the first grinding equipment control training feature is (D_{1i}=(d_{1i1}, d_{1i2}, ..., d_{1in}))((i=1, 2, ..., m), (m) is the number of first visual sensing training data, (n) is the dimension of the first grinding equipment control training feature), and the corresponding first grinding task training label is (K_{1i}=(k_{ki1}, l_{ki2}, ..., l_{kin})). Then the first training error can be: (M_1=frac{1}{m}sum_{i=1}^{m}sum_{j=1}^{n}(d_{1ij}-k_{1ij})^2). The meaning of this formula is to calculate the sum of square errors between each first grinding equipment control training feature and the corresponding training label, and then average them to obtain the first training error. For example, if (m=6000), (n=3), the value of the first training error is obtained by calculating the sum of square errors of each data point and averaging them.

[0081] Determination of the second training error: Similarly, the second training error is determined based on each second grinding equipment control training feature and each corresponding second grinding task training label. Assume that the second grinding equipment control training feature is (D_{2i}=(d_{2i1}, d_{2i2}, ..., d_{2in})), and the corresponding second grinding task training label is (K_{2i}=(k_{2i1}, k_{2i2}, ..., k_{2in}));

[0082] The second training error may be exemplarily expressed as: (M_2=frac{1}{k}sum_{i=1}^{k}sum_{j=1}^{n}(d_{2ij}-k_{2ij})^2) ((k) is the number of second visual sensor training data). For example, if (k=3000), (n=3), the value of the second training error is calculated according to the above formula.

[0083] 3) Optimize the algorithm weights of the initial intelligent grinding decision algorithm

[0084] The algorithm weights of the initial intelligent grinding decision algorithm are optimized based on the first training error and the second training error. In a neural network (such as MLP), the algorithm weights determine the mapping relationship from input data to output data. Taking the standard backpropagation algorithm as an example, based on the calculated first training error (M_1) and second training error (M_2), starting from the output layer, the error is backpropagated to adjust the connection weights between each layer.

[0085] Let (w_{ij}) be the connection weight from layer (i) to layer (j) in the neural network, and the learning rate be (eta) (e.g. (eta=0.01));

[0086] According to the formula of the back propagation algorithm (where (M=M_1+M_2), Represents the partial derivative function), calculates the updated amount of the weight (Back_{ij}), and then updates the weight to (w_{ij}=w_{ij}+Back_{ij}). By continuously adjusting the weight, the first training error and the second training error are gradually reduced.

[0087] The initial intelligent grinding decision algorithm after optimizing the algorithm weights is used as the initial intelligent grinding decision algorithm before the next round of debugging, and the above debugging process is repeated until the first training error and the second training error meet the set debugging standard requirements. The set debugging standard requirements can be a specific error threshold, such as (M_1leq0.05) and (M_2leq0.05). When this requirement is met, the target intelligent grinding decision algorithm is obtained, which can accurately obtain the equipment control decision features of the first grinding task and the second grinding task based on the input sensor monitoring data.

[0088] It can be seen that by screening training data according to different grinding task labels, the intelligent grinding decision algorithm can be debugged for specific grinding tasks (the first grinding task and the second grinding task). The use of mean square error to measure the debugging error can intuitively reflect the difference between the equipment control training characteristics and the training labels. The back propagation algorithm is used to optimize the algorithm weights according to the debugging error to continuously improve the accuracy of the algorithm. Repeat the debugging process until the set debugging requirements are met to ensure that the target intelligent grinding decision algorithm can make effective control decisions on the grinding equipment while meeting the accuracy requirements, thereby improving the work efficiency and quality of grinding.

[0089] Under another preferred design idea, a number of first visual sensor training data are input into the initial intelligent grinding decision algorithm to obtain each first grinding equipment control training feature, including: for each first visual sensor training data, the first visual sensor training data is subjected to multiple rounds of knowledge feature vector mining processing through the initial intelligent grinding decision algorithm to obtain a number of first sensor monitoring training knowledge feature vector spectra of feature pyramids; based on a number of first sensor monitoring training knowledge feature vector spectra, derived knowledge feature vector mining is performed to obtain a number of second sensor monitoring training knowledge feature vector spectra of feature pyramids; based on the second sensor monitoring training knowledge feature vector spectrum of the target feature pyramid, the first grinding equipment control training feature is determined.

[0090] The above design idea first processes multiple rounds of knowledge feature vector mining for each first visual sensor training data.

[0091] 1) Purpose and principle of multi-round knowledge feature vector mining

[0092] In the process of inputting a plurality of first visual sensing training data into the initial intelligent grinding decision algorithm to obtain each first grinding equipment control training feature, multiple rounds of knowledge feature vector mining processing are performed for each first visual sensing training data. The purpose of this multiple rounds of processing is to gradually extract a knowledge feature vector spectrum with more depth and semantic information from the original first visual sensing training data, so that the first grinding equipment control training feature can be determined more accurately later.

[0093] Its principle is based on the hierarchical abstraction of data. Each round of mining processing will further explore the hidden information in the data based on the previous one, which is similar to building a pyramid model, gradually building up higher-level knowledge representations from the bottom-level raw data.

[0094] 2) Example of multi-round knowledge feature vector mining process

[0095] For example, the first visual sensor training data is a vector containing visual sensor data, such as shape data (P), size data (M), defect location data (D) and surface roughness pressure data (F). In the first round of knowledge feature vector mining, the original data are firstly subjected to separate feature extraction, and these features are combined to form the first visual sensor training knowledge feature vector spectrum of the first feature pyramid.

[0096] Secondly, derived knowledge feature vector mining is performed based on several knowledge feature vector spectra of first visual sensor training data.

[0097] 1) Methods and significance of mining derived knowledge feature vectors

[0098] After obtaining the first visual sensing training knowledge feature vector spectra of several feature pyramids, the derived knowledge feature vector mining is carried out. The purpose of derived knowledge feature vector mining is to further mine more valuable information based on the existing knowledge feature vector spectra, generate new knowledge representation forms through data transformation and combination, and thus more comprehensively reflect the relationship between data and equipment control training features.

[0099] For example, a derivative knowledge feature vector mining method based on Fourier wavelet transform (STFT) can be used. Fourier wavelet transform can decompose the signal into components of different scales and frequencies, and is very effective in mining local features and trends in data.

[0100] 2) Specific operations of derived knowledge feature vector mining

[0101] Suppose the first visual sensing training knowledge feature vector spectrum is (Y_{k1}, Y_{k2}, ..., Y_{kn}) ((n) is the number of feature pyramid layers). For each (Y_{ki}), perform Fourier wavelet transform. Suppose the wavelet function is (psi(t)), perform Fourier wavelet transform on each element (y_{kij}) ((j=1, 2, ..., m), (m) is the dimension of the vector spectrum) in (Y_{ki}) to obtain the coefficients after Fourier wavelet transform (Q_{kij}). These coefficients constitute the derived knowledge feature vector spectrum. Then, the feature vector spectra after Fourier wavelet transform of different feature pyramids are combined to obtain the second visual sensing training knowledge feature vector spectra of several feature pyramids. For example, for the second visual sensing training knowledge feature vector spectrum of the second feature pyramid, it can be a combination of partial coefficients of the first visual sensing training knowledge feature vector spectrum of the first feature pyramid after Fourier wavelet transform and partial coefficients of the first visual sensing training knowledge feature vector spectrum of the second feature pyramid after Fourier wavelet transform.

[0102] Finally, the first grinding equipment control training features are determined according to the second visual sensing training knowledge feature vector spectrum of the target feature pyramid.

[0103] With this design, through multiple rounds of knowledge feature vector mining, different levels of information can be deeply mined from the original first visual sensor training data, and the knowledge feature vector spectrum of the pyramid model can be constructed to make the data representation more hierarchical and semantic. The derivative knowledge feature vector mining uses methods such as Fourier wavelet transform to further mine new knowledge representations, increasing the richness of data information. The first grinding equipment control training features are determined based on the second visual sensor training knowledge feature vector spectrum of the target feature pyramid, and the appropriate feature pyramid is selected through correlation analysis. The linear regression algorithm is used to determine the features, which improves the accuracy of equipment control training feature determination, thereby helping to improve the accuracy and effectiveness of grinding equipment control.

[0104] Specifically, the output of the trained feature dimensionality reduction mapping ResNeXt model is connected to the input of the feature pyramid, and the first visual sensor training data and the second visual sensor training data are given to the input of the feature pyramid through the output of the trained feature dimensionality reduction mapping ResNeXt model, and connected in a serial manner.

[0105] In the above embodiment, specifically, the specific implementation process of the target deviation analysis algorithm is:

[0106] According to each grinding task event, grinding deviation discrimination operation is performed on the grinding state image feature information respectively to obtain the deviation discrimination suggestion feature corresponding to each grinding task event, and the grinding equipment control instruction feature and grinding parameter are dynamically adjusted according to the deviation discrimination suggestion feature, including: determining a first attention iteration branch from the target deviation analysis algorithm; the first attention iteration branch includes a spatiotemporal attention model corresponding to each grinding task event; through the spatiotemporal attention model corresponding to each grinding task event, a deviation discrimination operation is performed on the grinding state image description set to obtain the deviation discrimination suggestion feature corresponding to each grinding task event; a spatiotemporal attention model is used to determine the deviation discrimination suggestion feature corresponding to a grinding task event;

[0107] The grinding state image description set is obtained by performing an image description set mining operation on the grinding state image feature information by an image description mining module of a first attention iteration branch;

[0108] Obtain historical data of grinding task events in the database, and calculate the similarity value between the features of the grinding task events in the historical data and the features of the current grinding task events using the cosine similarity algorithm. If the similarity value meets the preset threshold, find the features of the corresponding grinding task events in the historical data, and convert the features of the corresponding grinding task events in the historical data into grinding equipment control instruction features and grinding parameters.

[0109] It should be noted that, taking the hardware grinding task event as an example, the corresponding spatiotemporal attention model will focus on the part of the target grinding state image description set related to edge grinding. For example, the contour shape description set in the grinding state image description set shows that the edge of the hardware has an outward bulging deformation, and the spatiotemporal attention model will analyze this deformation according to predefined rules and algorithms.

[0110] For each metal grinding task event, the spatiotemporal attention model corresponding to the event performs deviation discrimination on the grinding state image description set as follows. For example, for a metal grinding task event, the spatiotemporal attention model constructs a feature vector (mbf{V}=[v_1, v_2,..., v_m]) based on the size description set, contour shape description set, and surface state description set of the hardware. This feature vector contains all the grinding state image description set information related to the task event. Then, the spatiotemporal attention model determines the deviation discrimination recommendation feature through a discriminant function (f(mbf{V})). For example, in the metal edge grinding task, if the discriminant function determines the depth (h) that the metal edge needs to be cut additionally based on the feature vector, this (h) is part of the deviation discrimination recommendation feature.

[0111] In the above embodiment, specifically, the robot grinding module A3 includes:

[0112] A force control sensor A9 is used to monitor the cutting force in the grinding process in real time and feed back the cutting force information to the control module;

[0113] The robot actuator A10 is used to perform grinding actions according to the instructions of the control module A2, and adjust the grinding posture and feed speed in real time according to the cutting force information.

[0114] In the above embodiment, specifically, the control module A2 further includes:

[0115] The grinding wheel wear detection unit A8 is used to detect the wear status of the grinding wheel in real time according to the image acquisition information and the changes in the grinding parameters, and to issue an alarm or automatically adjust the grinding parameters when the grinding wheel wear exceeds a preset threshold. The grinding parameters include grinding speed, feed rate, cutting depth and grinding wheel speed.

[0116] In the above embodiment, specifically, the grinding wheel wear detection unit A8 also includes:

[0117] Linear regression is used to predict the trend of grinding wheel wear;

[0118] Decision trees are used to identify conditions that may lead to grinding wheel wear problems;

[0119] The BR neural network is used to learn and predict complex grinding wheel wear patterns.

[0120] Example 2

[0121] A method for intelligent grinding of hardware based on machine vision, the method is used to execute any of the above systems, and the steps of the method include:

[0122] S1, collect images of the grinding of hardware through the machine vision module A1, and generate surface feature information of the hardware and image feature information of the grinding state;

[0123] S2, adjusting the grinding equipment control instruction characteristics and grinding parameters in real time through the control module A2 according to the surface feature information and the grinding state image feature information;

[0124] S3, grinding the hardware by the robot grinding module A3 according to the grinding equipment control instruction characteristics and grinding parameters of the control module A2;

[0125] S4. During the grinding process, the control module A2 analyzes the grinding state image feature information according to the target deviation analysis algorithm, generates analysis results to dynamically adjust the grinding equipment control instruction features and grinding parameters, and performs wear detection on the grinding wheel.

[0126] It should be understood that the above-mentioned embodiments are one or more embodiments of the present invention, and there are many other embodiments and variations thereof based on the present invention; the variations and modifications made by ordinary technicians in this industry through the present invention without making pioneering innovations all fall within the scope of protection of the present invention.

Claims

1. A hardware intelligent grinding system based on machine vision, characterized in that: include: The machine vision module is used to collect images of the hardware grinding and generate the surface feature information of the hardware and the image feature information of the grinding status; A robot grinding module, used for grinding the hardware according to the surface feature information and the grinding state image feature information; The control module is connected to the machine vision module and the robot grinding module, and is used to adjust the grinding equipment control instruction characteristics and grinding parameters in real time according to the surface feature information and the grinding state image feature information.

2. According to the machine vision-based intelligent hardware grinding system of claim 1, it is characterized in that: The machine vision module comprises: Visual sensor, used to collect multi-angle images of hardware. The collected information includes surface information and grinding status image information of hardware. An image processing unit is used to filter and preprocess the collected images, extract and analyze the features after preprocessing, and generate surface feature information of the hardware and image feature information of the grinding state, wherein the surface feature information includes shape, size, defect location and surface roughness; The specific implementation process of filtering preprocessing of the image includes: The image acquired by the visual sensor, which includes the surface information of the hardware and the image information of the grinding state, is converted into digital form to obtain the three color channel information of RGB; the grayscale information is calculated according to the three color channel information of RGB to obtain the grayscale information of the image; the two-dimensional wavelet transform filtering is performed according to the grayscale information of the image to obtain the image data; wherein the process of the two-dimensional wavelet transform filtering includes calculating the two-dimensional wavelet transform function corresponding to the pixels on the image: Where a is the scaling factor, x represents the horizontal coordinate value of the pixel in the two-dimensional coordinate system, y represents the vertical coordinate value of the pixel in the two-dimensional coordinate system, I(x,y) represents the wavelet transform result at the position (x,y), f represents the position parameter corresponding to x, g represents the position parameter corresponding to y, ψ * is the complex conjugate of the wavelet function, W(I,f,g) represents the wavelet transform filter function value at the (x,y) coordinate point; according to the wavelet transform filter function value, a two-dimensional convolution operation is performed with the image to obtain image filtering information; the two-dimensional convolution operation process is as follows: Among them, g(x,y) is the image filtering information at the (x,y) coordinates, f(xi,yj) is the grayscale information at the (xi,yj) coordinates; k is half of the maximum value minus the minimum value of the horizontal coordinate of the coordinate system.

3. According to the machine vision-based intelligent hardware grinding system of claim 1, it is characterized in that: The control module comprises: A path planning unit, used to generate a grinding equipment control instruction feature according to the surface feature information combined with a target intelligent grinding decision algorithm, wherein the grinding equipment control instruction feature is used to characterize the equipment control instruction feature corresponding to the grinding path; The adaptive adjustment unit is used to dynamically analyze the grinding state image feature information through the target deviation analysis algorithm according to the real-time feedback grinding state image feature information. If the grinding state is judged to be a deviation state, the grinding equipment control instruction characteristics and grinding parameters are dynamically adjusted according to the analysis results of the target deviation analysis algorithm.

4. According to claim 3, a hardware intelligent grinding system based on machine vision is characterized in that: The target intelligent grinding decision algorithm is debugged according to the following steps: obtaining a plurality of first visual sensing training data; the first visual sensing training data includes a first grinding task training label and does not include a second grinding task training label; obtaining a plurality of second visual sensing training data; the second visual sensing training data includes a second grinding task training label and does not include the first grinding task training label; The initial intelligent grinding decision processing algorithm is debugged at least once according to the first visual sensing training data and the second visual sensing training data until the first training error and the second training error meet the preset debugging requirements, thereby obtaining the target intelligent grinding decision algorithm; wherein the debugging process includes: inputting a plurality of the first visual sensing training data and a plurality of the second visual sensing training data into the initial intelligent grinding decision processing algorithm to obtain the first grinding equipment control training features of the corresponding first grinding tasks and the second grinding equipment control training features of the corresponding second grinding tasks; determining the first training error according to the first grinding equipment control training features and the corresponding first grinding task training labels; determining the second training error according to the second grinding equipment control training features and the corresponding second grinding task training labels; optimizing the algorithm weights of the initial intelligent grinding decision processing algorithm according to the first training error and the second training error, and using the initial intelligent grinding decision processing algorithm after optimizing the algorithm weights as the initial intelligent grinding decision processing algorithm before the next round of debugging; the obtaining of a plurality of the first visual sensing training data includes: obtaining a plurality of initial visual sensing training data; wherein the initial visual The sensor training data includes a first grinding task training label and a second grinding task training label; according to the set grinding process priority, the data sets that do not conform to the preset rules of each initial visual sensor training data are screened to obtain a number of corresponding first visual sensor training data; the first visual sensor training data are input into the initial intelligent grinding decision processing algorithm to obtain the control training features of each first grinding equipment, including: for each first visual sensor training data, the first visual sensor training data is subjected to multiple rounds of knowledge feature vector mining processing by the initial intelligent grinding decision processing algorithm to obtain the first sensor monitoring data of a number of feature pyramids. measuring training knowledge feature vector maps; performing derivative knowledge feature vector mining based on several first sensor monitoring training knowledge feature vector maps to obtain several feature pyramid second sensor monitoring training knowledge feature vector maps; determining the first grinding equipment control training features based on the second sensor monitoring training knowledge feature vector map of the target feature pyramid; the first visual sensor training data and the second visual sensor training data are both taken from the surface feature information of the hardware, the first visual sensor training data represents the shallow features of the surface of the hardware, and the second visual sensor training data represents the deep features of the surface of the hardware.

5. According to claim 3, the intelligent hardware grinding system based on machine vision is characterized in that: The specific implementation process of the target deviation analysis algorithm is as follows: According to each grinding task event, grinding deviation discrimination operation is performed on the grinding state image feature information respectively to obtain deviation discrimination suggestion features corresponding to each grinding task event. Dynamically adjusting the grinding equipment control instruction features and grinding parameters according to the deviation discrimination suggestion features, including: determining a first attention iteration branch from the target deviation analysis algorithm; the first attention iteration branch includes a spatiotemporal attention model corresponding to each grinding task event; performing a deviation discrimination operation on a grinding state image description set through the spatiotemporal attention model corresponding to each grinding task event, and obtaining the deviation discrimination suggestion features corresponding to each grinding task event; a spatiotemporal attention model is used to determine a deviation discrimination suggestion feature corresponding to a grinding task event; The grinding state image description set is obtained by performing an image description set mining operation on the grinding state image feature information by an image description mining module of a first attention iteration branch; Obtain historical data of grinding task events in the database, and calculate the similarity value between the features of the grinding task events in the historical data and the features of the current grinding task events using the cosine similarity algorithm. If the similarity value meets the preset threshold, find the features of the corresponding grinding task events in the historical data, and convert the features of the corresponding grinding task events in the historical data into grinding equipment control instruction features and grinding parameters.

6. The intelligent hardware grinding system based on machine vision according to claim 1 is characterized in that: The robot grinding module comprises: A force control sensor, used for real-time monitoring of the cutting force during the grinding process, and feeding back the cutting force information to the control module; The robot actuator is used to perform grinding actions according to the instructions of the control module and adjust the grinding posture and feed speed in real time according to the cutting force information.

7. The intelligent hardware grinding system based on machine vision according to claim 1 is characterized in that: The control module also includes: The grinding wheel wear detection unit is used to detect the wear status of the grinding wheel in real time according to the image acquisition information and the changes in the grinding parameters, and to issue an alarm or automatically adjust the grinding parameters when the grinding wheel wear exceeds a preset threshold. The grinding parameters include grinding speed, feed rate, cutting depth and grinding wheel speed.

8. The intelligent hardware grinding system based on machine vision according to claim 7 is characterized in that: The grinding wheel wear detection unit also includes: Linear regression is used to predict the trend of grinding wheel wear; Decision trees are used to identify conditions that may lead to grinding wheel wear problems; The BR neural network is used to learn and predict complex grinding wheel wear patterns.

9. A hardware intelligent grinding method based on machine vision, characterized in that: The method is used to execute the system according to any one of claims 1 to 8, and the steps of the method include: S1. Collect images of the grinding of hardware parts through a machine vision module, and generate surface feature information of the hardware parts and image feature information of the grinding state; S2, adjusting the control instruction characteristics and grinding parameters of the grinding equipment in real time through the control module according to the surface feature information and the grinding state image feature information; S3, grinding the hardware by the robot grinding module according to the grinding equipment control instruction characteristics and grinding parameters of the control module; S4. During the grinding process, the control module analyzes the grinding state image feature information according to the target deviation analysis algorithm, generates analysis results to dynamically adjust the grinding equipment control instruction features and grinding parameters, and performs wear detection on the grinding wheel.

Images