An electric glass through-hole etching monitoring robot and monitoring method and system thereof

Abstract

The present invention proposes an electric glass through hole etching monitoring robot and a monitoring method and system thereof, which belong to the field of glass through hole etching monitoring. The method includes: S1, the monitoring robot uses a vehicle-mounted chassis and a tire motor to move in the monitoring area, and obtains multi-angle original images of the electric glass through hole etching device in real time through a camera; S2, performs glass through hole image detection on the multi-angle original image, and classifies according to the results of the glass through hole image detection to obtain the category information of the glass through hole; S3, diagnoses and analyzes the etching degree of the glass through hole according to the category information of the glass through hole, and obtains glass through hole etching information; S4, feeds back the glass through hole etching information to the user, and the user controls the electric glass through hole etching device according to the glass through hole etching information. The present application realizes automatic, intelligent and all-round real-time monitoring of the glass through hole etching device by autonomously moving the vehicle-mounted chassis and tire motor of the monitoring robot in the monitoring area, improves the flexibility and efficiency of monitoring, and provides high-precision image recognition and intelligent diagnosis, effectively improving the monitoring efficiency and accuracy.

Description

Technical Field

[0001] The invention relates to the technical field of through-glass hole etching monitoring, and in particular to an electric through-glass hole etching monitoring robot and a monitoring method and system thereof. Background Art

[0002] Glass through-hole etching is a key process in the field of microelectronics and microfluidics. Currently, the mainstream technology combines ultrafast laser processing and chemical etching. First, an ultrafast laser is used to create a tiny modified area on the glass surface, and then a low concentration of hydrofluoric acid is used for selective etching to form the desired through-hole structure. The etching process is affected by multiple factors, including the uniformity of the glass material, slight fluctuations in laser parameters, and environmental conditions (such as temperature and humidity). These factors cause through-holes in different locations or different batches to show significant differences even under the same laser parameters.

[0003] The existing quality control methods mainly rely on real-time observation and control by humans through computers. The operator needs to continuously monitor the etching process, judge the formation of through holes based on experience, and adjust the etching parameters or terminate the process in a timely manner. Even if image processing and machine learning technologies are used to assist the detection process, since the monitoring system is mostly passive, it is difficult to achieve timely optimization and adjustment of etching parameters, and it is difficult to adapt to glass samples of different sizes and shapes.

[0004] Therefore, finding a monitoring method that can not only automatically move and monitor the etching situation in real time, but also automatically adjust the position of the camera is a technical problem that needs to be solved urgently by technical personnel in this field. Summary of the invention

[0005] In view of this, the present invention proposes an electric glass through-hole etching monitoring robot and a monitoring method and system thereof, which can realize automatic and intelligent all-round real-time monitoring, provide high-precision image recognition and intelligent diagnosis, greatly improve the monitoring efficiency and accuracy, and have strong adaptability and scalability, can adapt to different monitoring environments, and effectively improve the monitoring efficiency and accuracy.

[0006] The technical solution of the present invention is achieved in this way:

[0007] In a first aspect, the present invention provides a monitoring method based on an electric through-glass via etching monitoring robot, comprising the following steps:

[0008] S1. The monitoring robot uses the vehicle chassis and tire motor to move in the monitoring area, and obtains multi-angle original images of the electric glass through-hole etching device in real time through the camera;

[0009] S2, performing through-glass hole image detection on the multi-angle original images, and classifying the through-glass hole images according to the detection results to obtain category information of the through-glass holes;

[0010] S3, diagnosing and analyzing the etching degree of the through-glass via according to the category information of the through-glass via to obtain through-glass via etching information;

[0011] S4. Feedback the through glass hole etching information to the user, and the user controls the electric through glass hole etching device according to the through glass hole etching information.

[0012] On the basis of the above technical solution, preferably, step S1 specifically includes:

[0013] S11, the monitoring robot uses the vehicle chassis and tire motors to move in the monitoring area, uses the sensors of the monitoring robot to obtain environmental data, and pre-processes the environmental data to obtain processed environmental data;

[0014] S12, performing position estimation on the processed environmental data to obtain the current position of the monitoring robot;

[0015] S13, constructing an environmental map of the monitoring area according to the current position of the monitoring robot and the processed environmental data;

[0016] S14, performing path planning according to the environmental map of the monitoring area to obtain a moving trajectory of the monitoring robot, and the monitoring robot moves according to the moving trajectory;

[0017] S15. Control the camera of the monitoring robot through an adaptive control algorithm according to the moving trajectory of the monitoring robot to obtain multi-angle original images of the electric glass through-hole etching device.

[0018] On the basis of the above technical solution, preferably, the path planning specifically includes:

[0019] Step 1: Use the grid method to divide the environmental map of the monitoring area into a matrix model, initialize each grid as a neuron, and use the biologically inspired neural network algorithm to traverse the environmental map of the monitoring area;

[0020] Step 2: Perform static offline planning based on the environmental map of the monitoring area and calculate the initial path;

[0021] Step 3: Calculate the activity values ​​of the surrounding connected neurons according to the current position of the monitoring robot, select the adjacent neuron with the largest activity value as the next step, and set the activity value of the current neuron to 0;

[0022] Step 4: Repeat step 3 until the activity values ​​of the peripheral neurons of the monitoring robot are all 0, and the moving trajectory of the monitoring robot is obtained.

[0023] On the basis of the above technical solution, preferably, the adaptive control algorithm specifically includes:

[0024] The current position of the monitoring robot and the map of the monitoring area are fuzzy processed, and a control signal is generated according to the fuzzy rules, and the viewing angle of the camera is adjusted according to the control signal.

[0025] More preferably, the through-glass hole image detection specifically includes:

[0026] Preprocessing the original images from multiple angles to obtain processed images;

[0027] The convolutional layer of the convolutional neural network is used to extract features from the processed image to obtain local features. The convolutional neural network includes multiple convolutional layers, and the feature extraction formula is:

[0028]

[0029] in, represents the output of the (i, j)th position in the lth convolutional layer, f(·) represents the activation function, represents the convolution kernel weight at the (m, n)th position of the lth layer, represents the input of the (i+m, j+n)th position in the l-1th layer, b (l) Represents the bias of the lth layer, M represents the number of rows of the convolution kernel, N represents the number of columns of the convolution kernel, i represents the i-th row index in the feature map, j represents the j-th column index in the feature map, m represents the row index in the convolution kernel, and n represents the column index in the convolution kernel;

[0030] The local features are input into the pooling layer of the convolutional neural network for downsampling to obtain the pooled features;

[0031] The pooled features are input into the fully connected layer of the convolutional neural network for mapping to obtain the results of glass through-hole image detection.

[0032] Based on the above technical solution, preferably, the diagnostic analysis in step S3 specifically includes:

[0033] Constructing a through-glass via state Bayesian network and defining a conditional probability table for each node of the through-glass via state Bayesian network;

[0034] Calculate the joint probability distribution based on the through-glass via state Bayesian network and conditional probability table;

[0035] Evaluate the state probability of the through-glass hole based on the observed data and the joint probability distribution;

[0036] The state probability of through-glass vias is fuzzified using fuzzy logic.

[0037] On the basis of the above technical solution, preferably, the fuzzy processing specifically includes:

[0038] Fuzzify the state probability of the through-glass hole to obtain a fuzzy set;

[0039] Reasoning the fuzzy set according to fuzzy rules to obtain a fuzzy output;

[0040] The fuzzy output is defuzzified to obtain an accurate value.

[0041] On the basis of the above technical solution, preferably, the defuzzification formula is:

[0042]

[0043] μ Ci (z i )=min(μ A (x),μ B (y))

[0044]

[0045] Among them, μ Ci (z i ) represents the fuzzy set membership function, μ A (x) represents the membership degree of fuzzy set A, μ B (y) represents the degree of membership of the fuzzy set B, a and b represent the range of the fuzzy set A, c and d represent the range of the fuzzy set B, x represents the first parameter affecting the through-glass state, and y represents the second parameter affecting the through-glass state.

[0046] In a second aspect, the present invention provides a monitoring system based on an electric glass through hole etching monitoring robot, using the monitoring method as described above, comprising:

[0047] An image acquisition module is configured such that the monitoring robot moves in the monitoring area using the vehicle chassis and tire motors, and acquires multi-angle original images of the electric glass through-hole etching device in real time through a camera;

[0048] An image detection module configured to perform through-glass hole image detection on the multi-angle original images, and classify the through-glass hole images according to the detection results to obtain category information of the through-glass hole;

[0049] A diagnosis and analysis module configured to perform a diagnosis and analysis on the etching degree of the through glass via according to the category information of the through glass via to obtain through glass via etching information;

[0050] The information feedback module is configured to feed back the through glass via etching information to a user, and the user controls the electric through glass via etching device according to the through glass via etching information.

[0051] In a third aspect, the present invention provides an electric through-glass hole etching monitoring robot, which is applied to the monitoring method as described above, and includes a vehicle chassis, a driving device, a through-glass hole image acquisition device, a controller and a camera motion device, wherein:

[0052] The vehicle chassis includes wheel hubs and tires, which are powered by tire motors to support the free movement of the monitoring robot;

[0053] The driving device is powered by a battery box and supports the movement monitoring and filming monitoring of the robot;

[0054] The through-glass hole image acquisition device includes a camera movement device and an infrared sensor integration device;

[0055] The controller is a Raspberry Pi controller, which uses infrared sensors to achieve automatic guidance of the indoor floor;

[0056] The camera motion device includes a battery buckle, a DC motor assembly, a worm gear, a camera, a worm, a vertical rotation bracket, a vertical rotation servo, a horizontal rotation bracket, a horizontal rotation servo and a pan / tilt column. The camera motion device can perform vertical movement, vertical rotation movement and horizontal rotation movement.

[0057] The monitoring method of the present invention has the following beneficial effects compared with the prior art:

[0058] (1) By monitoring the robot's onboard chassis and tire motors moving autonomously in the monitoring area, the through-glass hole etching device can be monitored in an automated, intelligent, and all-round real-time manner, improving the flexibility and efficiency of monitoring. It also provides high-precision image recognition and intelligent diagnosis, effectively improving monitoring efficiency and accuracy.

[0059] (2) Through the grid method and bio-inspired neural network algorithm, efficient path planning and dynamic adjustment capabilities are achieved. The original images at different angles are captured by the monitoring robot’s path planning, and the monitoring robot’s camera is adjusted in time according to the monitoring robot’s moving trajectory through the adaptive control algorithm, thereby improving the flexibility of the monitoring robot, enabling the monitoring robot to adapt to different monitoring environments, and enhancing the reliability and accuracy of monitoring the electric glass etching device.

[0060] (3) Through multi-layer convolution and pooling operations, the subtle features of glass through-holes are accurately captured, greatly improving the detection accuracy. The nonlinear mapping of the fully connected layer is used to enhance the monitoring robot's ability to handle complex scenes. It can adaptively learn different types of glass through-hole images and improve the efficiency of glass through-hole image detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0062] Figure 1 is a flow chart of the monitoring method of the present invention;

[0063] Figure 2 It is a front view of the electric glass through-hole etching monitoring robot of the present invention;

[0064] Figure 3 It is a left view of the electric glass through-hole etching monitoring robot of the present invention;

[0065] Figure 4 An axonometric diagram of the electric glass through-hole etching monitoring robot of the present invention;

[0066] Figure 5 It is the wiring diagram of the infrared sensor of the electric glass through-hole etching monitoring robot of the present invention;

[0067] Figure 6 The wiring diagram of the Raspberry Pi and the steering gear of the electric through-glass hole etching monitoring robot of the present invention;

[0068] Figure 7 The wiring diagram of the Raspberry Pi, the motor and the driver board of the electric glass through hole etching monitoring robot of the present invention;

[0069] Figure 8 This is a control flow chart of the electric glass through-hole etching monitoring robot of the present invention. DETAILED DESCRIPTION

[0070] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] like Figure 1As shown, the present invention provides a monitoring method based on an electric glass through hole etching monitoring robot, comprising the following steps:

[0072] S1. The monitoring robot uses the vehicle chassis and tire motors to move in the monitoring area, and obtains multi-angle original images of the electric glass through-hole etching device in real time through the camera.

[0073] Specifically, step S1 specifically includes:

[0074] S11, the monitoring robot uses the vehicle chassis and tire motors to move in the monitoring area, uses the sensors of the monitoring robot to obtain environmental data, and pre-processes the environmental data to obtain processed environmental data;

[0075] S12, performing position estimation on the processed environmental data to obtain the current position of the monitoring robot;

[0076] S13, constructing an environmental map of the monitoring area according to the current position of the monitoring robot and the processed environmental data;

[0077] S14, performing path planning according to the environmental map of the monitoring area to obtain a moving trajectory of the monitoring robot, and the monitoring robot moves according to the moving trajectory;

[0078] S15. Control the camera of the monitoring robot through an adaptive control algorithm according to the moving trajectory of the monitoring robot to obtain multi-angle original images of the electric glass through-hole etching device.

[0079] The monitoring robot uses the vehicle-mounted chassis and tire motors to move autonomously in the monitoring area, enabling the monitoring robot to cover a larger monitoring range, improving the flexibility and efficiency of monitoring, and being able to fully observe the etching process, capture original images from different angles, and use environmental perception, precise positioning, and intelligent path planning to ensure the accuracy and efficiency of detection.

[0080] In one embodiment of the present application, environmental data is acquired using a variety of sensors such as a laser radar (LiDAR), a camera, and an infrared sensor.

[0081] In one embodiment of the present application, the environmental data is preprocessed, specifically including:

[0082] Filter, denoise and extract features from sensor data. For example, use a Kalman filter to smooth lidar data and reduce measurement noise. The calculation formula is as follows:

[0083]

[0084] P k =AP k-1 A T+Q

[0085] in, represents the state prediction at time k, A represents the state transfer matrix, represents the state estimation at the previous moment, B represents the control input matrix, u k represents the control input at time k, P k represents the prediction error covariance at time k, P k-1 represents the error covariance of the previous moment, A T represents the transpose of the state transfer matrix, Q represents the process noise covariance;

[0086] Use a median filter to denoise the camera image and reduce optical noise;

[0087] Use the ORB (Oriented FAST and Rotated BRIEF) algorithm to extract feature points from camera images.

[0088] In an embodiment of the present application, the position estimation specifically includes:

[0089] Use algorithms such as particle filtering and extended Kalman filtering (EKF) to estimate the current position of the monitoring robot. For example, sample multiple possible positions of the monitoring robot and obtain the most likely position through weighted averaging. The calculation formula is as follows:

[0090] p(x t |z t ,u t )∝p(z t |x t )∫p(x t |u t , x t-1 )p(x t-1 )dx t-1

[0091] Among them, p(x t |z t ,u t ) represents the state posterior probability at time t given the observed data zt and the control input ut, p(z t |x t ) represents the probability of observing data zt in state xt, p(x t |u t , x t-1 ) means that the state at the previous moment is x t-1 The probability of transferring to the current state xt when the control input is ut, p(x t-1 ) represents the posterior probability of the state at the previous moment;

[0092] Using the nonlinear state transfer and measurement model, the robot state is recursively estimated and monitored. The calculation formula is as follows:

[0093]

[0094] K=P k -1H T (HP k-1 H T +R) -1

[0095] in, represents the state estimation at time k, K represents the Kalman gain, z k represents the observation value at time k, H represents the observation matrix, represents the state estimation at the previous moment, P k-1 represents the error covariance of the previous moment, H T represents the transpose of the observation matrix, and R represents the observation noise covariance.

[0096] In one embodiment of the present application, constructing an environmental map of the monitoring area specifically includes:

[0097] Use the occupancy grid to represent obstacles and free areas in the environment. The calculation formula is as follows:

[0098]

[0099] Where m(x, y) represents the occupancy status at the coordinate (x, y) in the grid map, 1 means that the grid is occupied, and 0 means that the grid is not occupied;

[0100] A sparse feature map is constructed using the extracted feature points.

[0101] In one embodiment of the present application, closed-loop monitoring is also performed on the monitoring robot according to the environmental map of the monitored fishing area to detect whether the monitoring robot has returned to the known unknown and correct the accumulated error, specifically including:

[0102] Use the RANSAC (Random Sampling Consensus) algorithm to match the current feature points with the historical feature points;

[0103] If the match is successful, the ICP (Iterative Closest Point) algorithm is used to correct the position of the monitoring robot to reduce the drift error.

[0104] Furthermore, the path planning specifically includes:

[0105] Step 1: Use the grid method to divide the environmental map of the monitoring area into a matrix model, initialize each grid as a neuron, and use the biologically inspired neural network algorithm to traverse the environmental map of the monitoring area;

[0106] Step 2: Perform static offline planning based on the environmental map of the monitoring area and calculate the initial path;

[0107] Step 3: Calculate the activity values ​​of the surrounding connected neurons according to the current position of the monitoring robot, select the adjacent neuron with the largest activity value as the next step, and set the activity value of the current neuron to 0;

[0108] Step 4: Repeat step 3 until the activity values ​​of the peripheral neurons of the monitoring robot are all 0, and the moving trajectory of the monitoring robot is obtained.

[0109] Efficient path planning and dynamic adjustment capabilities are achieved through the grid method and bio-inspired neural network algorithm. By planning the path of the monitoring robot to capture original images at different angles, the monitoring robot can effectively adapt to complex environments, avoid local optimal solutions, and ensure comprehensive coverage of the monitoring area.

[0110] Specifically, the path planning is described with a specific embodiment:

[0111] a. Use the grid method to divide the environmental map of the monitoring area into a matrix model, initialize each grid as a neuron, set the initial activity value to 0, and mark the known obstacles and drivable areas;

[0112] b. Based on known environmental information, static offline planning is performed to calculate the initial path with the best time or energy consumption;

[0113] c. Set the drivable area as neuron excitation input and the obstacle area as neuron inhibition input;

[0114] d. Set the starting point and the end point, and activate the starting point neuron. Starting from the starting point, calculate the activity values ​​of the surrounding connected neurons, select the adjacent neuron with the largest activity value as the next step, and set the activity value of the current neuron to 0;

[0115] Repeat step d until the end point is reached or the activity values ​​of the surrounding neurons are all 0, and the moving trajectory of the monitoring robot is obtained.

[0116] In one embodiment of the present application, the activity values ​​of the surrounding neurons of the monitoring robot are all 0, and the monitoring robot has not reached the end of the environmental map of the monitoring area, then:

[0117] Check whether there are any neurons with an activity value of 0 in the environmental map of the monitoring area. If so, use the point-to-point planning method to move to the nearest non-zero neuron and continue the path.

[0118] In one embodiment of the present application, during the process of monitoring the movement of the robot, sensors are continuously used to detect the environment of the monitoring area. If new obstacles are found in the monitoring area, the inhibitory input of neurons in the environmental map of the monitoring area is updated to dynamically adjust the path planning.

[0119] Furthermore, the adaptive control algorithm specifically includes:

[0120] The current position of the monitoring robot and the map of the monitoring area are fuzzy processed, and a control signal is generated according to the fuzzy rules, and the viewing angle of the camera is adjusted according to the control signal.

[0121] Through fuzzy processing and fuzzy rules, the monitoring robot can intelligently adapt to different monitoring environments and conditions, improving the flexibility and robustness of the monitoring robot.

[0122] S2. Perform through-glass hole image detection on the multi-angle original images, and classify them according to the through-glass hole image detection results to obtain category information of the through-glass holes.

[0123] Specifically, the through-glass hole image detection specifically includes:

[0124] Preprocessing the original images from multiple angles to obtain processed images;

[0125] The convolutional layer of the convolutional neural network is used to extract features from the processed image to obtain local features. The convolutional neural network includes multiple convolutional layers, and the feature extraction formula is:

[0126]

[0127] in, represents the output of the (i, j)th position in the lth convolutional layer, f represents the activation function, represents the convolution kernel weight at the (m, n)th position of the lth layer, represents the input of the (i+m, j+n)th position in the l-1th layer, b (l) Represents the bias of the lth layer, M represents the number of rows of the convolution kernel, N represents the number of columns of the convolution kernel, i represents the i-th row index in the feature map, j represents the j-th column index in the feature map, m represents the row index in the convolution kernel, and n represents the column index in the convolution kernel;

[0128] The local features are input into the pooling layer of the convolutional neural network for downsampling to obtain the pooled features;

[0129] The local features and pooled features are input into the fully connected layer of the convolutional neural network for mapping to obtain the results of glass through-hole image detection.

[0130] The preprocessing includes image enhancement and image normalization. The diversity of data is enhanced by performing enhancement operations such as rotation, scaling, and translation on the original images at multiple angles, and the image pixels are normalized to the range of [0, 1] to improve the stability and efficiency of glass through-hole image detection.

[0131] Through multi-layer convolution and pooling operations, the subtle features of glass through-holes can be accurately captured, greatly improving the detection accuracy. The nonlinear mapping of the fully connected layer is used to enhance the monitoring robot's ability to handle complex scenes. It can adaptively learn different types of glass through-hole images and improve the efficiency of glass through-hole image detection, significantly improving the monitoring quality and efficiency of the monitoring robot.

[0132] Specifically, the pooling layer of the convolutional neural network performs downsampling, takes the maximum value in the pooling window, and obtains the pooling feature. The calculation formula is as follows:

[0133]

[0134] in, It represents the value of the output feature map of the lth layer in the pooling layer at the position (i, j). Represents the value of the local feature map of the l-1th layer at the position (i+m,j+n);

[0135] In one embodiment of the present application, the calculation formula of the fully connected layer is:

[0136]

[0137] in, represents the i-th element of the output vector of the l-th layer in the fully connected layer, represents the jth element of the input vector of the l-1th layer in the fully connected layer, represents the weight of the i-th row and j-th column in the weight matrix of the l-th layer, represents the i-th element of the bias vector of the l-th layer, and f(·) represents the activation function.

[0138] S3, diagnosing and analyzing the etching degree of the through glass hole according to the category information of the through glass hole to obtain through glass hole etching information; the through glass hole etching information includes the through glass hole etching amount and the through glass hole position number;

[0139] Specifically, the diagnostic analysis in step S3 specifically includes:

[0140] Constructing a through-glass via state Bayesian network and defining a conditional probability table for each node of the through-glass via state Bayesian network;

[0141] Calculate the joint probability distribution based on the through-glass via state Bayesian network and conditional probability table;

[0142] Evaluate the state probability of the through-glass hole based on the observed data and the joint probability distribution;

[0143] The state probability of through-glass vias is fuzzified using fuzzy logic.

[0144] As can be understood, a graphical model is established based on the glass through hole state, describing how various factors affect the etching state of the glass through hole. Each node represents a variable (such as etching time, temperature, etc.), and the arrows represent the dependencies between variables. The conditional probability table defines the probability distribution of each node under different states of its parent node.

[0145] The uncertainty in the through-glass via etching process is effectively handled through the Bayesian network, and precise probabilistic reasoning is performed by calculating the joint probability distribution to improve the accuracy and comprehensiveness of the through-glass via status diagnosis. Combined with fuzzy logic, the monitoring robot can better adapt to different etching conditions and standards to diagnose and analyze the etching degree of the through-glass via.

[0146] The calculation formula of the joint probability distribution is:

[0147]

[0148] Where P(X i |Parents(X i )) represents the probability of node Xi under the condition of its parent node.

[0149] Specifically, step S3 is described with a specific embodiment:

[0150] Construct a Bayesian network, including the following nodes:

[0151] Etching time (T): short / medium / long, temperature (Temp): low / medium / high, chemical concentration (C): low / medium / high, etching degree (E): mild / medium / severe;

[0152] Then the network structure is: T→E←Temp←C;

[0153] The conditional probability is expressed as: P(E|T, Temp) = {"mild": 0.7 (if T = short and Temp = low), "moderate": 0.6 (if T = medium and Temp = medium), "severe": 0.8 (if T = long and Temp = high)};

[0154] Calculate the joint probability distribution: P(T, Temp, C, E) = P(E|T, Temp)*P(Temp|C)*P(T)*P(C);

[0155] Assume that T = medium, Temp = medium, C = medium are observed, and use Bayesian inference to calculate P(E|T = medium, Temp = medium, C = medium):

[0156] If P(E = "mild") > 0.6, then it is determined as "mild etching";

[0157] If 0.4 < P(E = "moderate") ≤ 0.6, then it is determined as "moderate etching";

[0158] If P(E = "severe") > 0.5, then it is determined as "severe etching".

[0159] If the calculation result is P(E = "moderate"|observed data) = 0.55, after fuzzy processing, the system outputs "The glass through-hole is currently in a moderate etching state".

[0160] Furthermore, the fuzzy processing specifically includes:

[0161] Fuzzify the state probability of the glass through-hole to obtain a fuzzy set;

[0162] Infer the fuzzy set according to the fuzzy rules to obtain a fuzzy output;

[0163] Defuzzify the fuzzy output to obtain an exact value.

[0164] By converting the exact probability value into a fuzzy set, then obtaining the result through fuzzy inference, and finally defuzzifying to obtain the exact value, it realizes the perfect combination of precise numerical analysis and flexible decision-making, and uses the fuzzy inference rules to establish a non-linear mapping relationship between the input and the output, better reflecting the complexity of the etching process, enabling the monitoring robot to effectively combine the precision of quantitative analysis and the flexibility of qualitative judgment, and enhancing the reliability and accuracy of the electric glass etching monitoring.

[0165] Even further, the formula for defuzzification is:

[0166]

[0167] Wherein, represents the membership function of the fuzzy set, μ A (x) represents the membership degree of the fuzzy set A, μ B (y) represents the membership degree of the fuzzy set B, a and b represent the range of the fuzzy set A, c and d represent the range of the fuzzy set B, x represents the first parameter affecting the state of the glass through-hole, and y represents the second parameter affecting the state of the glass through-hole.

[0168] It can be understood that x and y can represent the through hole diameter, depth or other measurable characteristics, and can also represent etching time, temperature or other factors affecting the etching process. They are set according to actual conditions and this application does not impose specific limitations on this.

[0169] S4. Feedback the through glass hole etching information to the user, and the user controls the electric through glass hole etching device according to the through glass hole etching information.

[0170] It is understandable that the through glass hole etching information is fed back to the user in a timely manner to remind the user to remove the etching device from the corresponding position in time to avoid excessive etching.

[0171] This application realizes automated and intelligent all-round real-time monitoring through monitoring robots, provides high-precision image recognition and intelligent diagnosis, greatly improves monitoring efficiency and accuracy, and has strong adaptability and scalability, can adapt to different monitoring environments, and effectively improves monitoring efficiency and accuracy.

[0172] In one embodiment of the present application, the monitoring method further includes:

[0173] The actual position and angle of the monitoring robot camera are acquired in real time through the monitoring robot's sensors;

[0174] Calculate the target position and angle of the monitoring robot's camera based on the monitoring robot's positioning result;

[0175] The error between the actual position and angle of the monitoring robot camera and the target position and angle is calculated, and the position and angle of the monitoring robot camera is adjusted according to the error. The adjustment of the position and angle of the monitoring robot camera includes adjusting the viewing angles of the monitoring robot camera in the vertical, horizontal and vertical directions.

[0176] Specifically, the calculation formula for adjusting the monitoring robot camera in the vertical direction is as follows:

[0177] z ′ =z1+Δz

[0178] Among them, z ′ represents the vertical position of the camera after adjustment, z1 represents the vertical position of the camera before adjustment, and Δz represents the error between the actual position of the monitoring robot camera and the target position;

[0179] The calculation formula for adjusting the monitoring robot camera in the horizontal direction is as follows:

[0180] θ ′ h =θ h +Δθ h

[0181] Among them, θ′h Indicates the horizontal rotation angle of the camera after adjustment, θ h Indicates the horizontal rotation angle of the camera before adjustment, Δθ h Indicates the horizontal rotation angle error between the actual position of the monitoring robot camera and the target position;

[0182] The calculation formula for adjusting the monitoring robot camera in the vertical direction is as follows:

[0183] θ′ v =θ v +Δθ v

[0184] Among them, θ′ v Indicates the vertical rotation angle of the camera after adjustment, θ v Indicates the vertical rotation angle of the camera before adjustment, Δθ v Indicates the vertical rotation angle error between the actual position of the monitoring robot camera and the target position.

[0185] In one embodiment of the present application, the vertical adjustment range is: Δz min ≤Δz≤Δz max , Δz min Indicates the minimum adjustment range in the vertical direction, Δz max Indicates the maximum adjustment range in the vertical direction;

[0186] The horizontal rotation adjustment range is: Indicates the minimum adjustment range of horizontal rotation. Indicates the maximum adjustment range of horizontal rotation;

[0187] The vertical rotation adjustment range is: Indicates the minimum adjustment range of vertical rotation. Indicates the maximum adjustment range of vertical rotation.

[0188] Furthermore, adaptive fuzzy control is used to adjust the position and angle of the monitoring robot camera.

[0189] Specifically, the error between the actual position and angle of the monitoring robot camera and the target position and angle is converted into a fuzzy variable; the calculation formula is:

[0190] e z =Z target -z1

[0191]

[0192] Among them, e z Represents the positioning error in the vertical direction, e h represents the positioning error of horizontal rotation, e vIndicates the positioning error of vertical rotation;

[0193] The membership function is used to calculate the fuzziness of the error between the actual position and angle of the monitoring robot camera and the target position and angle, and a control decision is generated based on the fuzzy rules; the fuzziness includes the membership of the vertical error. Membership of horizontal rotation error And the membership of the vertical rotation error

[0194] Convert control decisions into specific adjustments.

[0195] In one embodiment of the present application, the error between the actual position of the camera and the target position is calculated in real time;

[0196] Generate a control signal based on the error to adjust the camera position and angle, where the calculation formula is:

[0197]

[0198] Among them, u z Represents the control signal in the vertical direction, u h Indicates the control signal for horizontal rotation, u v Indicates the control signal of vertical rotation, K p Represents proportional gain, K i represents the integral gain;

[0199] Apply control signals to the camera motion device to adjust the position and angle.

[0200] The present invention provides a monitoring system based on an electric glass through hole etching monitoring robot, which adopts the above-mentioned monitoring method and comprises:

[0201] An image acquisition module is configured such that the monitoring robot moves in the monitoring area using the vehicle chassis and tire motors, and acquires multi-angle original images of the electric glass through-hole etching device in real time through a camera;

[0202] An image detection module configured to perform through-glass hole image detection on the multi-angle original images, and classify the through-glass hole images according to the detection results to obtain category information of the through-glass hole;

[0203] A diagnosis and analysis module configured to perform a diagnosis and analysis on the etching degree of the through glass via according to the category information of the through glass via to obtain through glass via etching information;

[0204] The information feedback module is configured to feed back the through glass via etching information to a user, and the user controls the electric through glass via etching device according to the through glass via etching information.

[0205] like Figure 2 , Figure 3 , Figure 4 and Figure 5 As shown, the present invention provides an electric glass through hole etching monitoring robot, which is applied to the monitoring method as described above, including a vehicle chassis 31, a driving device 33, a glass through hole image acquisition device 34, a controller and a camera movement device 35, wherein:

[0206] The vehicle chassis 31 includes a wheel hub 15 and a tire 22, which is powered by a tire motor 23 to support the free movement of the monitoring robot;

[0207] The driving device 33 is powered by the battery box 24 and supports the monitoring of the robot's movement and filming monitoring;

[0208] The through-glass hole image acquisition device 34 includes a camera movement device 35 and an infrared sensor integration device 36;

[0209] The controller is a Raspberry Pi controller, which uses infrared sensors to achieve automatic guidance of the indoor floor;

[0210] The camera motion device 35 includes a battery buckle 2, a DC motor assembly 3, a worm gear 6, a camera 7, a worm 9, a vertical rotation bracket 17, a vertical rotation servo 18, a horizontal rotation bracket 19, a horizontal rotation servo 26 and a pan-tilt column 20. The camera motion device 35 can perform vertical movement, vertical rotation movement and horizontal rotation movement.

[0211] In one embodiment of the present application, four wheel hubs 15 and four tires 22 are attached to the vehicle chassis 31, and four tire motors 23 provide power to realize the free movement of the monitoring robot; the entire vehicle chassis 31 is supported by the upper chassis 10 and the lower chassis 13, and the upper chassis 10 is supported by four chassis columns 21 on the lower chassis 13, and is fastened by four upper chassis M3 screws 1; the infrared sensor 14 is installed on the lower chassis 13 by the lower chassis M2.5 screws 12.

[0212] In one embodiment of the present application, the DC motor assembly is composed of a battery buckle 2, a DC motor front component 3, a DC motor support frame 4, a DC motor rear component 5, and a DC motor shaft 8.

[0213] In the embodiment of the present application, the vertical movement of the camera relies on the power support provided by the DC motor assembly, and then the rotational motion of the motor is converted into vertical motion by shifting the worm wheel 6 and the worm 9, thereby realizing the vertical movement of the entire camera; the worm gear mechanism is often used to transmit the motion and power between the two staggered axes to realize the lifting and lowering movement of the camera.

[0214] The power for the camera's rotational movement in the vertical direction comes from the vertical rotating servo 18, which is installed on the vertical rotating bracket 17; the power for the camera's rotational movement in the horizontal direction comes from the horizontal rotating servo 26, which is installed on the horizontal rotating bracket 19; the camera movement device 35 can perform vertical movement, vertical rotation movement and horizontal rotation movement.

[0215] See also Figure 2 In one embodiment of the present application, the Raspberry Pi integration 32 includes a Raspberry Pi controller 11, an expansion board 16 and a Raspberry Pi bracket 25. The Raspberry Pi controller 11 is installed on the expansion board 16, and is installed on the lower chassis 13 through the Raspberry Pi bracket 25. The infrared sensor is used to realize the automatic guidance function of the monitoring robot on the indoor ground.

[0216] like Figure 6 and Figure 7 As shown, in one embodiment of the present application, the Raspberry Pi is connected to a servo, wherein the servo is a position (angle) servo driver, which is suitable for control systems that require angles to be constantly changed and maintained. The servo can receive digital signals or analog signals, corresponding to the swing mode and the turn mode, respectively, and then generate a PPM control signal for the servo, so that the servo swings back and forth or rotates to a set angle and stops.

[0217] An infrared sensor 14 is used to realize the automatic guidance function of the monitoring robot on the indoor floor, and a USB camera 7 is used to realize the acquisition of glass through-hole images. The camera bracket provides support for the camera and reserves two steering degrees of freedom. Two servos are connected to the bracket to provide steering power in the directions of two degrees of freedom respectively, so as to realize multi-angle glass through-hole image acquisition.

[0218] The through-glass hole image acquisition device 34 can acquire and record images of through-glass holes at multiple angles through the camera 7; the two electric pan-tilts cooperate to realize the horizontal and vertical rotation of the camera 7, and cooperate with the wide-angle camera to realize the feature pickup in a large range. The horizontal rotation in the horizontal plane uses the horizontal rotation servo 26 to connect with the sleeve, and the sleeve and the pan-tilt 1 have an interference fit; the pan-tilt 1 and the pan-tilt 2 need to rotate relative to each other, and are connected by a pin shaft; the vertical rotation in the vertical plane uses the vertical rotation servo 18 to connect with the pan-tilt 2 using screws and plastic connectors.

[0219] During operation, the PTZ generally moves in the horizontal and vertical planes at ±90° and ±45° respectively. The data tape on the Raspberry Pi wide-angle camera passes through the strip hole above the PTZ 2 to connect to the Raspberry Pi at the rear to achieve real-time monitoring and detection.

[0220] like Figure 8As shown, in the embodiment of the present application, the monitoring robot glass through hole etching image acquisition interface includes manual control mode, camera parameter settings, etc. Through the Python language programming of the Raspberry Pi, the glass through hole image in the glass through hole etching device is acquired and monitored, and the movement, tracking, and photo-taking functions of the monitoring robot are controlled. The openov library function is called to control the USB camera to obtain the image processing of the glass through hole morphology. In order to meet the needs of users who do not understand programming, the tkiter library function is used to design a graphical user interface (GU1)

[0221] After starting manual control, you can press WASD in the manual control window to control the movement of the monitoring robot. After starting automatic control, the monitoring robot will move along the black line track. Click Stop Automatic to terminate the automatic tracking of the monitoring robot. After turning on the camera, a window will appear to display the content shot by the monitoring robot. After clicking the camera reset, the camera position returns to the initial state and records its own position in the program. Click the rotation button with the mouse to rotate the camera in the specified direction. The rotation limit position is limited by the program.

Claims

1. A monitoring method based on an electric glass through hole etching monitoring robot, characterized in that: The following steps are involved: S1. The monitoring robot uses the vehicle chassis and tire motor to move in the monitoring area, and obtains multi-angle original images of the electric glass through-hole etching device in real time through the camera; S2, performing through-glass hole image detection on the multi-angle original images, and classifying the through-glass hole images according to the detection results to obtain category information of the through-glass holes; S3, diagnosing and analyzing the etching degree of the through-glass via according to the category information of the through-glass via to obtain through-glass via etching information; S4, feeding back the through-glass hole etching information to the user, and the user controls the electric through-glass hole etching device according to the through-glass hole etching information; The through-glass hole image detection specifically includes: Preprocessing the original images from multiple angles to obtain processed images; The convolutional layer of the convolutional neural network is used to extract features from the processed image to obtain local features. The convolutional neural network includes multiple convolutional layers, and the feature extraction formula is: in, represents the output of the (i, j)th position in the lth convolutional layer, f(·) represents the activation function, represents the convolution kernel weight at the (m, n)th position of the lth layer, represents the input of the (i+m, j+n)th position in the l-1th layer, b (l) Represents the bias of the lth layer, M represents the number of rows of the convolution kernel, N represents the number of columns of the convolution kernel, i represents the i-th row index in the feature map, j represents the j-th column index in the feature map, m represents the row index in the convolution kernel, and n represents the column index in the convolution kernel; Input the local features into the pooling layer of the convolutional neural network for downsampling to obtain the pooled features; The pooled features are input into the fully connected layer of the convolutional neural network for mapping to obtain the results of glass through-hole image detection; The diagnostic analysis in step S3 specifically includes: Constructing a through-glass via state Bayesian network and defining a conditional probability table for each node of the through-glass via state Bayesian network; Calculate the joint probability distribution based on the through-glass via state Bayesian network and conditional probability table; Evaluate the state probability of the through-glass hole based on the observed data and the joint probability distribution; Fuzzy logic is used to perform fuzzy processing on the state probability of through-glass holes; The fuzzy processing specifically includes: Fuzzify the state probability of the through-glass hole to obtain a fuzzy set; Reasoning the fuzzy set according to fuzzy rules to obtain a fuzzy output; Defuzzifying the fuzzy output to obtain an accurate value; The defuzzification formula is: in, represents the fuzzy set membership function, μ A (x) represents the membership degree of fuzzy set A, μ B (y) represents the membership degree of the fuzzy set B, a and b represent the range of the fuzzy set A, c and d represent the range of the fuzzy set B, x represents the first parameter affecting the through-glass state, and y represents the second parameter affecting the through-glass state.

2. A monitoring method based on an electric through-glass hole etching monitoring robot as claimed in claim 1, characterized in that: Step S1 specifically includes: S11, the monitoring robot uses the vehicle chassis and tire motors to move in the monitoring area, uses the sensors of the monitoring robot to obtain environmental data, and pre-processes the environmental data to obtain processed environmental data; S12, performing position estimation on the processed environmental data to obtain the current position of the monitoring robot; S13, constructing an environmental map of the monitoring area according to the current position of the monitoring robot and the processed environmental data; S14, performing path planning according to the environmental map of the monitoring area to obtain a moving trajectory of the monitoring robot, and the monitoring robot moves according to the moving trajectory; S15. Control the camera of the monitoring robot through an adaptive control algorithm according to the moving trajectory of the monitoring robot to obtain multi-angle original images of the electric glass through-hole etching device.

3. A monitoring method based on an electric through-glass hole etching monitoring robot as claimed in claim 2, characterized in that: The path planning specifically includes: Step 1: Use the grid method to divide the environmental map of the monitoring area into a matrix model, initialize each grid as a neuron, and use the biologically inspired neural network algorithm to traverse the environmental map of the monitoring area; Step 2: Perform static offline planning based on the environmental map of the monitoring area and calculate the initial path; Step 3: Calculate the activity values ​​of the surrounding connected neurons according to the current position of the monitoring robot, select the adjacent neuron with the largest activity value as the next step, and set the activity value of the current neuron to 0; Step 4: Repeat step 3 until the activity values ​​of the peripheral neurons of the monitoring robot are all 0, and the moving trajectory of the monitoring robot is obtained.

4. A monitoring method based on an electric glass through hole etching monitoring robot as claimed in claim 3, characterized in that: The adaptive control algorithm specifically includes: The current position of the monitoring robot and the map of the monitoring area are fuzzy processed, and a control signal is generated according to the fuzzy rule, and the viewing angle of the camera is adjusted according to the control signal.

5. A monitoring system based on an electric glass through hole etching monitoring robot, characterized in that: The monitoring method according to any one of claims 1 to 4 is adopted, comprising: An image acquisition module is configured such that the monitoring robot moves in the monitoring area using the vehicle chassis and tire motors, and acquires multi-angle original images of the electric glass through-hole etching device in real time through a camera; An image detection module configured to perform through-glass hole image detection on the multi-angle original images, and classify the through-glass hole images according to the detection results to obtain category information of the through-glass hole; A diagnosis and analysis module configured to perform a diagnosis and analysis on the etching degree of the through glass via according to the category information of the through glass via to obtain through glass via etching information; The information feedback module is configured to feed back the through glass via etching information to a user, and the user controls the electric through glass via etching device according to the through glass via etching information.

6. An electric glass through-hole etching monitoring robot, characterized in that: The monitoring method applied to any one of claims 1 to 4 comprises a vehicle chassis, a driving device, a through-glass hole image acquisition device, a controller and a camera motion device, wherein: The vehicle chassis includes wheel hubs and tires, which are powered by tire motors to support the free movement of the monitoring robot; The driving device is powered by a battery box and supports the movement monitoring and filming monitoring of the robot; The through-glass hole image acquisition device includes a camera movement device and an infrared sensor integration device; The controller is a Raspberry Pi controller, which uses infrared sensors to achieve automatic guidance of the indoor floor; The camera motion device includes a battery buckle, a DC motor assembly, a worm gear, a camera, a worm, a vertical rotation bracket, a vertical rotation servo, a horizontal rotation bracket, a horizontal rotation servo and a pan / tilt column. The camera motion device can perform vertical movement, vertical rotation movement and horizontal rotation movement.

Images