Intelligent spraying control system and method based on computer vision technology

Abstract

The invention relates to the field of intelligent control, and particularly discloses an intelligent spraying control system and method based on a computer vision technology. An artificial intelligence technology based on the deep learning field is used to carry out feature extraction and coding on a monitoring video for removing impurities on the surface of a base material in a preset time period, air knife airflow velocity values at a plurality of preset time points in the preset time period and distance values between an air knife and a spraying surface; and obtaining a result that the airflow speed of the air knife at the current time point needs to be increased or decreased. Therefore, the efficiency of cleaning the substrate surface by the air knife can be improved, and the cleaning effect is ensured.

Description

Technical Field

[0001] The present application relates to the field of intelligent control, and more specifically to an intelligent spraying control system and method based on computer vision technology. Background Art

[0002] Air knives are highly efficient industrial equipment that are widely used in water removal and dust control. They remove moisture and dust from the surface of objects by blowing out strong, high-speed airflow. In the spraying process, air knives can be used for water removal and dust removal to ensure the quality of spraying. Before spraying, the use of air knives can effectively remove dust and impurities on the surface of the substrate, which can prevent particles and unevenness in the paint film. The surface removed by the air knife is cleaner, which helps to improve the adhesion of the paint, thereby improving the appearance quality and durability of the product after spraying. In the traditional spraying process, the operator adjusts the distance between the air knife and the spraying surface, the air flow speed and direction based on experience to achieve the best water removal and dust removal effect. However, this traditional control method, in which the operator needs to adjust the parameters of the air knife based on experience, may lead to inefficiency and inconsistent results.

[0003] Therefore, an optimized spraying control solution is expected. Summary of the invention

[0004] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides an intelligent spray control system and method based on computer vision technology, which uses artificial intelligence technology based on the field of deep learning to extract and encode the monitoring video of impurity removal on the surface of the substrate within a predetermined time period, as well as the air knife air flow velocity values ​​and the distance values ​​between the air knife and the spraying surface at multiple predetermined time points within the predetermined time period, so as to obtain the result that the air flow velocity of the air knife should be increased or decreased at the current time point. In this way, the efficiency of the air knife in cleaning the surface of the substrate can be improved, and the cleaning effect can be guaranteed.

[0005] According to one aspect of the present application, there is provided an intelligent spraying control method based on computer vision technology, which includes:

[0006] Acquire a monitoring video of impurity removal from the substrate surface within a predetermined time period, as well as air knife airflow velocity values ​​and distance values ​​between the air knife and the spraying surface at multiple predetermined time points within the predetermined time period;

[0007] Processing the monitoring video of impurity removal from the substrate surface to obtain a feature vector of the spatial structure change of the substrate;

[0008] Processing the air knife airflow velocity value and the distance value between the air knife and the spraying surface to obtain an air knife feature vector;

[0009] The substrate spatial structure change feature vector and the air knife feature vector are comprehensively analyzed to obtain a result indicating whether the airflow velocity of the air knife should be increased or decreased at the current time point.

[0010] According to another aspect of the present application, there is also provided an intelligent spraying control system based on computer vision technology, which includes:

[0011] A spray substrate cleaning data acquisition module is used to acquire a monitoring video of impurity removal on the substrate surface within a predetermined time period, as well as air knife air flow velocity values ​​and distance values ​​between the air knife and the spray surface at multiple predetermined time points within the predetermined time period;

[0012] An impurity removal monitoring feature analysis module is used to process the impurity removal monitoring video on the substrate surface to obtain a feature vector of the spatial structure change of the substrate;

[0013] An air knife parameter characteristic analysis module is used to process the air knife air flow velocity value and the distance value between the air knife and the spraying surface to obtain an air knife characteristic vector;

[0014] The airflow velocity control module is used to comprehensively analyze the characteristic vector of the spatial structure change of the substrate and the characteristic vector of the air knife to obtain a result indicating whether the airflow velocity of the air knife should be increased or decreased at the current time point.

[0015] Compared with the prior art, the intelligent spray control system and method based on computer vision technology provided by the present application uses artificial intelligence technology based on the field of deep learning to extract and encode the monitoring video of impurity removal on the surface of the substrate within a predetermined time period, as well as the air knife air flow velocity values ​​and the distance values ​​between the air knife and the spraying surface at multiple predetermined time points within the predetermined time period, so as to obtain the result that the air knife air flow velocity should be increased or decreased at the current time point. In this way, the efficiency of the air knife in cleaning the surface of the substrate can be improved, and the cleaning effect can be guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0017] Figure 1 It is a flow chart of an intelligent spraying control method based on computer vision technology according to an embodiment of the present application.

[0018] Figure 2A flowchart of obtaining a feature vector of the spatial structure change of the substrate is provided for processing the monitoring video of impurity removal from the substrate surface in the intelligent spray control method based on computer vision technology according to an embodiment of the present application.

[0019] Figure 3 A flow chart of obtaining an air knife feature vector is provided for processing the air knife airflow velocity value and the distance value between the air knife and the spraying surface in the intelligent spraying control method based on computer vision technology according to an embodiment of the present application.

[0020] Figure 4 It is a block diagram of an intelligent spray control system based on computer vision technology according to an embodiment of the present application.

[0021] Figure 5 is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0023] Figure 1 Flow chart of the intelligent spraying control method based on computer vision technology according to an embodiment of the present application. Figure 1 As shown, according to the embodiment of the present application, the intelligent spraying control method based on computer vision technology includes: S110, obtaining a monitoring video of impurity removal on the surface of the substrate within a predetermined time period and the air knife airflow velocity values ​​and the distance values ​​between the air knife and the spraying surface at multiple predetermined time points within the predetermined time period; S120, processing the monitoring video of impurity removal on the surface of the substrate to obtain a characteristic vector of the spatial structure change of the substrate; S130, processing the air knife airflow velocity values ​​and the distance values ​​between the air knife and the spraying surface to obtain an air knife characteristic vector; and S140, comprehensively analyzing the substrate spatial structure change characteristic vector and the air knife characteristic vector to obtain a result indicating that the airflow velocity of the air knife at the current time point should be increased or decreased.

[0024] In step S110, a monitoring video of impurity removal from the substrate surface within a predetermined time period and the air knife airflow velocity values ​​and the distance values ​​between the air knife and the spraying surface at multiple predetermined time points within the predetermined time period are obtained. As mentioned in the above background technology, in the traditional spraying process, the operator adjusts the distance between the air knife and the spraying surface, the airflow velocity and direction based on experience to achieve the best water removal and dust removal effect. However, this traditional control method requires the operator to adjust the parameters of the air knife based on experience, which may lead to low efficiency and inconsistent effects. Therefore, an optimized spraying control solution is expected.

[0025] In response to the above technical problems, an intelligent spraying control solution based on computer vision technology is proposed, which uses artificial intelligence technology based on the field of deep learning to extract and encode the monitoring video of impurity removal on the substrate surface within a predetermined time period, as well as the air knife air flow velocity values ​​and the distance values ​​between the air knife and the spraying surface at multiple predetermined time points within the predetermined time period, so as to obtain the result that the air knife air flow velocity should be increased or decreased at the current time point. In this way, the efficiency of the air knife in cleaning the substrate surface can be improved and the cleaning effect can be guaranteed.

[0026] More specifically, in the technical solution of the present application, first, using computer vision technology, the system can monitor the impurity removal on the surface of the substrate in real time, and the captured images can be used for subsequent analysis and decision-making. Then, using a deep learning model for feature extraction, such as a convolutional neural network (CNN), the features of the impurities can be automatically extracted from the monitoring video, including shape, size, color, and texture. At the same time, the system not only analyzes the monitoring video, but also combines multiple parameters such as the air knife airflow velocity and the distance between the air knife and the spraying surface for comprehensive analysis to determine the optimal airflow velocity. Based on the feature extraction and encoding of deep learning, the system can intelligently determine whether the airflow velocity of the air knife at the current time point needs to be increased or decreased to meet different cleaning requirements. Through automated intelligent decision-making, the interference of human factors is reduced and the efficiency of the air knife cleaning the substrate surface is improved. In this way, the system can adjust the air knife parameters in real time to adapt to the changing production conditions, thereby ensuring the consistency and reliability of the cleaning effect. At the same time, the intelligent control system can adjust the airflow velocity according to actual needs, avoid unnecessary energy consumption, and achieve energy-saving production. The deep learning model has a good generalization ability, can adapt to different substrate types and impurity conditions, and improve the adaptability and flexibility of the system. This solution can be integrated with the existing spraying production line to achieve automation and intelligent upgrades.

[0027] Specifically, first, obtain the monitoring video of impurity removal on the surface of the substrate within a predetermined time period, as well as the air knife airflow velocity values ​​and the distance values ​​between the air knife and the spraying surface at multiple predetermined time points within the predetermined time period. The monitoring video and air knife parameter data can provide real-time information on the surface state and cleaning effect of the substrate during the spraying process. The method for obtaining the above data includes: 1. Video monitoring system: Install a high-speed camera to capture the impurity removal process on the surface of the substrate to ensure that the video clarity and frame rate meet the analysis requirements. 2. Sensor technology: Use high-precision airflow velocity sensors and distance sensors to monitor the working status of the air knife in real time.

[0028] When collecting data, you need to pay attention to the following: 1. Data synchronization: Ensure the synchronous collection of video data and air knife parameter data to accurately associate time and events. 2. Data quality: Ensure the quality of video and sensor data to avoid misreading due to equipment failure or interference. 3. Environmental factors: Consider the impact of environmental factors such as light, temperature, humidity, etc. on data collection, and take corresponding measures. 4. Data security: Ensure the security and privacy of data, especially when video data is involved. 5. Data storage: Consider the storage and management of data, because video data and sensor data can be very large. 6. Real-time: Ensure the real-time performance of the data acquisition system to meet the needs of real-time monitoring and control. 7. Calibration and maintenance: Regularly calibrate sensors and camera equipment to ensure the accuracy and reliability of data.

[0029] In step S120, the substrate surface impurity removal monitoring video is processed to obtain a substrate spatial structure change feature vector.

[0030] Figure 2 A flowchart of processing the substrate surface impurity removal monitoring video in the intelligent spraying control method based on computer vision technology according to the embodiment of the present application to obtain the substrate spatial structure change feature vector. Figure 2 As shown, the substrate surface impurity removal monitoring video is processed to obtain a substrate spatial structure change feature vector, including: S121, extracting multiple substrate surface impurity removal key frames from the substrate surface impurity removal monitoring video; S122, passing the multiple substrate surface impurity removal key frames through a substrate surface spatial feature extraction module to obtain multiple substrate spatial structure feature maps; S123, passing the multiple substrate spatial structure feature maps through a substrate surface time feature extraction module to obtain the substrate spatial structure change feature vector.

[0031] More specifically, multiple substrate surface impurity removal key frames are extracted from the substrate surface impurity removal monitoring video. The substrate surface impurity removal key frames represent the most important information moments in the video. By extracting these frames, the essence of the entire substrate surface impurity removal monitoring video can be condensed and the amount of data processing can be reduced. The substrate surface impurity removal key frames often contain important visual information of the impurity removal process, which is convenient for subsequent feature extraction and analysis. In real-time monitoring and control, processing the entire video sequence may be computationally intensive and time-consuming, while the substrate surface impurity removal key frames can quickly provide the necessary visual information. The substrate surface impurity removal key frames usually represent the most important changes in the visual scene, similar to the focus of human attention, which helps the model to concentrate on learning important features. Compared with processing the entire video stream, analyzing the substrate surface impurity removal key frames can significantly reduce the computational complexity and resource consumption. In the substrate surface impurity removal monitoring video, extracting the substrate surface impurity removal key frames can improve the efficiency of subsequent image processing and analysis.

[0032] Specifically, in the above-mentioned intelligent spray control method based on computer vision technology, multiple substrate surface impurity removal key frames are extracted from the substrate surface impurity removal monitoring video, including: extracting multiple substrate surface impurity removal key frames from the substrate surface impurity removal monitoring video at a predetermined sampling frequency.

[0033] More specifically, the multiple substrate surface impurity removal key frames are respectively passed through the substrate surface spatial feature extraction module to obtain multiple substrate spatial structure feature maps, and the substrate surface spatial feature extraction module is a convolutional neural network model using a spatial attention mechanism. The spatial attention mechanism is a technology that allows the model to adaptively focus on specific spatial regions in input data (such as images). In a convolutional neural network (CNN), spatial attention can determine which parts of the image the network should allocate more representation capabilities to. The spatial attention mechanism enables the model to focus on the most important areas in the image, strengthen the feature representation of these areas, and thus improve the feature recognition ability of the impurity removal effect. The key information in the substrate surface impurity removal key frame is screened out through the attention mechanism, and irrelevant or unimportant background is ignored, so that the model is more focused on task-related features. The attention mechanism helps to improve the accuracy of the model because it can simulate human visual attention and focus on the visual content that is most helpful to the current task. By focusing only on the key parts of the substrate surface impurity removal key frame, the comprehensive calculation of the entire image can be reduced, thereby reducing the amount of calculation and increasing the processing speed.

[0034] Specifically, in the above-mentioned intelligent spraying control method based on computer vision technology, the substrate surface spatial feature extraction module is a convolutional neural network model using a spatial attention mechanism.

[0035] Specifically, in the above-mentioned intelligent spray control method based on computer vision technology, the multiple substrate surface impurity removal key frames are respectively passed through a substrate surface spatial feature extraction module to obtain multiple substrate spatial structure feature maps, including: using the convolution coding part of the substrate surface spatial feature extraction module to perform deep convolution coding on the substrate surface impurity removal key frames to obtain an initial convolution feature map; inputting the initial convolution feature map into the spatial attention part of the substrate surface spatial feature extraction module to obtain a spatial attention map; passing the spatial attention map through a Softmax activation function to obtain a spatial attention feature map; and calculating the point-by-point multiplication of the spatial attention feature map and the initial convolution feature map to obtain the substrate spatial structure feature map.

[0036] Specifically, in the above-mentioned intelligent spraying control method based on computer vision technology, the initial convolution feature map is input into the spatial attention part of the substrate surface spatial feature extraction module to obtain a spatial attention map, including: performing average pooling and maximum pooling on the initial convolution feature map along the channel dimension to obtain an average feature matrix and a maximum feature matrix; cascading and channel-adjusting the average feature matrix and the maximum feature matrix to obtain a channel feature matrix; and, using the convolution layer of the spatial attention feature map to perform convolution encoding on the channel feature matrix to obtain a spatial attention map.

[0037] More specifically, the plurality of substrate spatial structure feature maps are passed through a substrate surface temporal feature extraction module to obtain the substrate spatial structure change feature vector, and the substrate surface temporal feature extraction module is a convolutional neural network model using a temporal attention mechanism. The temporal attention mechanism is a technique for sequence data that allows the model to assign different attention or weights to different time points in the sequence. This mechanism is particularly suitable for processing time series data or video frame sequences, where the relative temporal relationship between data points is crucial to understanding the context of the entire sequence.

[0038] In the technical solution of the present application, the role of the temporal attention mechanism is as follows: 1. Selective attention: The model can selectively focus on the most important parts of the sequence, such as the frames where the impurity removal effect is most obvious. 2. Context modeling: The temporal attention mechanism can help the model establish long-term and short-term contextual relationships and understand the evolution of features over time. 3. Feature enhancement: By strengthening the features of key time points, the temporal attention mechanism can increase the model's sensitivity to these features. 4. Reduce information loss: When processing long sequence data, the temporal attention mechanism can reduce the loss in the information compression process. 5. Improve interpretability: The weights of temporal attention can be visualized to help understand which parts of the sequence the model pays attention to when making decisions. 6. Optimize resource allocation: The model can allocate computing resources to more important time points, thereby improving overall processing efficiency. 7. Enhance generalization ability: The temporal attention mechanism enables the model to better generalize to different data sets and conditions, improving the robustness of the model.

[0039] In the technical solution of the present application, the temporal attention mechanism enables the model to focus on key information at different time points in the video sequence, which is crucial for understanding dynamic processes that change over time. By identifying and emphasizing key events or stages in the video, the temporal attention mechanism helps the model identify and respond to key moments in the process of removing impurities from the substrate surface. Applying the temporal attention mechanism in the feature maps of consecutive frames can extract features that change over time, which is very helpful for understanding the entire spraying and cleaning process. By focusing on the most important time points, the temporal attention mechanism can reduce the processing of non-critical time points and improve computational efficiency. This mechanism enables the model to adapt to different spraying speeds and processes and improves the generalization ability of the model. The temporal attention mechanism helps to fuse and extract the most representative features from the feature maps of multiple time points.

[0040] Specifically, in the above-mentioned intelligent spraying control method based on computer vision technology, the substrate surface time feature extraction module is a convolutional neural network model using a time attention mechanism.

[0041] Specifically, in the above-mentioned intelligent spraying control method based on computer vision technology, the multiple substrate spatial structure feature maps are passed through a substrate surface time feature extraction module to obtain the substrate spatial structure change feature vector, including: extracting adjacent first substrate spatial structure feature maps and second substrate spatial structure feature maps from the multiple substrate spatial structure feature maps; passing the first substrate spatial structure feature map and the second substrate spatial structure feature map through the first convolution layer and the second convolution layer of the substrate surface time feature extraction module respectively to obtain a first convolution feature map and a second convolution feature map; calculating the position point multiplication between the first convolution feature map and the second convolution feature map to obtain a time attention map; inputting the time attention map into a Softmax activation function to obtain a time attention feature map; passing the second substrate spatial structure feature map through the third convolution layer of the substrate surface time feature extraction module to obtain a third convolution feature map; and calculating the position point multiplication between the third convolution feature map and the time attention feature map to obtain a time attention feature map corresponding to the second substrate spatial structure feature map.

[0042] In step S130, the air knife airflow velocity value and the distance value between the air knife and the spraying surface are processed to obtain an air knife feature vector.

[0043] Figure 3 A flowchart of processing the air knife airflow velocity value and the distance value between the air knife and the spraying surface in the intelligent spraying control method based on computer vision technology according to the embodiment of the present application to obtain the air knife feature vector. Figure 3As shown, the air knife airflow velocity value and the distance value between the air knife and the spraying surface are processed to obtain the air knife feature vector, including: S131, arranging the air knife airflow velocity value and the distance value between the air knife and the spraying surface into an airflow velocity input vector and a distance input vector according to the time dimension respectively; S132, fusing the airflow velocity input vector and the distance input vector to obtain a transfer air knife matrix; and S133, passing the transfer air knife matrix through an air knife feature extraction module based on a convolutional neural network model to obtain the air knife feature vector.

[0044] More specifically, the air knife air velocity value and the distance value between the air knife and the spraying surface are arranged according to the time dimension as the air velocity input vector and the distance input vector. The spraying process is a dynamic time series process, and the changes of air velocity and distance over time have a direct impact on the spraying effect. Arranging these parameters in time order can be used for time series analysis to capture the dynamic changes in the spraying process.

[0045] Furthermore, the airflow velocity input vector and the distance input vector are fused to obtain the transfer air knife matrix. The airflow velocity and the distance between the air knife and the spraying surface are two key parameters that affect the spraying effect. Fusion of these two vectors can integrate the features of these two dimensions and provide more comprehensive information for the model. The fused transfer air knife matrix can be used as the basis for feature selection and feature extraction, which helps to discover more important features and reduce feature dimensions.

[0046] Specifically, in the above-mentioned intelligent spraying control method based on computer vision technology, the airflow velocity input vector and the distance input vector are fused to obtain a transfer air knife matrix, including: fusing the airflow velocity input vector and the distance input vector with the following fusion formula to obtain a transfer air knife matrix; wherein the fusion formula is:

[0047]

[0048] in represents vector multiplication, M represents the transfer air knife matrix, F1 represents the airflow velocity input vector, F2 represents the distance input vector, Represents the transpose of the distance input vector.

[0049] More specifically, the transferred air knife matrix is ​​passed through an air knife feature extraction module based on a convolutional neural network (CNN) model to obtain an air knife feature vector. Deep CNN can learn complex relationships and high-level features in the data and improve the expressiveness of features. The trained CNN model has good generalization ability and can adapt to different spraying conditions and substrate types.

[0050] Specifically, in the above-mentioned intelligent spraying control method based on computer vision technology, the transfer air knife matrix is ​​passed through an air knife feature extraction module based on a convolutional neural network model to obtain the air knife feature vector, including: using each layer of the air knife feature extraction module based on the convolutional neural network model to perform respectively in the forward transfer of the layer: convolution processing on the input data to obtain a convolution feature map; mean pooling processing of each feature matrix along the channel dimension of the convolution feature map to obtain a pooled feature map; and nonlinear activation of the pooled feature map to obtain an activation feature map; wherein the output of the last layer of the air knife feature extraction module based on the convolutional neural network model is the air knife feature vector, and the input of the first layer of the air knife feature extraction module based on the convolutional neural network model is the transfer air knife matrix.

[0051] In step S140, the substrate spatial structure change feature vector and the air knife feature vector are comprehensively analyzed to obtain a result indicating whether the air flow velocity of the air knife should be increased or decreased at the current time point.

[0052] Specifically, in the above-mentioned intelligent spraying control method based on computer vision technology, the substrate spatial structure change feature vector and the air knife feature vector are comprehensively analyzed to obtain a result indicating that the airflow velocity of the air knife at the current time point should be increased or decreased, including: performing perspective projection matching on the air knife feature vector and the substrate spatial structure change feature vector based on feature topological association to obtain an air knife parameter control feature vector; and, performing classification analysis on the air knife parameter control feature vector to obtain a result indicating that the airflow velocity of the air knife at the current time point should be increased or decreased.

[0053] In particular, in the technical solution of the present application, it is considered that the air knife feature vector is extracted from the working parameters of the air knife (the air flow velocity value and the distance value between the air knife and the spraying surface), while the substrate spatial structure change feature vector is extracted from the key frame of the substrate surface impurity removal monitoring video. These two feature vectors are derived from two different data sources, one is the numerical data of the time series, and the other is the visual image data. Because they are essentially different, there is a significant feature heterogeneity between them. That is, the air knife feature vector and the substrate spatial structure change feature vector have more significant feature heterogeneity. Specifically, the air knife feature vector and the substrate spatial structure change feature vector may exist on different manifolds, which means that they have different distribution laws in their respective spaces. Directly splicing them together may result in the final feature vector not having a clear manifold in the new space, that is, the feature manifold smoothness is insufficient. At the same time, since the two feature vectors come from different sources, they may not be well mapped to each other in the fused feature space, that is, one type of feature may not accurately reflect the feature changes of another type, resulting in increased uncertainty in feature expression. Based on this, in the technical solution of the present application, the air knife feature vector and the substrate spatial structure change feature vector are matched by perspective projection based on feature topological association.

[0054] Specifically, the air knife feature vector and the substrate spatial structure change feature vector are subjected to perspective projection matching based on feature topological association to obtain an air knife parameter control feature vector, including: constructing an air knife parameter distance topological matrix between the air knife feature vector and the substrate spatial structure change feature vector; constructing an air knife parameter eigenvalue granularity association matrix between the air knife feature vector and the substrate spatial structure change feature vector; performing topological association modulation on the air knife parameter eigenvalue granularity association matrix based on the air knife parameter distance topological matrix to obtain an air knife parameter topological modulation association matrix; using the air knife parameter topological modulation association matrix as a perspective co-projection space, projecting the air knife feature vector and the substrate spatial structure change feature vector into the perspective co-projection space to obtain a perspective modulated air knife feature vector and a perspective modulated substrate spatial structure change feature vector; and fusing the perspective modulated air knife feature vector and the perspective modulated substrate spatial structure change feature vector to obtain the air knife parameter control feature vector.

[0055] In an embodiment of the present application, another possible implementation method of performing perspective projection matching based on feature topological association on the air knife feature vector and the feature vector of the spatial structure change of the substrate to obtain the air knife parameter control feature vector may be: performing perspective projection matching based on feature topological association on the air knife feature vector and the feature vector of the spatial structure change of the substrate to obtain the air knife parameter control feature vector according to the following formula; wherein the formula is:

[0056] M t =D(V1,V2)

[0057]

[0058] V c =α·V1′+β·V2′

[0059] Wherein, V1 represents the air knife feature vector, V2 represents the substrate spatial structure change feature vector, and D(V1, V2) is the distance topological matrix between the air knife feature vector and the substrate spatial structure change feature vector, that is, d i,j =d(v 1i ,v 2j ), d i,j is the eigenvalue of the position (i, j) in the distance topology matrix, d(v 1i ,v 2j ) represents the distance between the eigenvalue of the i-th position of the air knife eigenvector and the eigenvalue of the j-th position of the substrate spatial structure change eigenvector, and V1 and V2 are both column vectors, M t represents the air knife parameter distance topology matrix, T represents the transpose of the vector, Represents matrix multiplication, M v represents the air knife parameter eigenvalue particle size correlation matrix, M c represents the topological modulation correlation matrix of air knife parameters, V1' represents the eigenvector of the air knife modulated by the viewing angle, V2' represents the eigenvector of the spatial structure change of the substrate modulated by the viewing angle, α and β represent weighted hyperparameters, V c Represents the air knife parameter control eigenvector.

[0060] Based on this, in the technical solution of the present application, the air knife feature vector and the substrate spatial structure change feature vector are matched by perspective projection based on feature topological association, and the air knife parameter distance topological matrix between the air knife feature vector and the substrate spatial structure change feature vector is first constructed. This step utilizes the idea of ​​graph theory, takes the eigenvalues ​​of each position in the air knife feature vector and the substrate spatial structure change feature vector as nodes, and uses the distance metric function to determine the low-dimensional embedding expression of the edges between nodes, so as to construct a feature expression that can quantify the topological association relationship between the air knife feature vector and the substrate spatial structure change feature vector. This step is crucial to understanding the spatial distribution of data points and their mutual connections, and also provides basic structural information for subsequent steps.

[0061] Next, an air knife parameter eigenvalue granularity association matrix is ​​constructed between the air knife feature vector and the substrate spatial structure change feature vector. This step focuses on exploring the correlation between the internal attributes of the feature vector. In a specific example, the product between the air knife feature vector and the transposed vector of the substrate spatial structure change feature vector is calculated to obtain the air knife parameter eigenvalue granularity association matrix. Furthermore, the air knife parameter eigenvalue granularity association matrix is ​​topologically modulated based on the air knife parameter distance topological matrix to obtain the air knife parameter topological modulation association matrix. That is, the eigenvalue granularity association information of the air knife feature vector and the substrate spatial structure change feature vector is driven to wander on the distance topological map to construct a low-dimensional modulation perspective co-projection space domain for mapping the air knife feature vector and the substrate spatial structure change feature vector to a continuous high-dimensional regression space attribute.

[0062] Next, the air knife parameter topological modulation correlation matrix is ​​used as the perspective co-projection space, and the air knife feature vector and the substrate space structure change feature vector are projected into the perspective co-projection space to obtain the perspective modulation air knife feature vector and the perspective modulation substrate space structure change feature vector. That is, the key to this technical solution is to find an implicit third space with modulation capability, in which as much information of the original high-dimensional data as possible is retained, so that the converted new feature vector not only reduces the dimension, but also maintains the relative position relationship between the original features.

[0063] Finally, the view-modulated air knife feature vector and the view-modulated substrate spatial structure change feature vector are fused to obtain the air knife parameter control feature vector. Fusion can be achieved in a variety of forms, ranging from simple arithmetic average to complex ensemble learning algorithms, depending on the application scenario and the goals pursued. The final air knife parameter control feature vector combines the main features of the two sets of input data and provides a more comprehensive data representation, which is not only conducive to improving the performance of machine learning models, but also promotes the ability of cross-modal or multi-source data analysis, thereby significantly enhancing the understanding and utilization efficiency of complex data structures.

[0064] More specifically, the air knife parameter control feature vector is classified and analyzed to obtain a result indicating whether the airflow velocity of the air knife at the current time point should be increased or decreased. That is, the air knife parameter control feature vector is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the airflow velocity of the air knife at the current time point should be increased or decreased. The classifier is used to make a decision based on the information in the feature vector, that is, to determine whether the airflow velocity needs to be increased or decreased to adapt to the current spraying conditions. In an automated and intelligent spraying production line, the air knife parameters need to be adjusted in real time to maintain the optimal state of the spraying process, and the classifier can quickly provide such real-time decisions. The automated classification results reduce the need for manual monitoring and adjustment of air knife parameters, reduce human errors and improve production efficiency.

[0065] Specifically, in the above-mentioned intelligent spraying control method based on computer vision technology, the air knife parameter control feature vector is classified and analyzed to obtain a result indicating that the air flow velocity of the air knife at the current time point should be increased or decreased, including: passing the air knife parameter control feature vector through a classifier to obtain a classification result, and the classification result is used to indicate that the air flow velocity of the air knife at the current time point should be increased or decreased.

[0066] In summary, the intelligent spraying control method based on computer vision technology according to the embodiment of the present application has been explained, which uses artificial intelligence technology based on the field of deep learning to extract and encode the monitoring video of impurity removal on the surface of the substrate within a predetermined time period, as well as the air knife air flow velocity values ​​and the distance values ​​between the air knife and the spraying surface at multiple predetermined time points within the predetermined time period, so as to obtain the result that the air knife air flow velocity should be increased or decreased at the current time point. In this way, the efficiency of the air knife in cleaning the surface of the substrate can be improved, and the cleaning effect can be guaranteed.

[0067] Based on the same inventive concept, the embodiments of the present application also provide an intelligent spraying control system based on computer vision technology, which can be used to implement the method described in the above embodiments, as described in the following embodiments. Since the principle of solving the problem by the intelligent spraying control system based on computer vision technology is similar to the above method, the implementation of the intelligent spraying control system based on computer vision technology can refer to the implementation of the above method, and the repetitions will not be repeated. As used below, the term "unit" or "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0068] Figure 4 FIG. 1 is a block diagram of an intelligent spray control system based on computer vision technology according to an embodiment of the present application. Figure 4As shown, according to the embodiment of the present application, the intelligent spray control system 100 based on computer vision technology includes: a spray substrate cleaning data acquisition module 110, which is used to obtain a monitoring video of impurity removal on the substrate surface within a predetermined time period and the air knife airflow velocity value and the distance value between the air knife and the spraying surface at multiple predetermined time points within the predetermined time period; an impurity removal monitoring feature analysis module 120, which is used to process the substrate surface impurity removal monitoring video to obtain a substrate spatial structure change feature vector; an air knife parameter feature analysis module 130, which is used to process the air knife airflow velocity value and the distance value between the air knife and the spraying surface to obtain an air knife feature vector; and an airflow speed control module 140, which is used to comprehensively analyze the substrate spatial structure change feature vector and the air knife feature vector to obtain a result indicating that the airflow speed of the air knife at the current time point should be increased or decreased.

[0069] Here, those skilled in the art can understand that the specific functions and operations of each unit and module in the above-mentioned intelligent spray control system 100 based on computer vision technology have been described in the above reference. Figures 1 to 3 The description of the intelligent spraying control method based on computer vision technology has been introduced in detail, and therefore, its repeated description will be omitted.

[0070] In summary, the intelligent spray control system based on computer vision technology according to the embodiment of the present application has been explained, which uses artificial intelligence technology based on the field of deep learning to extract and encode the monitoring video of impurity removal on the surface of the substrate within a predetermined time period, as well as the air knife air flow velocity values ​​and the distance values ​​between the air knife and the spraying surface at multiple predetermined time points within the predetermined time period, so as to obtain the result that the air flow velocity of the air knife should be increased or decreased at the current time point. In this way, the efficiency of the air knife in cleaning the surface of the substrate can be improved, and the cleaning effect can be guaranteed.

[0071] The devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is an electronic device, and the electronic device may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0072] In a typical example, the electronic device specifically includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the above-mentioned intelligent spraying control method based on computer vision technology are implemented.

[0073] Figure 5 FIG. 1 is a block diagram of an electronic device according to an embodiment of the present application. Figure 5 As shown, based on the above data processing method, this specification also provides an electronic device, at least one processor; a memory connected to the at least one processor in communication; wherein the memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute any of the above-mentioned intelligent spraying control methods based on computer vision technology. The processor can take the form of a computer-readable medium, logic gate, switch, application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), programmable logic controller and embedded microcontroller, etc., such as a microprocessor or processor and a computer-readable program code (such as software or firmware) that can be executed by the (micro) processor. The memory includes but is not limited to random access memory (Random Access Memory, RAM), read-only memory (Read-Only Memory, ROM), cache (Cache), hard disk (Hard Disk Drive, HDD) or memory card (Memory Card). The electronic device can be a server of the method described in the above embodiment, etc. Of course, the electronic device can also include module structures such as input devices and transmission modules. In this embodiment, the functions and effects specifically implemented by the electronic device can be explained in comparison with other embodiments, and will not be repeated here.

[0074] In addition, the embodiments of this specification also provide a computer-readable storage medium on which computer instructions are stored, and when the instructions are executed, the steps of the method described in any one or more of the above embodiments are implemented. The storage medium may include a physical device for storing information, which is usually to digitize the information and then store it in a medium using electrical, magnetic or optical methods. The storage medium may include: devices that store information using electrical energy, such as various types of memories, such as RAM, ROM, etc.; devices that store information using magnetic energy, such as hard disks, floppy disks, magnetic tapes, magnetic core memories, bubble memories, and U disks; devices that store information optically, such as CDs or DVDs. Of course, there are other types of readable storage media, such as quantum memories, graphene memories, and so on.

[0075] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0076] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0077] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0078] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0079] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. An intelligent spraying control method based on computer vision technology, characterized in that: include: Acquire a monitoring video of impurity removal from the substrate surface within a predetermined time period, as well as air knife airflow velocity values ​​and distance values ​​between the air knife and the spraying surface at multiple predetermined time points within the predetermined time period; Processing the monitoring video of impurity removal on the surface of the substrate to obtain a feature vector of spatial structure change of the substrate; Processing the air knife airflow velocity value and the distance value between the air knife and the spraying surface to obtain an air knife feature vector; The substrate spatial structure change feature vector and the air knife feature vector are comprehensively analyzed to obtain a result indicating whether the airflow velocity of the air knife should be increased or decreased at the current time point.

2. The intelligent spraying control method based on computer vision technology according to claim 1 is characterized in that: Processing the monitoring video of impurity removal on the substrate surface to obtain a feature vector of the spatial structure change of the substrate includes: Extracting a plurality of substrate surface impurity removal key frames from the substrate surface impurity removal monitoring video; Passing the plurality of substrate surface impurity removal key frames through a substrate surface spatial feature extraction module to obtain a plurality of substrate spatial structure feature maps; The plurality of substrate spatial structure feature maps are passed through a substrate surface time feature extraction module to obtain the substrate spatial structure change feature vector.

3. The intelligent spraying control method based on computer vision technology according to claim 2 is characterized in that: The substrate surface spatial feature extraction module is a convolutional neural network model using a spatial attention mechanism.

4. The intelligent spraying control method based on computer vision technology according to claim 3 is characterized in that: The substrate surface temporal feature extraction module is a convolutional neural network model using a temporal attention mechanism.

5. The intelligent spraying control method based on computer vision technology according to claim 4 is characterized in that: Processing the air knife airflow velocity value and the distance value between the air knife and the spraying surface to obtain the air knife feature vector includes: Arrange the air knife air flow velocity value and the distance value between the air knife and the spraying surface into an air flow velocity input vector and a distance input vector respectively according to the time dimension; Merging the airflow velocity input vector and the distance input vector to obtain a transfer air knife matrix; The transferred air knife matrix is ​​passed through an air knife feature extraction module based on a convolutional neural network model to obtain the air knife feature vector.

6. The intelligent spraying control method based on computer vision technology according to claim 5 is characterized in that: Comprehensively analyzing the substrate spatial structure change feature vector and the air knife feature vector, a result indicating whether the airflow velocity of the air knife should be increased or decreased at the current time point is obtained, including: Performing perspective projection matching on the air knife feature vector and the substrate spatial structure change feature vector based on feature topological association to obtain an air knife parameter control feature vector; The air knife parameter control feature vector is classified and analyzed to obtain a result indicating whether the air flow velocity of the air knife should be increased or decreased at the current time point.

7. The intelligent spraying control method based on computer vision technology according to claim 6 is characterized in that: Performing perspective projection matching based on feature topological association on the air knife feature vector and the substrate spatial structure change feature vector to obtain an air knife parameter control feature vector, including: Constructing an air knife parameter distance topological matrix between the air knife feature vector and the substrate spatial structure change feature vector; Constructing an air knife parameter eigenvalue particle size correlation matrix between the air knife eigenvector and the substrate spatial structure change eigenvector; Based on the air knife parameter distance topological matrix, the air knife parameter eigenvalue granularity association matrix is ​​topologically modulated to obtain an air knife parameter topological modulation association matrix; Taking the air knife parameter topological modulation correlation matrix as the perspective co-projection space, projecting the air knife feature vector and the substrate space structure change feature vector into the perspective co-projection space to obtain the perspective modulation air knife feature vector and the perspective modulation substrate space structure change feature vector; The viewing angle modulated air knife feature vector and the viewing angle modulated substrate spatial structure change feature vector are fused to obtain the air knife parameter control feature vector.

8. The intelligent spraying control method based on computer vision technology according to claim 7 is characterized in that: The air knife parameter control feature vector is classified and analyzed to obtain a result indicating whether the air flow velocity of the air knife should be increased or decreased at the current time point, including: The air knife parameter control feature vector is passed through a classifier to obtain a classification result, and the classification result is used to indicate whether the air flow velocity of the air knife should be increased or decreased at the current time point.

9. An intelligent spraying control system based on computer vision technology, characterized in that: include: A spray substrate cleaning data acquisition module is used to acquire a monitoring video of impurity removal on the substrate surface within a predetermined time period, as well as air knife air flow velocity values ​​and distance values ​​between the air knife and the spray surface at multiple predetermined time points within the predetermined time period; An impurity removal monitoring feature analysis module is used to process the impurity removal monitoring video on the substrate surface to obtain a feature vector of the spatial structure change of the substrate; An air knife parameter characteristic analysis module is used to process the air knife air flow velocity value and the distance value between the air knife and the spraying surface to obtain an air knife characteristic vector; The airflow velocity control module is used to comprehensively analyze the characteristic vector of the spatial structure change of the substrate and the characteristic vector of the air knife to obtain a result indicating whether the airflow velocity of the air knife should be increased or decreased at the current time point.

10. The intelligent spraying control system based on computer vision technology according to claim 9 is characterized in that: The air flow speed control module comprises: An air knife parameter control feature matching generation unit, used for performing perspective projection matching on the air knife feature vector and the substrate spatial structure change feature vector based on feature topological association to obtain an air knife parameter control feature vector; The air knife parameter control feature classification unit is used to classify and analyze the air knife parameter control feature vector to obtain a result indicating whether the air flow velocity of the air knife should be increased or decreased at the current time point.

Images