Automobile accessory production data real-time monitoring system and method

Abstract

The invention relates to the field of production monitoring, and particularly discloses an automobile part production data real-time monitoring system and method, and the method comprises the steps: firstly obtaining automobile part pipe fitting position data collected by a position sensor and an automobile part punching production image collected by a visual sensor, and then carrying out the deep learning technology, and feature extraction and correlation analysis are carried out on the two values, and finally a classification result is obtained through a classifier so as to judge whether to send out an automobile part punching deviation early warning, so that manual intervention is reduced, the punching precision is improved, waste products caused by deviation are reduced, the production cost is reduced, and the production efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of production monitoring, and more specifically, to a real-time monitoring system and method for automobile parts production data. Background Art

[0002] Auto parts are the basic units that make up a car. Each part works together through precise design and manufacturing to ensure the normal operation and safety of the vehicle.

[0003] In the process of drilling holes in automobile parts, the traditional method usually requires manual rotation of the handle to position and fix the pipe through the clamp. However, this positioning method has a low degree of automation and cannot achieve automatic positioning and position correction, which easily leads to unstable clamping. Since only the surface of the pipe can be fixed, when the drill is drilling, the vibration generated by the drilling will cause the pipe to vibrate, causing the drilling position to shift and causing a large error. This not only affects the accuracy of the drilling, but may also have a negative impact on the quality and safety of the final product.

[0004] Therefore, a real-time monitoring system and method for automobile parts production data is desired. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a real-time monitoring system and method for automobile parts production data, which first obtains automobile parts pipe position data collected by a position sensor and automobile parts punching production images collected by a visual sensor, and then uses deep learning technology to perform feature extraction and correlation analysis on the two, and finally obtains classification results through a classifier to determine whether to issue an automobile parts punching deviation warning, thereby reducing manual intervention, improving punching accuracy, and further reducing scrap caused by deviation, reducing production costs, and improving production efficiency.

[0006] According to one aspect of the present application, a real-time monitoring system for automobile parts production data is provided, which includes: The automobile parts production data acquisition module is used to acquire the automobile parts pipe position data acquired by the position sensor and the automobile parts punching production image acquired by the visual sensor; An automobile parts production data extraction module is used to extract automobile parts pipe coordinate text associated feature vectors and automobile parts punching production global feature vectors from the automobile parts pipe position data collected by the position sensor and the automobile parts punching production images collected by the visual sensor; The punching offset warning judgment module is used to judge whether to issue an automobile part punching offset warning based on the automobile part pipe coordinate text association feature vector and the automobile part punching production global feature vector.

[0007] According to another aspect of the present application, a method for real-time monitoring of automobile parts production data is provided, which comprises: Acquire the position data of automobile parts pipes collected by the position sensor and the punching production images of automobile parts collected by the visual sensor; Extracting the auto parts pipe coordinate text associated feature vector and the auto parts punching production global feature vector from the auto parts pipe position data collected by the position sensor and the auto parts punching production image collected by the visual sensor; Based on the automobile parts pipe coordinate text associated feature vector and the automobile parts drilling production global feature vector, it is determined whether to issue an automobile parts drilling deviation warning.

[0008] Compared with the prior art, the present application provides a real-time monitoring system and method for automobile parts production data, which first obtains automobile parts pipe position data collected by position sensors and automobile parts punching production images collected by visual sensors, and then uses deep learning technology to perform feature extraction and correlation analysis on the two, and finally uses a classifier to obtain a classification result to determine whether to issue an automobile parts punching deviation warning, thereby reducing manual intervention, improving punching accuracy, and further reducing scrap caused by deviation, reducing production costs, and improving production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0010] Figure 1 4 is a block diagram of a real-time monitoring system for automobile parts production data according to an embodiment of the present application.

[0011] Figure 2 It is a block diagram of an automobile parts production data extraction module in a real-time monitoring system for automobile parts production data according to an embodiment of the present application.

[0012] Figure 3 It is a block diagram of an automobile parts pipe position feature extraction unit in a real-time monitoring system for automobile parts production data according to an embodiment of the present application.

[0013] Figure 4 It is a block diagram of an automobile parts punching production feature extraction unit in a real-time monitoring system of automobile parts production data according to an embodiment of the present application.

[0014] Figure 5It is a block diagram of a drilling deviation early warning judgment module in a real-time monitoring system for automobile parts production data according to an embodiment of the present application.

[0015] Figure 6 The present invention is a flowchart of a method for real-time monitoring of automobile parts production data according to an embodiment of the present application. DETAILED DESCRIPTION

[0016] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0017] Figure 1 Schematic diagram of the real-time monitoring system for automobile parts production data according to the embodiment of the present application. Figure 1 As shown, according to the embodiment of the present application, the real-time monitoring system 100 for automobile parts production data includes: an automobile parts production data acquisition module 110, which is used to acquire automobile parts pipe position data collected by a position sensor and automobile parts punching production images collected by a visual sensor; an automobile parts production data extraction module 120, which is used to extract automobile parts pipe coordinate text-associated feature vectors and automobile parts punching production global feature vectors from the automobile parts pipe position data collected by the position sensor and the automobile parts punching production images collected by the visual sensor; a punching offset warning judgment module 130, which is used to judge whether to issue an automobile parts punching offset warning based on the automobile parts pipe coordinate text-associated feature vector and the automobile parts punching production global feature vector.

[0018] In the above-mentioned real-time monitoring system 100 for automobile parts production data, the automobile parts production data acquisition module 110 is used to acquire the automobile parts pipe position data collected by the position sensor and the automobile parts punching production image collected by the visual sensor. It should be understood that automobile parts are the basic units that constitute the automobile, and each accessory works together through precise design and manufacturing to ensure the normal operation and safety of the vehicle. In the process of punching pipes of automobile parts, the traditional method usually requires manual rotation of the handle to position and fix the pipes through the clamping plate. However, the degree of automation of this positioning method is low, and automatic positioning and position correction cannot be achieved, which easily leads to unstable clamping. Since only the surface of the pipe can be fixed, when the drill is punching, the vibration generated by the punching will cause the vibration of the pipe, thereby causing the punching position to shift, resulting in a large error. This not only affects the accuracy of the punching, but may also have a negative impact on the quality and safety of the final product.

[0019] Specifically, the position sensor can monitor the specific position of the pipe fittings on the production line in real time, ensuring that each fitting can be accurately positioned to avoid assembly errors or quality problems caused by position deviation. Commonly used sensors include laser rangefinders, ultrasonic sensors and encoders, which provide high-precision position data by measuring the distance to the reference point. On the other hand, through high-resolution cameras and image processing technology, visual sensors can capture the production images of punching in real time and analyze key parameters such as hole position and aperture. This image data is not only used for instant feedback and adjustment of production equipment, but also for quality inspection after production is completed to ensure that each punching process meets the standards. Integrating the data of these two sensors can realize the intelligent upgrade of the automated production line. Therefore, in the technical solution of the present application, by obtaining the position data of the automobile parts pipe fittings collected by the position sensor and the automobile parts punching production images collected by the visual sensor, and combining with deep learning technology, it is determined whether it is necessary to issue an early warning of the deviation of the automobile parts punching, so as to reduce manual intervention, improve the punching accuracy, and thus reduce the waste caused by the deviation, reduce production costs, and improve production efficiency.

[0020] In the above-mentioned automobile parts production data real-time monitoring system 100, the automobile parts production data extraction module 120 is used to extract the automobile parts pipe coordinate text associated feature vector and the automobile parts punching production global feature vector from the automobile parts pipe position data collected by the position sensor and the automobile parts punching production image collected by the visual sensor. In this way, the intelligent identification and positioning of automobile parts pipes can be realized, and the efficiency and accuracy of the production process can be improved. This cross-sensor data fusion technology can not only optimize the production process, but also provide important data support for subsequent quality control and process improvement.

[0021] Figure 2 FIG. 1 is a block diagram of an automobile parts production data extraction module in a real-time monitoring system for automobile parts production data according to an embodiment of the present application. Figure 2 In a specific embodiment of the present application, the automobile parts production data extraction module 120 includes: an automobile parts pipe position feature extraction unit 121, which is used to perform feature extraction on the automobile parts pipe position data collected by the position sensor to obtain the automobile parts pipe coordinate text associated feature vector; an automobile parts punching production feature extraction unit 122, which is used to perform feature extraction on the automobile parts punching production image collected by the visual sensor to obtain the automobile parts punching production global feature vector.

[0022] It should be understood that in the automobile manufacturing process, the original position information often contains a lot of noise and redundant data, and directly using this data for analysis or decision-making may lead to errors. Therefore, through feature extraction, the key information of the pipe fittings in space can be refined to form an efficient feature vector. Specifically, the raw data collected by the position sensor can be preprocessed to remove outliers and noise to ensure the reliability of the data. Then, the data is converted into spatial coordinates to extract the geometric features of the pipe fittings, such as the center point coordinates, bounding box, and aspect ratio of the pipe fittings. These geometric features can effectively describe the shape and position of the pipe fittings. In addition, algorithms can be used to encode these features to form a vector representation. For example, dimensionality reduction techniques such as principal component analysis (PCA) can be used to reduce feature dimensions, extract the most representative features, and then form a feature vector. In this way, complex position information can be simplified into a fixed-length vector, which is convenient for subsequent machine learning or data mining.

[0023] Furthermore, the original image data often contains complex background information and noise, and it is not only difficult but also inaccurate to use these data directly for analysis. Through feature extraction, key information can be refined. Among them, edge detection algorithms (such as Canny edge detection) can be applied to identify the outline of the hole, and the specific features of the hole, such as position and size, can be extracted through contour analysis. In addition, image segmentation technology can be used to separate the hole area from the background to ensure that subsequent feature extraction is only focused on the relevant part. In addition, shape descriptors (such as Hu invariant moments, contour features, etc.) can be used to further characterize the geometric characteristics of the hole. These descriptors can provide information about the shape, symmetry, and distribution of the hole. The extracted features can be processed by standardization and normalization. Through this process of feature extraction and vectorization, real-time monitoring and rapid feedback of the punching process of automotive parts can be achieved, thereby improving production efficiency and product quality.

[0024] Figure 3 FIG. 1 is a block diagram of an automobile parts pipe position feature extraction unit in a real-time monitoring system for automobile parts production data according to an embodiment of the present application. Figure 3 As shown, in a specific embodiment of the present application, the automobile parts pipe position feature extraction unit 121 includes: an automobile parts pipe position context encoding subunit 1211, which is used to pass the automobile parts pipe position data collected by the position sensor through an automobile parts pipe coordinate text context encoder to obtain multiple automobile parts pipe coordinate text feature vectors; an automobile parts pipe coordinate text multi-scale feature extraction subunit 1212, which is used to pass the multiple automobile parts pipe coordinate text feature vectors through an automobile parts pipe coordinate text multi-scale neighborhood feature extraction module to obtain an automobile parts pipe coordinate text associated feature vector.

[0025] It should be understood that although the original position data contains the coordinate information of the pipe in space, it is usually sparse and simple, and it is difficult to capture the relationship and contextual information between the pipes. Through the context encoder, these data can be converted into feature vectors with more semantic meaning and higher dimensions. Specifically, first, the original coordinate data obtained by the position sensor needs to be sorted and preprocessed to remove noise and outliers to ensure the quality of the data. Next, these coordinate data are input into the automotive parts pipe coordinate text context encoder. The encoder is designed to understand the contextual information of the pipe coordinates, including the relative position, connection relationship between different pipes, and the position in the overall manufacturing process. Among them, the context encoder can adopt a variety of technologies, including deep learning-based models such as recurrent neural networks (RNNs) or transformer models. These models can extract features with contextual dependence by analyzing the time series or spatial sequence information in the data. In this process, the encoder performs multi-layer nonlinear transformations on the input coordinate data to generate high-dimensional feature vectors containing rich information. In this way, the level of intelligence in the manufacturing process is improved, thereby optimizing production efficiency and ensuring product quality.

[0026] Specifically, the automobile parts and pipes position data collected by the position sensor is passed through an automobile parts and pipes coordinate text context encoder to obtain multiple automobile parts and pipes coordinate text feature vectors, including: performing word segmentation processing on the automobile parts and pipes position data collected by the position sensor to obtain an automobile parts word sequence; using the embedding layer of the automobile parts and pipes coordinate text context encoder to respectively map each automobile parts word in the automobile parts word sequence into a word embedding vector to obtain a sequence of automobile parts word embedding vectors; using the converter-based Bert model of the automobile parts and pipes coordinate text context encoder to perform global context semantic encoding on the sequence of automobile parts word embedding vectors to obtain multiple automobile parts and pipes coordinate text feature vectors.

[0027] Furthermore, processing multiple auto parts and pipe fittings coordinate text feature vectors through the auto parts and pipe fittings coordinate text multi-scale neighborhood feature extraction module can mine the potential associations in these feature vectors, providing a more comprehensive perspective for subsequent analysis and decision-making. Specifically, first, multiple coordinate text feature vectors are input into the multi-scale neighborhood feature extraction module, which is designed to analyze features at different scales. Through multi-scale processing, local features and global features can be focused on at the same time, so as to effectively capture the distribution law and mutual relationship of pipe fittings in space. This module can extract features at different scales using technologies such as convolutional neural networks (CNNs), thereby enhancing the model's sensitivity to spatial relationships. Specifically, the multi-scale neighborhood feature extraction module defines multiple neighborhoods around each feature vector, which can be areas of different radii. The module extracts richer feature information by aggregating and analyzing the feature vectors in each neighborhood. This process not only includes direct feature calculation, but also can further enhance the feature expression capability by calculating the distance, similarity and other statistics between features. Finally, these processing results will be summarized into a new associated feature vector, reflecting the deep relationship between the space and features of automobile parts and pipes. Through this multi-scale feature extraction method, the resulting auto parts and pipes coordinate text associated feature vector can not only more comprehensively describe the characteristics of each pipe, but also effectively support subsequent intelligent decision-making and manufacturing optimization.

[0028] In a specific embodiment of the present application, the multi-scale neighborhood feature extraction module for automobile parts pipe coordinate text includes: a first convolution layer, a second convolution layer parallel to the first convolution layer, and a cascade layer connected to the first convolution layer and the second convolution layer, wherein the first convolution layer uses a one-dimensional convolution kernel with a first scale, and the second convolution layer uses a one-dimensional convolution kernel with a second scale.

[0029] More specifically, the multiple auto parts and tube coordinate text feature vectors are passed through an auto parts and tube coordinate text multi-scale neighborhood feature extraction module to obtain an auto parts and tube coordinate text associated feature vector, including: using the first convolution layer of the auto parts and tube coordinate text multi-scale neighborhood feature extraction module to perform one-dimensional convolution encoding on the multiple auto parts and tube coordinate text feature vectors to obtain a first scale feature vector; using the second convolution layer of the auto parts and tube coordinate text multi-scale neighborhood feature extraction module to perform one-dimensional convolution encoding on the multiple auto parts and tube coordinate text feature vectors to obtain a second scale feature vector; and cascading the first scale feature vector and the second scale feature vector to obtain the auto parts and tube coordinate text associated feature vector.

[0030] Among them, the first convolution layer uses a one-dimensional convolution kernel with a first scale, which enables this layer to focus on extracting local features and capturing detailed information of the pipe fittings within a smaller range. Through small-scale convolution operations, the first layer can effectively identify features such as the boundaries, angles, and subtle geometric changes of the pipe fittings. These details are critical in identifying the shape of the pipe fittings and their positioning in space, which helps with subsequent accurate analysis and processing. At the same time, the second convolution layer uses a one-dimensional convolution kernel with a second scale, which means that it will focus on features within a larger range. Through large-scale convolutions, the second layer can integrate more contextual information to capture the relative position and overall structural characteristics between pipe fittings. The use of large-scale convolution kernels allows the model to not only be limited to a single pipe fitting when analyzing, but also to identify the association and layout between pipe fittings, which is particularly important for quality control and optimization decisions in the overall manufacturing process. Finally, the cascade layer integrates the features extracted by the first and second convolution layers to generate a richer feature expression. The design of this layer enables features of different scales to be effectively combined to form a multi-dimensional feature vector that contains both detailed information and a global perspective. Through this cascading method, the feature extraction module can give full play to the advantages of each layer and comprehensively analyze a variety of information, thus providing a more solid foundation for subsequent intelligent manufacturing and data-driven decision-making.

[0031] Figure 4 FIG. 1 is a block diagram of an automobile parts drilling production feature extraction unit in a real-time monitoring system for automobile parts production data according to an embodiment of the present application. Figure 4 As shown, in a specific embodiment of the present application, the automobile parts punching production feature extraction unit 122 includes: an automobile parts punching production image denoising subunit 1221, which is used to pass the automobile parts punching production image collected by the visual sensor through an automobile parts punching production image denoising module to obtain an automobile parts punching production denoising enhanced image; an automobile parts punching production denoising feature encoding subunit 1222, which is used to feature encode the automobile parts punching production denoising enhanced image to obtain the automobile parts punching production global feature vector.

[0032] It should be understood that the original image is often affected by a variety of interference factors, such as illumination changes, motion blur, and sensor noise, which may cause background noise and loss of details in the image, thus affecting the accurate assessment of the quality and location of the hole. Through noise reduction processing, these unnecessary interferences can be effectively eliminated, the clarity and readability of the image can be enhanced, and the foundation for subsequent feature extraction and analysis can be laid. Specifically, appropriate noise reduction algorithms such as Gaussian filtering, median filtering, or more advanced deep learning methods (such as convolutional neural networks) can be used to process the image. Taking Gaussian filtering as an example, the algorithm smoothes the image and removes random noise by weighted averaging the neighborhood pixels around each pixel. The median filter effectively removes salt and pepper noise and maintains edge details by replacing each pixel with the median of its neighborhood pixels. When performing noise reduction processing, it is particularly important to keep important feature information in the image, such as the location and size of the hole, as much as possible while removing noise. Modern noise reduction algorithms usually combine the ability to understand image content and achieve more intelligent noise reduction by learning the distribution characteristics of the image. During the processing, the model can identify which areas are important features and which are noise, so as to carry out targeted processing. Finally, the noise reduction and enhanced image of automobile parts punching production after being processed by the noise reduction module will significantly improve the image quality and reduce the error in subsequent analysis.

[0033] Specifically, the automobile parts perforation production image acquired by the visual sensor is passed through an automobile parts perforation production image denoising module to obtain an automobile parts perforation production denoising enhanced image, including: inputting the automobile parts perforation production image acquired by the visual sensor into an encoder of the automobile parts perforation production image denoising module, wherein the encoder uses a convolutional layer to perform explicit spatial encoding on the automobile parts perforation production image acquired by the visual sensor to obtain image features; and inputting the image features into a decoder of the automobile parts perforation production image denoising module, wherein the decoder uses a deconvolution layer to perform deconvolution processing on the image features to obtain the automobile parts perforation production denoising enhanced image.

[0034] Furthermore, the denoised image can usually show the features of the holes more clearly, such as the location, shape, size and distribution of the holes. However, having a clear image alone is not enough to meet the analysis requirements, so it is necessary to extract the key information in the image and convert it into a vector form through feature encoding. Specifically, computer vision algorithms such as edge detection, corner detection or feature point descriptors (such as SIFT or SURF) can be used to extract the significant features in the image. These features may include information such as the edges of the holes, the contours of the holes and the surrounding geometric shapes. These features can reflect the most important content in the image and help understand the quality and production of the holes of the pipe fittings. Next, the extracted local features are encoded, usually using deep learning models such as convolutional neural networks (CNNs). CNN can gradually extract features from low to high layers through multi-layer convolution operations, converting complex image information into high-dimensional feature vectors. Each layer of the network can learn different levels of feature representation, so that the final output feature vector can fully contain the global information and context information in the image.

[0035] In a specific embodiment of the present application, the automobile parts punching production denoising feature encoding subunit 1222 is used to: pass the automobile parts punching production denoising enhanced image through a U-Net-based automobile parts punching production convolutional neural network to obtain an automobile parts punching production edge segmentation feature map; pass the automobile parts punching production edge segmentation feature map through an automobile parts punching production edge segmentation feature encoder to obtain the automobile parts punching production global feature vector.

[0036] It should be understood that U-Net is a deep learning architecture specially designed for image segmentation tasks, especially in the fields of industrial inspection. By inputting the denoised and enhanced image into the U-Net model, the edge features of the perforated area can be effectively extracted, thereby providing accurate data support for subsequent quality inspection and defect identification. Among them, U-Net consists of a contraction path (encoder) and an expansion path (decoder). The encoder part gradually reduces the spatial dimension of the image through a series of convolution and pooling operations, while extracting features; the decoder restores the feature map to the size of the original image through upsampling and convolution. This symmetrical structure enables the network to capture local features at a low level while integrating global information at a high level, thereby achieving efficient image segmentation. In specific operations, the denoised and enhanced image is first input into the encoder part of the U-Net. After multiple layers of convolution and downsampling, the network can extract key feature information in the image, such as the contour, edge and shape of the hole. In this process, U-Net can effectively avoid information loss and combine features at different levels through skip connections to retain detail information. After encoding, the feature map will be passed to the decoder, which gradually restores the image size through upsampling and further refines the edge features through convolution. Finally, the edge segmentation feature map output by the network can clearly show the edges and contours of the perforated area. This segmentation feature map can not only be used as a basis for quality detection, but also provide important support for subsequent feature analysis and intelligent decision-making.

[0037] Furthermore, the edge segmentation feature map contains the precise edge information of the perforated area in the image, which is crucial for identifying and evaluating the perforation quality. However, in order to make more effective use of this information, it needs to be encoded to generate a global feature vector for subsequent data processing and analysis. Among them, the edge segmentation feature encoder usually consists of several convolutional layers and fully connected layers. First, the encoder takes the segmentation feature map as input, and gradually extracts high-level feature representations through multiple layers of convolution and activation functions (such as ReLU). In this process, the convolution operation can effectively learn the spatial relationship and structural characteristics of the edge features, further enhancing the expressiveness of the features. At this point, the dimension of the feature map will gradually decrease, but the semantic information of the features will be enhanced, retaining the most important edge information. Next, the encoder usually flattens the processed feature map and converts it into a one-dimensional vector. During the flattening process, the spatial structure of the feature map is broken, but the integrity of the information is still retained. After flattening, the one-dimensional vector will be input into the fully connected layer, which is responsible for further mapping the extracted features into the global feature vector. Through weighting and biasing, the network can integrate the influence of each layer's features to generate a compact and representative global feature vector. Ultimately, the resulting global feature vector for automobile parts punching production will effectively summarize the important information in the image. Through this process, the feature encoder can convert the edge information in the segmentation feature map into a usable data form, thereby supporting the decision-making process in intelligent manufacturing and automated production.

[0038] Specifically, the automobile parts punching production edge segmentation feature map is passed through an automobile parts punching production edge segmentation feature encoder to obtain the automobile parts punching production global feature vector, including: using each layer of the automobile parts punching production edge segmentation feature encoder to perform convolution processing, mean pooling processing based on the local feature matrix and nonlinear activation processing on the input data in the forward transfer of the layer to output the automobile parts punching production global feature vector from the last layer of the automobile parts punching production edge segmentation feature encoder, wherein the input of the automobile parts punching production edge segmentation feature encoder is the automobile parts punching production edge segmentation feature map.

[0039] In the above-mentioned real-time monitoring system 100 for automobile parts production data, the punching offset warning judgment module 130 is used to judge whether to issue an automobile parts punching offset warning based on the automobile parts pipe coordinate text association feature vector and the automobile parts punching production global feature vector. It should be understood that judging whether to issue an automobile parts punching offset warning based on the automobile parts pipe coordinate text association feature vector and the automobile parts punching production global feature vector is an important part of realizing intelligent monitoring and quality control. This process combines position data and image features, and aims to timely identify possible production deviations by comprehensively analyzing the relationship between the two, thereby improving product quality and production efficiency.

[0040] Figure 5 FIG. 1 is a block diagram of a punching deviation warning judgment module in a real-time monitoring system for automobile parts production data according to an embodiment of the present application. Figure 5 As shown, in a specific embodiment of the present application, the punching offset warning judgment module 130 includes: an automobile parts production data feature association unit 131, which is used to perform low-dimensional modulation perspective co-projection spatial domain mapping based on the spatial distribution of data points on the automobile parts pipe coordinate text association feature vector and the automobile parts punching production global feature vector to obtain a punching offset judgment classification feature vector; an automobile parts punching offset warning judgment classification unit 132, which is used to pass the punching offset judgment classification feature vector through a classifier to obtain a classification result, and the classification result is used to determine whether to issue an automobile parts punching offset warning.

[0041] It should be understood that the coordinate text-associated feature vector of the automobile parts pipe contains the precise position and orientation information of the pipe in three-dimensional space. This vector is obtained by parsing and processing the coordinates of the pipe, and specifically includes the coordinates, size, angle and other information of the punching part. This information is crucial to ensure the accuracy of the punching and can help identify deviations from the design specifications. The global feature vector of automobile parts punching production is extracted from image processing, covering information such as the shape, size, and edge features of the hole. These features reflect the quality and characteristics of the actual punching in the production process, and provide an intuitive basis for judging whether there is a punching offset.

[0042] In particular, in the technical solution of the present application, the auto parts pipe coordinate text-associated feature vector is generated based on the data collected by the position sensor, focusing on expressing the position, direction and relationship of the pipe in three-dimensional space. Its feature vector is usually in a highly structured text form, containing coordinate information and contextual relationships, emphasizing static geometric features and positional relationships, with clear numerical meanings, and can accurately describe the position and motion state of the pipe on the production line. In contrast, the global feature vector of automobile parts punching production is obtained by deep learning processing of image data, emphasizing the visual information in the image, such as edge features, texture and shape. The global feature vector of automobile parts punching production is a dynamic and complex visual expression, which mainly reflects the shape, position and relative relationship of the punching area with the surrounding environment during the production process. There are fundamental differences between the auto parts pipe coordinate text-associated feature vector and the auto parts punching production global feature vector in terms of data type, expression method and contextual information, resulting in the incompatibility of feature manifold and feature space. Further, when the two feature vectors are fused, heterogeneity will cause the punching offset judgment classification feature vector to lack feature manifold smoothness, thereby affecting the certainty of feature expression. Specifically, the non-smoothness of the feature manifold will make it difficult for the classifier to learn and extract potential association patterns, resulting in a decrease in classification performance. For example, the coordinate features of the position data may present a discrete distribution in the feature space, while the image features may form a relatively continuous feature manifold. The fusion of the two may produce noise and redundancy, affecting the decision-making process of the classifier. Therefore, in the technical solution of the present application, the auto parts pipe coordinate text association feature vector and the auto parts punching production global feature vector are mapped to the spatial domain based on the low-dimensional modulated perspective co-projection of the data point spatial distribution to obtain the punching offset judgment classification feature vector.

[0043] Among them, the auto parts and pipes coordinate text associated feature vectors and the auto parts punching production global feature vectors are mapped in a low-dimensional modulated perspective co-projection space domain based on the spatial distribution of data points to obtain the punching offset judgment classification feature vector, including: constructing an auto parts distance topological matrix between the auto parts and pipes coordinate text associated feature vectors and the auto parts punching production global feature vectors; constructing an auto parts feature value granularity association matrix between the auto parts and pipes coordinate text associated feature vectors and the auto parts punching production global feature vectors; performing topological association modulation on the auto parts feature value granularity association matrix based on the auto parts distance topological matrix to obtain an auto parts topological modulation association matrix; using the auto parts topological modulation association matrix as the perspective co-projection space, projecting the auto parts and pipes coordinate text associated feature vectors and the auto parts punching production global feature vectors to the perspective co-projection space to obtain the perspective modulated auto parts and pipes coordinate text associated feature vectors and the perspective modulated auto parts punching production global feature vectors; fusing the perspective modulated auto parts and pipes coordinate text associated feature vectors and the perspective modulated auto parts punching production global feature vectors to obtain the punching offset judgment classification feature vector.

[0044] The fusion step is specifically expressed as: ; is the automobile parts distance topological matrix between the automobile parts pipe coordinate text association feature vector and the automobile parts punching production global feature vector, that is, ,and and are column vectors.

[0045] ;in, represents the auto parts pipe coordinate text associated feature vector, represents the global feature vector of the automobile parts punching production, The automobile parts distance topological matrix between the automobile parts pipe coordinate text association feature vector and the automobile parts punching production global feature vector, represents the automobile parts distance topology matrix, represents the transpose of a vector, represents matrix multiplication, represents the granularity correlation matrix of the eigenvalues ​​of automobile parts, represents the topological modulation association matrix of automobile parts, Represents the coordinate text-related feature vector of automobile parts and pipes modulated by perspective, Represents the global feature vector of the perspective-modulated automobile parts punching production, and represents the weighted hyperparameter, Represents the classification feature vector for punch offset judgment.

[0046] In the technical solution of the present application, the automobile parts pipe coordinate text-associated feature vector and the automobile parts punching production global feature vector have relatively significant feature heterogeneity, which results in that when the feature fusion of the automobile parts pipe coordinate text-associated feature vector and the automobile parts punching production global feature vector is performed, the feature manifold and feature space mismatch caused by the heterogeneity will cause the punching offset judgment classification feature vector to fail to have feature manifold smoothness, thereby affecting the certainty of its feature expression.

[0047] Based on this, in the technical solution of the present application, the auto parts pipe coordinate text associated feature vector and the auto parts punching production global feature vector are subjected to low-dimensional modulated perspective co-projection spatial domain mapping based on the spatial distribution of data points, which first constructs the auto parts distance topological matrix between the auto parts pipe coordinate text associated feature vector and the auto parts punching production global feature vector. This step utilizes the idea of ​​graph theory, takes the eigenvalues ​​of each position in the auto parts pipe coordinate text associated feature vector and the auto parts punching production global feature vector as nodes, and uses the distance metric function to determine the low-dimensional embedding expression of the edges between nodes, so as to construct a feature expression that can quantify the topological association relationship between the auto parts pipe coordinate text associated feature vector and the auto parts punching production global feature vector. This step is crucial to understanding the spatial distribution of data points and their mutual connections, and also provides basic structural information for subsequent steps.

[0048] Next, an automobile parts feature value granularity association matrix between the automobile parts pipe coordinate text association feature vector and the automobile parts punching production global feature vector is constructed. This step focuses on exploring the correlation between the internal attributes of the feature vector. In a specific example, the product between the automobile parts pipe coordinate text association feature vector and the transposed vector of the automobile parts punching production global feature vector is calculated to obtain the automobile parts feature value granularity association matrix. Furthermore, the automobile parts feature value granularity association matrix is ​​topologically associated modulated based on the automobile parts distance topological matrix to obtain the automobile parts topological modulation association matrix. That is, the feature value granularity association information of the automobile parts pipe coordinate text association feature vector and the automobile parts punching production global feature vector is driven to wander on the distance topological graph to construct a low-dimensional modulated perspective co-projection space domain for mapping the automobile parts pipe coordinate text association feature vector and the automobile parts punching production global feature vector to a continuous high-dimensional regression space attribute.

[0049] Next, the auto parts topology modulation association matrix is ​​used as the perspective co-projection space, and the auto parts pipe coordinate text association feature vector and the auto parts punching production global feature vector are projected into the perspective co-projection space to obtain the perspective modulation auto parts pipe coordinate text association feature vector and the perspective modulation auto parts punching production global feature vector. That is, the key to this technical solution is to find an implicit third space with modulation capability, in which as much information of the original high-dimensional data as possible is retained, so that the converted new feature vector not only reduces the dimension, but also maintains the relative position relationship between the original features.

[0050] Finally, the perspective modulated automobile parts pipe coordinate text association feature vector and the perspective modulated automobile parts punching production global feature vector are fused to obtain the punching offset judgment classification feature vector. Fusion can be achieved in various forms, ranging from simple arithmetic average to complex ensemble learning algorithms, depending on the application scenario and the goals pursued. The final punching offset judgment classification feature vector combines the main features of the two sets of input data and provides a more comprehensive data representation, which is not only conducive to improving the performance of machine learning models, but also promotes the ability of cross-modal or multi-source data analysis, thereby significantly enhancing the understanding and utilization efficiency of complex data structures.

[0051] Furthermore, the punching offset judgment classification feature vector contains various features related to the punching process, including the coordinate information of automobile parts and pipes, the global features of punching production, and the previous offset judgment data. These feature vectors are obtained through association analysis and can effectively reflect the actual situation of the punching process. By inputting these high-dimensional features into the classifier, the possibility of punching offset can be identified and judged by training the model. By inputting the feature vector, the classifier uses the pre-learned pattern and decision boundary to output the corresponding classification result. This process includes two main steps: training and prediction. In the training stage, the classifier learns the relationship between features and offset categories through a large amount of labeled data to determine the best decision boundary; while in the prediction stage, the classifier inputs the new feature vector into the model and makes judgments based on the learned pattern. The classification result will indicate whether the punching meets the predetermined quality standards. If the classification result shows that there is a risk of offset (such as marked as "offset" or "abnormal"), the system will issue an early warning to notify the operator to take necessary measures to adjust and intervene. This real-time feedback mechanism enables problems on the production line to be discovered in time, reducing potential quality risks.

[0052] In summary, the embodiment of the present application first obtains the automobile parts pipe position data collected by the position sensor and the automobile parts punching production image collected by the visual sensor, and then uses deep learning technology to perform feature extraction and correlation analysis on the two, and finally obtains the classification result through the classifier to determine whether to issue an automobile parts punching deviation warning, thereby reducing manual intervention, improving punching accuracy, and then reducing scrap caused by deviation, reducing production costs, and improving production efficiency.

[0053] As described above, the real-time monitoring system 100 for automobile parts production data according to the embodiment of the present application can be implemented in various terminal devices. In one example, the real-time monitoring system 100 for automobile parts production data can be integrated into the terminal device as a software module and / or a hardware module. For example, the real-time monitoring system 100 for automobile parts production data can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the real-time monitoring system 100 for automobile parts production data can also be one of the many hardware modules of the terminal device.

[0054] Alternatively, in another example, the automobile parts production data real-time monitoring system 100 and the terminal device may also be separate devices, and the automobile parts production data real-time monitoring system 100 may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0055] Figure 6 Flow chart of the method for real-time monitoring of automobile parts production data according to an embodiment of the present application. Figure 6 As shown, according to the real-time monitoring method of automobile parts production data of the embodiment of the present application, it includes: S110, acquiring automobile parts pipe position data collected by the position sensor and automobile parts punching production images collected by the visual sensor; S120, extracting automobile parts pipe coordinate text associated feature vectors and automobile parts punching production global feature vectors from the automobile parts pipe position data collected by the position sensor and the automobile parts punching production images collected by the visual sensor; S130, judging whether to issue an automobile parts punching offset warning based on the automobile parts pipe coordinate text associated feature vectors and the automobile parts punching production global feature vectors.

[0056] Here, those skilled in the art will appreciate that the specific operations of each step in the above-mentioned method for real-time monitoring of automobile parts production data have been described in detail above. Figures 1 to 5 The description of the real-time monitoring system for automobile parts production data has been introduced in detail, and therefore, its repeated description will be omitted.

Claims

1. A real-time monitoring system for automobile parts production data, characterized in that: include: The automobile parts production data acquisition module is used to acquire the automobile parts pipe position data acquired by the position sensor and the automobile parts punching production image acquired by the visual sensor; An automobile parts production data extraction module is used to extract automobile parts pipe coordinate text associated feature vectors and automobile parts punching production global feature vectors from the automobile parts pipe position data collected by the position sensor and the automobile parts punching production images collected by the visual sensor; The punching offset warning judgment module is used to judge whether to issue an automobile part punching offset warning based on the automobile part pipe coordinate text association feature vector and the automobile part punching production global feature vector.

2. The real-time monitoring system for automobile parts production data according to claim 1 is characterized in that: The automobile parts production data extraction module includes: An automobile parts pipe position feature extraction unit, used for extracting features from the automobile parts pipe position data collected by the position sensor to obtain the automobile parts pipe coordinate text associated feature vector; The automobile parts punching production feature extraction unit is used to extract features from the automobile parts punching production image collected by the visual sensor to obtain the automobile parts punching production global feature vector.

3. The real-time monitoring system for automobile parts production data according to claim 2 is characterized in that: The automobile parts pipe position feature extraction unit comprises: The automobile parts pipe position context encoding subunit is used to pass the automobile parts pipe position data collected by the position sensor through the automobile parts pipe coordinate text context encoder to obtain a plurality of automobile parts pipe coordinate text feature vectors; The automobile parts and pipes coordinate text multi-scale feature extraction subunit is used to obtain automobile parts and pipes coordinate text associated feature vectors by passing the plurality of automobile parts and pipes coordinate text feature vectors through the automobile parts and pipes coordinate text multi-scale neighborhood feature extraction module.

4. The real-time monitoring system for automobile parts production data according to claim 3 is characterized in that: The multi-scale neighborhood feature extraction module for automobile parts pipe coordinate text includes: a first convolution layer, a second convolution layer parallel to the first convolution layer, and a cascade layer connected to the first convolution layer and the second convolution layer, wherein the first convolution layer uses a one-dimensional convolution kernel with a first scale, and the second convolution layer uses a one-dimensional convolution kernel with a second scale.

5. The real-time monitoring system for automobile parts production data according to claim 4 is characterized in that: The automobile parts punching production feature extraction unit comprises: The automobile parts punching production image denoising subunit is used to pass the automobile parts punching production image collected by the visual sensor through the automobile parts punching production image denoising module to obtain the automobile parts punching production denoising enhanced image; The automobile parts punching production noise reduction feature encoding subunit is used to perform feature encoding on the automobile parts punching production noise reduction enhanced image to obtain the automobile parts punching production global feature vector.

6. The real-time monitoring system for automobile parts production data according to claim 5 is characterized in that: The automobile parts punching production noise reduction feature encoding subunit is used for: The automobile parts punching production denoising and enhancement image is passed through the automobile parts punching production convolutional neural network based on U-Net to obtain an automobile parts punching production edge segmentation feature map; The automobile parts punching production edge segmentation feature map is passed through an automobile parts punching production edge segmentation feature encoder to obtain the automobile parts punching production global feature vector.

7. The real-time monitoring system for automobile parts production data according to claim 6 is characterized in that: The punching deviation early warning judgment module includes: The automobile parts production data feature association unit is used to perform low-dimensional modulated perspective co-projection spatial domain mapping based on the spatial distribution of data points on the automobile parts pipe coordinate text association feature vector and the automobile parts drilling production global feature vector to obtain a drilling offset judgment classification feature vector; The automobile parts drilling deviation warning judgment classification unit is used to pass the drilling deviation judgment classification feature vector through a classifier to obtain a classification result, and the classification result is used to determine whether to issue an automobile parts drilling deviation warning.

8. The real-time monitoring system for automobile parts production data according to claim 7 is characterized in that: The automobile parts production data feature association unit includes: Constructing an automobile parts distance topology matrix between the automobile parts pipe coordinate text association feature vector and the automobile parts punching production global feature vector; Constructing an automobile parts feature value granularity association matrix between the automobile parts pipe coordinate text association feature vector and the automobile parts punching production global feature vector; Performing topological correlation modulation on the automobile parts feature value granularity correlation matrix based on the automobile parts distance topological matrix to obtain an automobile parts topological modulation correlation matrix; Taking the automobile parts topological modulation association matrix as the perspective co-projection space, projecting the automobile parts pipe coordinate text association feature vector and the automobile parts punching production global feature vector into the perspective co-projection space to obtain the perspective modulation automobile parts pipe coordinate text association feature vector and the perspective modulation automobile parts punching production global feature vector; The perspective-modulated automobile parts pipe coordinate text-associated feature vector and the perspective-modulated automobile parts punching production global feature vector are fused to obtain the punching offset judgment classification feature vector.

9. A method for real-time monitoring of automobile parts production data, characterized in that: include: Acquire the position data of automobile parts pipes collected by the position sensor and the punching production images of automobile parts collected by the visual sensor; Extracting the auto parts pipe coordinate text associated feature vector and the auto parts punching production global feature vector from the auto parts pipe position data collected by the position sensor and the auto parts punching production image collected by the visual sensor; Based on the automobile parts pipe coordinate text associated feature vector and the automobile parts drilling production global feature vector, it is determined whether to issue an automobile parts drilling deviation warning.

10. The method for real-time monitoring of automobile parts production data according to claim 9, characterized in that: Extracting the auto parts pipe coordinate text associated feature vector and the auto parts punching production global feature vector from the auto parts pipe position data collected by the position sensor and the auto parts punching production image collected by the visual sensor, including: Performing feature extraction on the automobile parts pipe position data collected by the position sensor to obtain the automobile parts pipe coordinate text associated feature vector; Feature extraction is performed on the automobile parts punching production image collected by the visual sensor to obtain the automobile parts punching production global feature vector.

Images