Automobile part quality detection method, device, equipment and medium

Abstract

The invention discloses an automobile part quality detection method, device and equipment and a medium, and relates to the technical field of automobile manufacturing. The method comprises the following steps: acquiring a surface image and three-dimensional point cloud data of an automobile part; utilizing a convolutional neural network to extract visual aerial view features of the surface image; extracting point cloud aerial view features of the three-dimensional point cloud data by using a deep learning model; performing spatial position alignment on the visual aerial view features and the point cloud aerial view features through affine transformation; adopting a sine function and a cosine function to respectively embed position codes for the visual aerial view features and the point cloud aerial view features after the spatial position alignment; inputting the visual aerial view features after position coding and the point cloud aerial view features after position coding into a Transform model for deep fusion to obtain fusion features; and carrying out quality detection on the automobile part according to the fusion features to obtain a quality detection result. The quality detection precision and efficiency of the automobile parts can be improved, and the quality of the automobile parts is ensured.

Description

Technical Field

[0001] The present application relates to the field of automobile manufacturing technology, and in particular to a quality inspection method, device, equipment and medium for automobile parts. Background Art

[0002] In the modern automobile manufacturing industry, the quality of automobile parts has a vital impact on the performance, safety and durability of the entire vehicle. With the rapid development of automobile manufacturing towards intelligence, automation and personalization, parts quality inspection technology is also facing higher requirements. Traditional inspection methods, such as manual visual inspection, laser measurement, X-ray inspection, etc., although they guarantee the quality of parts to a certain extent, their limitations are becoming more and more obvious. These methods not only have limited accuracy, but also have limited coverage and low efficiency in the inspection of complex part structures or special materials, and the equipment cost is high, which cannot meet the current industrial manufacturing environment. The demand for efficient and accurate inspection. Therefore, in the current manufacturing context, how to break through the limitations of traditional inspection methods has become an urgent problem to be solved. Summary of the invention

[0003] The purpose of this application is to provide a method, device, equipment and medium for quality inspection of automobile parts, which can improve the quality inspection accuracy and efficiency of automobile parts and ensure the quality of automobile parts.

[0004] To achieve the above objectives, this application provides the following solutions.

[0005] In the first aspect, the present application provides a quality inspection method for automobile parts, comprising: collecting surface images and three-dimensional point cloud data of automobile parts; using a convolutional neural network to extract visual bird's-eye view features of the surface image; using a deep learning model to extract point cloud bird's-eye view features of the three-dimensional point cloud data; through affine transformation, the visual bird's-eye view features and the point cloud bird's-eye view features are spatially aligned; using sine function and cosine function to embed position codes for the spatially aligned visual bird's-eye view features and the point cloud bird's-eye view features respectively; inputting the position-encoded visual bird's-eye view features and the position-encoded point cloud bird's-eye view features into a Transformer model for deep fusion to obtain fused features; performing quality inspection of automobile parts according to the fused features to obtain quality inspection results; the quality inspection results include surface defects, geometric dimension deviations and structural abnormalities.

[0006] Optionally, a convolutional neural network is used to extract visual bird's-eye view features of the surface image, specifically including: denoising the surface image using a filtering algorithm; adjusting the size and resolution of the denoised surface image so that the format of the adjusted surface image matches the format of input data of the input layer of the convolutional neural network; inputting the adjusted surface image into ResNet to obtain visual bird's-eye view features of the surface image.

[0007] Optionally, a deep learning model is used to extract point cloud bird's-eye view features of the three-dimensional point cloud data, specifically including: denoising the three-dimensional point cloud data; downsampling the denoised three-dimensional point cloud data; normalizing the coordinates of the downsampled three-dimensional point cloud data; aligning and rotating the three-dimensional point cloud data after coordinate normalization to obtain preprocessed three-dimensional point cloud data; inputting the preprocessed three-dimensional point cloud data into a Pointpillars network to obtain point cloud bird's-eye view features of the three-dimensional point cloud data.

[0008] Optionally, the visual bird's-eye view features and the point cloud bird's-eye view features are spatially aligned through affine transformation, specifically including: according to the intrinsic parameters and extrinsic parameters of the camera, according to the formula , obtaining an affine transformation matrix; wherein the camera is a camera for taking surface images of automobile parts; is the affine transformation matrix, is the intrinsic parameter of the camera, is the rotation matrix, is the translation vector, , is the external parameter of the camera; according to the affine transformation matrix, the visual bird's-eye view features are affine transformed according to the formula to obtain the visual bird's-eye view features aligned with the spatial position of the point cloud bird's-eye view features; where, is the visual bird's-eye view feature after alignment, Visual bird's-eye view features for affine transformation.

[0009] Optionally, the sine function and the cosine function are used to embed the position codes for the visual bird's-eye view features and the point cloud bird's-eye view features after spatial position alignment, specifically including: using the sine function and the cosine function, according to the formula and , generate position codes for the visual bird's-eye view features and point cloud bird's-eye view features after spatial position alignment; among them, is the lateral position index of the bird's-eye view feature, is the vertical position index of the bird's-eye view feature, For the position code whose vertical position index is an even position, For the position code with the odd-numbered vertical position index, is the number of dimensions of the bird’s-eye view feature, is a sine function, is the cosine function; according to the formula , adding position encoding to the visual bird's-eye view features after spatial position alignment; where, To add the visual bird's-eye view features after position encoding, is the visual bird's-eye view feature after spatial position alignment, Represents the position encoding generated by the visual bird's-eye view feature; according to the formula , add position coding to the point cloud bird's-eye view features after spatial position alignment; where, To add the point cloud bird's-eye view features after position coding, It is the feature of the point cloud bird's-eye view after spatial position alignment. Represents the position encoding generated for the bird's-eye view features of a point cloud.

[0010] Optionally, quality inspection of automobile parts is performed according to the fusion features to obtain quality inspection results, specifically including: according to the fusion features, using a deep learning model with multiple detection heads to perform quality inspection on automobile parts to identify surface defects, geometric dimension deviations and structural anomalies.

[0011] In the second aspect, the present application provides a quality inspection device for automobile parts, comprising: a camera, a laser radar (Light Detection and Ranging, LiDAR) and a computer. The camera and the laser radar are both connected to the computer; the camera is used to capture the surface image of the automobile part; the laser radar is used to collect the three-dimensional point cloud data of the automobile part; the computer is used to perform quality inspection on the automobile part according to the surface image and the three-dimensional point cloud data of the automobile part using the above-mentioned quality inspection method for automobile parts.

[0012] Optionally, the quality inspection device for automobile parts further includes: a first track, a second track, a first mechanical arm, and a second mechanical arm. The first track and the second track are arranged on both sides of the automobile part; the first mechanical arm is slidably arranged on the first track, and the second mechanical arm is slidably arranged on the second track; the camera is arranged on the first mechanical arm, and the laser radar is arranged on the second mechanical arm; the first mechanical arm is used to slide along the first track so that the camera can take surface images of automobile parts at different positions and angles; the second mechanical arm is used to slide along the second track so that the laser radar can collect three-dimensional point cloud data of automobile parts at different positions and angles.

[0013] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned automobile parts quality inspection method.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned automobile parts quality inspection method when executed by a processor.

[0015] According to the specific embodiments provided in this application, this application has the following technical effects.

[0016] The present application provides a method, device, equipment and medium for quality inspection of automobile parts. By fusing the multimodal features of surface images and three-dimensional point cloud data, and combining the advantages of visual perception and point cloud data, it can accurately and quickly detect surface defects, geometric dimension deviations and structural anomalies of automobile parts, thereby ensuring the quality of automobile parts during the manufacturing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A schematic flow chart of a method for quality inspection of automobile parts provided in one embodiment of the present application.

[0019] Figure 2 A schematic diagram of a framework of a method for quality inspection of automobile parts provided in one embodiment of the present application.

[0020] Figure 3 A schematic diagram of the working process of the Transformer model provided in another embodiment of the present application.

[0021] Figure 4 A schematic structural diagram of a quality inspection device for automobile parts provided in one embodiment of the present application.

[0022] Figure 5 A schematic diagram of the structure of a computer device provided in one embodiment of the present application.

[0023] Figure numerals: camera-1, lidar-2, first track-3, second track-4, first robotic arm-5, second robotic arm-6. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0025] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0026] The introduction of multimodal fusion perception technology provides an innovative solution for the quality inspection of automotive parts. Multimodal fusion perception technology realizes multi-angle and multi-level inspection of the appearance and geometric dimensions of automotive parts by fusing data from multiple sensors, such as vision and laser radar. This detection method can make up for the shortcomings of single sensor technology and greatly improve the comprehensiveness, accuracy and efficiency of detection. In the manufacturing of automotive parts, common appearance defects (such as scratches, dents, etc.), tiny cracks and layer separation inside the material will directly affect the performance of the whole vehicle. Although single visual inspection has certain capabilities in identifying surface defects of automotive parts, it has limited capabilities in detecting geometric dimensions and complex structural information. Laser radar can generate high-precision three-dimensional point cloud data, thereby accurately detecting the shape, dimensional deviation and complex geometric features of parts. Therefore, how to efficiently fuse data from different perception technologies to achieve all-round inspection of the quality of automotive parts is a key link in improving the quality of modern automobile manufacturing.

[0027] To this end, based on multimodal fusion perception, this application proposes a quality inspection method for automobile parts, which can efficiently and accurately complete the comprehensive inspection of the appearance and geometric information of automobile parts.

[0028] In an exemplary embodiment, Figure 1 As shown, a quality inspection method for automobile parts includes the following steps 101 to 107.

[0029] Step 101: Collect surface images and three-dimensional point cloud data of automobile parts.

[0030] Step 102: extracting visual bird's-eye view features of the surface image using a convolutional neural network.

[0031] Step 103: Use a deep learning model to extract point cloud bird's-eye view features of the three-dimensional point cloud data.

[0032] Step 104: spatially align the visual bird's-eye view features with the point cloud bird's-eye view features through affine transformation.

[0033] Step 105: Use sine function and cosine function to embed position codes for the visual bird's-eye view features and the point cloud bird's-eye view features after spatial position alignment.

[0034] Step 106: Input the position-encoded visual bird's-eye view features and the position-encoded point cloud bird's-eye view features into the Transformer model for deep fusion to obtain fused features.

[0035] Step 107: Perform quality inspection on automobile parts according to the fusion features to obtain quality inspection results; the quality inspection results include surface defects, geometric dimension deviations and structural abnormalities.

[0036] The implementation of the above steps 101 to 107 aims to provide an efficient and accurate quality inspection method for automobile parts by integrating multiple inspection methods such as vision and lidar. This method can effectively identify problems such as appearance defects and geometric dimension deviations of automobile parts, thereby improving the overall quality of automobile parts and reducing manufacturing costs. In view of the problems of single inspection means, insufficient inspection accuracy and efficiency in the prior art, this application improves the comprehensiveness, automation and intelligence of the inspection process by extracting bird's-eye-view (BEV) features from image and point cloud data and combining multimodal feature fusion technology. A multimodal fusion perception algorithm is used to achieve efficient and accurate quality inspection of automobile parts (e.g., body parts). This method uses multimodal perception technology, which can not only improve the manufacturing quality of body parts, but also effectively reduce production costs.

[0037] The framework of a quality inspection method for automobile parts of the present application is as follows Figure 2 shown. Figure 2 The method shown can be divided into three parts: visual BEV feature extraction, point cloud BEV feature extraction, and multimodal feature fusion and perception. The core of visual BEV feature extraction lies in its accurate detection of appearance defects, which can automatically capture tiny defects on the surface of parts, including scratches, dents, wear and tear, etc. Through the feature learning of convolutional neural networks, it is possible to monitor the surface of automotive parts in mass production in real time, automatically judge the surface quality and feedback the test results. Multimodal feature fusion and perception is the core of the entire method. Its main task is to align and fuse visual BEV features and point cloud BEV features to achieve comprehensive quality inspection of automotive parts.

[0038] In another exemplary embodiment of the present application, in order to comprehensively inspect the appearance and geometric structure of automobile parts, it is first necessary to collect surface images and three-dimensional point cloud data of automobile parts. In the present application, cameras (e.g., high-resolution cameras) and laser radar devices can be used as main perception sensors. The three-dimensional point cloud data of automobile parts is collected by laser radar, and at the same time, a high-resolution camera is used to capture high-definition surface images of parts.

[0039] In another exemplary embodiment of the present application, appearance features are extracted from the surface image of the automobile part. The surface image of the automobile part is acquired by a camera, and the surface image is analyzed and processed using a deep learning algorithm. Specifically, through deep learning models such as Convolutional Neural Networks (CNN), key features such as texture, color, shape, and geometric contour of automobile parts can be automatically identified and extracted. Compared with traditional manual visual inspection, this application significantly improves the detection accuracy and efficiency by introducing deep learning technology, and can better deal with the complex and diverse defect types on the surface of automobile parts.

[0040] The extraction process of visual BEV features is as follows.

[0041] First, the surface image is preprocessed to obtain a standardized image suitable for the convolutional neural network input. The preprocessing process is as follows: First, the surface image is denoised using a filtering algorithm to enhance the quality of the surface image and eliminate unnecessary noise. Next, the size and resolution of the surface image are adjusted to match the input layer size of the convolutional neural network. This step includes scaling or cropping the surface image to ensure that the dimension of the input data is consistent with the expected input format of the model, thereby ensuring that the subsequent feature extraction and learning process can proceed normally.

[0042] Secondly, based on the preprocessed surface image, visual BEV features are extracted through a convolutional neural network model; specifically, the convolutional neural network model is ResNet; the preprocessed surface image is input into ResNet to obtain visual BEV features.

[0043] Therefore, the above step 102 can be replaced by the following steps 201 to 203.

[0044] Step 201: De-noise the surface image using a filtering algorithm.

[0045] Step 202: Adjust the size and resolution of the denoised surface image so that the format of the adjusted surface image matches the format of the input data of the input layer of the convolutional neural network.

[0046] Step 203: Input the adjusted surface image into ResNet to obtain visual bird's-eye view features of the surface image.

[0047] In another exemplary embodiment of the present application, three-dimensional point cloud data generated by laser radar is used for geometric feature extraction. Laser radar generates high-precision point cloud data by emitting a laser beam to the surface of the part and receiving its reflected signal. These point cloud data contain the three-dimensional coordinates and geometric shape information of the automobile parts. The point cloud data are processed using a deep learning model to extract the geometric features, spatial distribution features and size information of the automobile parts. Laser radar can efficiently detect automobile parts with complex geometric structures, especially those with complex shapes and irregular appearances. Compared with traditional measuring tools, the three-dimensional point cloud data generated by laser radar can cover all the geometric information of automobile parts, ensuring the comprehensiveness and high precision of the detection. Therefore, the present application can not only meet the needs of geometric dimension detection, but also identify structural anomalies inside automobile parts, greatly expanding the scope of application of quality detection.

[0048] The process of extracting point cloud BEV features from three-dimensional point cloud data is as follows.

[0049] First, the 3D point cloud data is preprocessed to obtain standardized point cloud data suitable for deep learning model input. The preprocessing process is as follows: first, the 3D point cloud data is denoised to remove noise and outliers and improve data quality; second, the 3D point cloud data is downsampled to reduce redundant data and control the data scale to make it suitable for the input requirements of the model; then, coordinate normalization is performed to normalize the 3D point cloud data to a uniform scale range to ensure data consistency; finally, the 3D point cloud data is aligned and rotated according to the requirements of the model to ensure that the 3D point cloud data is consistent from different perspectives.

[0050] Secondly, based on the preprocessed 3D point cloud data, the point cloud BEV features are extracted through a deep learning model; specifically, the deep learning model is Pointpillars; the preprocessed 3D point cloud data is input into the deep learning model, and the deep learning model first extracts features from the input 3D point cloud data, and encodes the 3D spatial coordinates (x, y, z) and reflection intensity of each point into a high-dimensional feature vector to characterize the local characteristics of the 3D point cloud data. Next, the 3D point cloud data is divided into multiple pillars, and the point features in each pillar are pooled to generate a pillar-level feature representation, namely, pillar features, to reduce data sparsity and retain geometric information. Then, these pillar features are input into the convolutional neural network, and after a series of operations such as convolutional layers and pooling layers, the global and local features of the 3D point cloud data from a bird's-eye view are further extracted. Finally, the deep learning model outputs the BEV feature map of the 3D point cloud data, which retains the spatial distribution and geometric structure information of the 3D point cloud data, and can provide effective support for subsequent tasks (such as target detection, size measurement, etc.).

[0051] Therefore, the above step 103 can be replaced by the following steps 301 to 305.

[0052] Step 301: De-noising the three-dimensional point cloud data.

[0053] Step 302: down-sample the denoised three-dimensional point cloud data.

[0054] Step 303: normalize the coordinates of the downsampled three-dimensional point cloud data.

[0055] Step 304: performing point cloud alignment and rotation standardization on the three-dimensional point cloud data after coordinate normalization to obtain pre-processed three-dimensional point cloud data.

[0056] Step 305: Input the preprocessed three-dimensional point cloud data into the Pointpillars network to obtain point cloud bird's-eye view features of the three-dimensional point cloud data.

[0057] In order to accurately obtain the geometric shape and size information of automobile parts, and extract the surface texture and color information of automobile parts, the data of LiDAR and camera are pre-processed to generate accurate surface images (visual data) and 3D point cloud data, which can be used to construct the 3D geometric model and surface appearance features of automobile parts, so as to conduct a comprehensive inspection of the geometric size and surface quality of parts.

[0058] In another exemplary embodiment of the present application, the visual BEV features and the point cloud BEV features are preliminarily aligned in space by using affine transformation combined with the internal and external parameters of the camera. Then the above step 104 can be replaced by the following steps 401 to 402.

[0059] Step 401: Based on the intrinsic and extrinsic parameters of the camera, according to the formula , obtaining an affine transformation matrix; wherein the camera is a camera for taking surface images of automobile parts; is the affine transformation matrix, is the intrinsic parameter of the camera, is the rotation matrix, is the translation vector, , are the extrinsic parameters of the camera.

[0060] Camera intrinsic parameters The specific form is: ; and is the focal length, and are the optical center coordinates.

[0061] Step 402: According to the affine transformation matrix, according to the formula Affine transformation is performed on the visual bird's-eye view features to obtain the visual bird's-eye view features aligned with the spatial position of the point cloud bird's-eye view features; wherein, is the visual bird's-eye view feature after alignment, Visual bird's-eye view features for affine transformation.

[0062] In another exemplary embodiment of the present application, after the visual BEV features and the point cloud BEV features are spatially aligned, in order to further improve the accuracy of feature alignment, the present application uses a unified position encoding method to further implicitly align the features of the two modalities. Position encoding helps the system better maintain the spatial consistency of different feature points during the fusion process by embedding position information for each feature. Position encoding enables the system to perceive the relative positions of feature points through encodings of different frequencies generated by sine and cosine functions. Then the above step 105 can be replaced by the following steps 501 to 503.

[0063] Step 501: Use sine function and cosine function according to the formula and , generate position codes for the visual bird's-eye view features and point cloud bird's-eye view features after spatial position alignment; among them, is the lateral position index of the bird's-eye view feature, is the vertical position index of the bird's-eye view feature, For the position code whose vertical position index is an even position, For the position code with the odd-numbered vertical position index, is the number of dimensions of the bird’s-eye view feature, is a sine function, is the cosine function.

[0064] Step 502: According to the formula , adding position encoding to the visual bird's-eye view features after spatial position alignment; where, To add the visual bird's-eye view features after position encoding, is the visual bird's-eye view feature after spatial position alignment, Represents the position encoding generated for the visual bird's-eye view features.

[0065] Step 503: According to the formula , add position coding to the point cloud bird's-eye view features after spatial position alignment; where, To add the point cloud bird's-eye view features after position coding, It is the feature of the point cloud bird's-eye view after spatial position alignment. Represents the position encoding generated for the bird's-eye view features of a point cloud.

[0066] In another exemplary embodiment of the present application, the Transformer network is used to deeply fuse the features of the two modalities to generate a unified multimodal feature representation. The Transformer network can further implicitly align the features of the two modalities in space, and can automatically learn the correlation and difference between the multimodal features, giving full play to the advantages of both and improving the accuracy of detection. Figure 3 As shown in the figure, the multi-head self-attention mechanism of the Transformer network generates a fused feature representation by calculating the similarity between each feature point. This mechanism can effectively capture the spatial correlation between visual and point cloud features, thereby improving the accuracy of multimodal feature fusion. The Transformer network learns the global relationship between different modal features through multiple stacked self-attention layers, thereby generating a unified multimodal feature representation. This process can be expressed as the following formula.

[0067] , , .

[0068] .

[0069] .

[0070] in, , and are query vector, key vector and value vector respectively, , and are different attention weight matrices, Represents the key vector The dimension of is the obtained attention value, is the normalized exponential function, is the transpose of the key vector, For fusion features.

[0071] In another exemplary embodiment of the present application, referring to Figure 3 After performing affine transformation on the visual bird's-eye view features to obtain the visual bird's-eye view features that are spatially aligned with the point cloud bird's-eye view features, a linear layer can be used to perform linear transformation on the spatially aligned visual bird's-eye view features and the spatially aligned point cloud bird's-eye view features. The colored squares obtained after the linear transformation of the spatially aligned visual bird's-eye view features represent the features at different spatial positions in the visual bird's-eye view features, and the colored squares obtained after the linear transformation of the spatially aligned point cloud bird's-eye view features represent the features at different spatial positions in the point cloud bird's-eye view features. The corresponding colored squares represent the position encodings of different spatial positions generated by the visual bird’s-eye view features. The corresponding colored squares represent the position encodings of different spatial locations generated for the point cloud bird’s-eye view features. Multilayer Perceptron and Fusion Features The colored squares between represent the fusion of visual bird's-eye view features and point cloud bird's-eye view features at different spatial locations.

[0072] In another exemplary embodiment of the present application, after the multimodal fusion of the Transformer model, a unified fusion feature representation will be generated. Next, a more comprehensive quality inspection of automobile parts will be performed based on the fusion features. The quality inspection tasks include appearance defect detection, geometric dimension deviation analysis, and structural anomaly detection. Then the above step 107 can be: according to the fusion features, a deep learning model with multiple detection heads is used to perform quality inspection on automobile parts to identify surface defects, geometric dimension deviations, and structural anomalies.

[0073] In general, the combination of visual inspection and LiDAR point cloud inspection enables comprehensive inspection of automotive parts, which can accurately identify appearance defects and efficiently measure geometric size deviations. Through the fusion of multimodal features, the inspection process has achieved a high degree of automation and intelligence, significantly improving inspection efficiency and reducing human errors, providing an advanced quality inspection solution for the automotive manufacturing industry.

[0074] The beneficial effects of the method of the present application are as follows.

[0075] 1. The quality inspection method for automobile parts proposed in this application can accurately detect appearance defects, geometric dimension deviations and other problems of automobile parts by integrating multiple sensing methods such as vision and lidar, providing a high-precision and high-efficiency solution for the quality inspection of automobile parts.

[0076] 2. This application uses deep fusion of multimodal perception data to construct a comprehensive quality inspection model using convolutional neural networks, deep learning models, and deep learning models with multiple detection heads, combining the advantages of visual perception and point cloud data to achieve comprehensive inspection of the appearance and geometric shape of automotive parts. This method significantly improves the accuracy and comprehensiveness of inspection, reduces errors and blind spots in traditional inspection methods, and ensures the high quality of automotive parts during the manufacturing process.

[0077] The comprehensive quality inspection model learns to identify defect-related features on the surface of automotive parts by training on a large number of high-definition images with defects annotated. Through convolutional neural networks, local and global information in images can be automatically extracted and analyzed to identify different types of surface defects, such as scratches, dents, cracks, etc. The comprehensive quality inspection model will classify and mark the inspection results according to the shape, size and location of the defects to achieve high-precision surface defect detection to detect surface defects of automotive parts, such as scratches, dents, cracks, etc. Similarly, for dimensional measurement, global feature extraction is performed on the point cloud data of automotive parts to capture the three-dimensional geometric information of the parts. The deep learning model can accurately calculate the geometric dimensions and shape features of the parts. Combined with the point cloud features, the geometric dimensions of the parts can be accurately measured to ensure that they meet the design tolerance requirements. These inspection results will be used to provide real-time feedback to the production line to ensure that the quality control in the production process meets the expected goals.

[0078] 3. The quality inspection method for automobile parts of this application has wide applicability. It is not only suitable for automobile parts with complex structures or special materials, but also can support all aspects of parts research and development, production and quality control. By improving the inspection efficiency and automation level, this application helps to reduce production costs, improve production efficiency, and promote the development of the automobile manufacturing industry towards intelligence and automation.

[0079] Based on the same inventive concept, the embodiment of the present application also provides a quality inspection device for automobile parts for implementing the quality inspection method for automobile parts involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of one or more quality inspection devices for automobile parts provided below can refer to the limitations of the quality inspection method for automobile parts above, and will not be repeated here.

[0080] In an exemplary embodiment, Figure 4 As shown, a quality inspection device for automobile parts is provided, including: a camera 1, a laser radar 2 and a computer. The camera 1 and the laser radar 2 are both connected to the computer; the camera 1 is used to capture the surface image of the automobile part; the laser radar 2 is used to collect the three-dimensional point cloud data of the automobile part; the computer is used to perform quality inspection on the automobile part according to the surface image and the three-dimensional point cloud data of the automobile part using the above-mentioned quality inspection method for automobile parts.

[0081] As an optional embodiment, the quality inspection device for automobile parts further includes: a first track 3, a second track 4, a first mechanical arm 5, and a second mechanical arm 6. The first track 3 and the second track 4 are arranged on both sides of the automobile part; the first mechanical arm 5 is slidably arranged on the first track 3, and the second mechanical arm 6 is slidably arranged on the second track 4; the camera 1 is arranged on the first mechanical arm 5, and the laser radar 2 is arranged on the second mechanical arm 6. The first mechanical arm 5 is used to slide along the first track 3, so that the camera 1 can take surface images of automobile parts at different positions and angles. The second mechanical arm 6 is used to slide along the second track 4, so that the laser radar 2 can collect three-dimensional point cloud data at different positions and angles of automobile parts.

[0082] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store quality inspection results. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a quality inspection method for automobile parts is implemented.

[0083] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0084] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0085] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0086] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0087] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0088] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0089] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A quality inspection method for automobile parts, characterized in that: include: Collect surface images and 3D point cloud data of automotive parts; extracting visual bird's-eye view features of the surface image using a convolutional neural network; Extracting point cloud bird's-eye view features of the three-dimensional point cloud data using a deep learning model; Through affine transformation, the visual bird's-eye view features and the point cloud bird's-eye view features are spatially aligned; Sine function and cosine function are used to embed position codes for the visual bird's-eye view features and point cloud bird's-eye view features after spatial position alignment. The position-encoded visual bird's-eye view features and the position-encoded point cloud bird's-eye view features are input into the Transformer model for deep fusion to obtain fused features; The quality inspection of automobile parts is performed according to the fusion features to obtain quality inspection results; the quality inspection results include surface defects, geometric size deviations and structural abnormalities.

2. The quality inspection method for automobile parts according to claim 1, characterized in that: The visual bird's-eye view features of the surface image are extracted using a convolutional neural network, specifically including: De-noising the surface image using a filtering algorithm; Adjusting the size and resolution of the denoised surface image so that the format of the adjusted surface image matches the format of the input data of the input layer of the convolutional neural network; The adjusted surface image is input into ResNet to obtain visual bird's-eye view features of the surface image.

3. The quality inspection method for automobile parts according to claim 1, characterized in that: The point cloud bird's-eye view features of the three-dimensional point cloud data are extracted using a deep learning model, specifically including: De-noising the three-dimensional point cloud data; Down-sample the denoised 3D point cloud data; Normalize the coordinates of the downsampled 3D point cloud data; Perform point cloud alignment and rotation standardization on the three-dimensional point cloud data after coordinate normalization to obtain pre-processed three-dimensional point cloud data; The preprocessed three-dimensional point cloud data is input into the Pointpillars network to obtain the point cloud bird's-eye view features of the three-dimensional point cloud data.

4. The quality inspection method for automobile parts according to claim 1, characterized in that: Through affine transformation, the visual bird's-eye view features and the point cloud bird's-eye view features are spatially aligned, including: According to the camera's internal and external parameters, according to the formula , obtaining an affine transformation matrix; wherein the camera is a camera for taking surface images of automobile parts; is the affine transformation matrix, is the intrinsic parameter of the camera, is the rotation matrix, is the translation vector, , are the external parameters of the camera; According to the affine transformation matrix, according to the formula Affine transformation is performed on the visual bird's-eye view features to obtain the visual bird's-eye view features aligned with the spatial position of the point cloud bird's-eye view features; wherein, is the visual bird's-eye view feature after alignment, Visual bird's-eye view features for affine transformation.

5. The quality inspection method for automobile parts according to claim 1, characterized in that: Sine function and cosine function are used to embed position codes for the visual bird's-eye view features and point cloud bird's-eye view features after spatial position alignment, including: Using sine and cosine functions, according to the formula and , generate position codes for the visual bird's-eye view features and point cloud bird's-eye view features after spatial position alignment; among them, is the lateral position index of the bird's-eye view feature, is the vertical position index of the bird's-eye view feature, For the position code whose vertical position index is an even position, For the position code with the odd-numbered vertical position index, is the number of dimensions of the bird’s-eye view feature, is a sine function, is the cosine function; According to the formula , adding position encoding to the visual bird's-eye view features after spatial position alignment; where, To add the visual bird's-eye view features after position encoding, is the visual bird's-eye view feature after spatial position alignment, Represents the position encoding generated for the visual bird's-eye view features; According to the formula , add position coding to the point cloud bird's-eye view features after spatial position alignment; where, To add the point cloud bird's-eye view features after position coding, It is the feature of the point cloud bird's-eye view after spatial position alignment. Represents the position encoding generated for the bird's-eye view features of a point cloud.

6. The quality inspection method for automobile parts according to claim 1, characterized in that: Performing quality inspection of automobile parts according to the fusion features to obtain quality inspection results specifically includes: Based on the fused features, a deep learning model with multiple detection heads is used to perform quality inspection on automotive parts to identify surface defects, geometric dimension deviations, and structural anomalies.

7. A quality inspection device for automobile parts, characterized in that: The quality inspection device for automobile parts comprises: a camera, a laser radar and a computer; Both the camera and the lidar are connected to a computer; The camera is used to capture the surface image of the automobile part; The laser radar is used to collect three-dimensional point cloud data of automobile parts; The computer is used to perform quality inspection on automobile parts according to the surface image and three-dimensional point cloud data of the automobile parts by using the automobile parts quality inspection method according to any one of claims 1 to 6.

8. The quality inspection device for automobile parts according to claim 7, characterized in that: The quality inspection device for automobile parts further comprises: a first track, a second track, a first mechanical arm and a second mechanical arm; The first track and the second track are arranged on both sides of the automobile part; the first mechanical arm is slidably arranged on the first track, and the second mechanical arm is slidably arranged on the second track; the camera is arranged on the first mechanical arm, and the laser radar is arranged on the second mechanical arm; The first mechanical arm is used to slide along the first track so that the camera can capture surface images of the automobile part at different positions and angles; The second robotic arm is used to slide along the second track, so that the laser radar collects three-dimensional point cloud data of automobile parts at different positions and angles.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the quality inspection method for automobile parts according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the quality inspection method for automobile parts according to any one of claims 1 to 6 is implemented.

Images