New energy vehicle shock absorber rod detection system and method based on machine vision
Through machine vision technology based on deep learning, the multi-scale semantic features of shock absorber rods of new energy vehicles are extracted, and the problems of inefficiency of traditional detection methods and influence of human factors are solved, and high accuracy and high efficiency shock absorber rod detection is achieved.
Patent Information
- Application Number
- CN202410663669.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-27
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-05-27
AI Technical Summary
Traditional shock absorber rod detection methods are inefficient and susceptible to human factors, and cannot meet the inspection requirements of complex design needs of new energy vehicles.
Using machine vision technology based on deep learning, by extracting multi-scale semantic features of the surface state image of the shock absorber rod, using the semantic feature difference between the surface state image of the shock absorber rod to be detected and the qualified shock absorber rod to be detected, intelligently judge the subtraction to be detected. Whether there are surface defects in the shock rod.
It improves the accuracy and efficiency of shock absorber quality detection, and reduces the false detection and missed detection rates caused by human factors.
Smart Images

Figure CN118396980B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and more specifically, to a new energy vehicle shock absorber rod detection system and method based on machine vision. Background Art
[0002] With the rapid development of the new energy vehicle industry, consumers have higher and higher requirements for vehicle quality and safety. As an important part of the suspension system of new energy vehicles, the performance of the shock absorber rod directly affects the driving stability and riding comfort of the vehicle. Therefore, accurate quality testing of the shock absorber rod is crucial to ensure the quality and safety of new energy vehicles.
[0003] Traditional shock absorber rod inspection methods mainly rely on manual visual inspection or simple mechanical inspection tools. These methods are not only inefficient, but also easily affected by human factors, resulting in low accuracy and reliability of the inspection results. At the same time, with the increasing complexity of the shock absorber rod design of new energy vehicles, the types and manifestations of its defects are becoming more and more diverse, and traditional inspection methods can no longer meet the needs of modern industrial production.
[0004] In recent years, machine vision technology has been widely used in the field of industrial production automation due to its advantages of high precision, high efficiency and non-contact detection. Therefore, a new energy vehicle shock absorber rod detection system and method based on machine vision is expected. Summary of the invention
[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a new energy vehicle shock absorber rod detection system and method based on machine vision, which uses machine vision technology based on deep learning to perform surface defect detection on the shock absorber rod of the new energy vehicle, extracts multi-scale semantic features of the surface state image of the shock absorber rod, and intelligently determines whether the shock absorber rod of the new energy vehicle to be detected has surface defects based on the semantic feature difference between the surface state image of the shock absorber rod to be detected and the surface state reference image of the qualified shock absorber rod. In this way, the accuracy and efficiency of the shock absorber rod quality detection can be effectively improved, while reducing the false detection and missed detection rates caused by human factors.
[0006] Accordingly, according to one aspect of the present application, a new energy vehicle shock absorber rod detection system based on machine vision is provided, which includes:
[0007] A shock absorber rod reference image retrieval module is used to extract a set of shock absorber rod surface state reference images marked as qualified from a background database;
[0008] A reference image essential feature extraction module is used to perform multi-scale semantic feature extraction and essential feature screening on the set of reference images of the surface state of the shock absorber rod marked as qualified to obtain a multi-scale essential semantic representation feature vector of the surface state of the shock absorber rod;
[0009] The image acquisition module of the shock absorber rod to be detected is used to obtain the surface state image of the shock absorber rod of the new energy vehicle to be detected collected by the camera;
[0010] The semantic feature extraction module of the image to be detected is used to extract multi-scale semantic features of the surface state image of the shock absorber rod of the new energy vehicle to be detected to obtain a multi-scale detection semantic feature vector of the surface state of the shock absorber rod;
[0011] The global matching metric module is used to determine whether the shock absorber rod of the new energy vehicle to be inspected has surface defects based on the global matching features between the multi-scale essential semantic representation feature vector of the shock absorber rod surface state and the multi-scale detection semantic feature vector of the shock absorber rod surface state.
[0012] According to another aspect of the present application, a method for detecting a shock absorber rod of a new energy vehicle based on machine vision is provided, which comprises:
[0013] Extracting a set of reference images of the surface conditions of the shock absorber rods marked as qualified from the background database;
[0014] Performing multi-scale semantic feature extraction and essential feature screening on the set of reference images of the surface state of the shock absorber rod marked as qualified to obtain a multi-scale essential semantic representation feature vector of the surface state of the shock absorber rod;
[0015] Acquire a surface state image of the shock absorber rod of the new energy vehicle to be inspected, which is captured by a camera;
[0016] Performing multi-scale semantic feature extraction on the surface state image of the shock absorber rod of the new energy vehicle to be detected to obtain a multi-scale detection semantic feature vector of the surface state of the shock absorber rod;
[0017] Based on the global matching features between the multi-scale essential semantic representation feature vector of the shock absorber rod surface state and the multi-scale detection semantic feature vector of the shock absorber rod surface state, it is determined whether the shock absorber rod of the new energy vehicle to be inspected has surface defects.
[0018] Compared with the prior art, the machine vision-based new energy vehicle shock absorber rod detection system and method provided by the present application uses machine vision technology based on deep learning to detect surface defects of the shock absorber rod of the new energy vehicle, extracts multi-scale semantic features of the surface state image of the shock absorber rod, and intelligently determines whether the shock absorber rod of the new energy vehicle to be detected has surface defects based on the semantic feature difference between the surface state image of the shock absorber rod to be detected and the surface state reference image of the qualified shock absorber rod. In this way, the accuracy and efficiency of the shock absorber rod quality detection can be effectively improved, while reducing the false detection and missed detection rate caused by human factors. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.
[0020] Figure 1 The block diagram of the new energy vehicle shock absorber rod detection system based on machine vision according to an embodiment of the present application.
[0021] Figure 2 Schematic diagram of the architecture of a new energy vehicle shock absorber rod detection system based on machine vision according to an embodiment of the present application.
[0022] Figure 3 The present invention is a block diagram of a reference image essential feature extraction module in a machine vision-based new energy vehicle shock absorber rod detection system according to an embodiment of the present application.
[0023] Figure 4 The present invention is a block diagram of a semantic feature extraction module for an image to be detected in a machine vision-based new energy vehicle shock absorber rod detection system according to an embodiment of the present application.
[0024] Figure 5 The block diagram is a global matching metric module in a machine vision-based new energy vehicle shock absorber rod detection system according to an embodiment of the present application.
[0025] Figure 6 The block diagram is a training module in the machine vision-based new energy vehicle shock absorber rod detection system according to an embodiment of the present application.
[0026] Figure 7 The present invention is a flowchart of a method for detecting a shock absorber rod of a new energy vehicle based on machine vision according to an embodiment of the present application. DETAILED DESCRIPTION
[0027] Below, the embodiments of the present application will be described in more detail in conjunction with the accompanying drawings, and the above and other purposes, features and advantages of the present application will become more apparent. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein.
[0028] As mentioned in the above background technology, traditional shock absorber rod detection methods mainly rely on artificial vision or simple mechanical detection tools. These detection methods are not only inefficient, but also easily interfered by human factors, which leads to doubts about the accuracy and reliability of the detection results. In addition, with the increase in the complexity of the design of shock absorber rods of new energy vehicles, the types and manifestations of its defects are also becoming increasingly rich, and traditional detection methods can no longer meet the actual needs of modern industrial production. In response to the above technical problems, the technical concept of this application is to use machine vision technology based on deep learning to perform surface defect detection on the shock absorber rods of new energy vehicles, extract multi-scale semantic features of the surface state image of the shock absorber rod, and based on the semantic feature differences between the surface state image of the shock absorber rod to be detected and the surface state reference image of a qualified shock absorber rod, intelligently judge whether the shock absorber rod of the new energy vehicle to be detected has surface defects. In this way, the accuracy and efficiency of the shock absorber rod quality detection can be effectively improved, while reducing the false detection and missed detection rates caused by human factors.
[0029] Figure 1 The block diagram of the new energy vehicle shock absorber rod detection system based on machine vision according to an embodiment of the present application. Figure 2 FIG. 1 is a schematic diagram of the architecture of a new energy vehicle shock absorber rod detection system based on machine vision according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the embodiment of the present application, the new energy vehicle shock absorber rod detection system 100 based on machine vision includes: a shock absorber rod reference image retrieval module 110, which is used to extract a set of shock absorber rod surface state reference images marked as qualified from the background database; a reference image essential feature extraction module 120, which is used to perform multi-scale semantic feature extraction and essentialized feature screening on the set of shock absorber rod surface state reference images marked as qualified to obtain a shock absorber rod surface state multi-scale essential semantic representation feature vector; a shock absorber rod image acquisition module 130 to be detected, which is used to obtain a surface state image of the shock absorber rod to be detected by a camera; a to-be-detected image semantic feature extraction module 140, which is used to perform multi-scale semantic feature extraction on the surface state image of the shock absorber rod to be detected to obtain a shock absorber rod surface state multi-scale detection semantic feature vector; a global matching metric module 150, which is used to determine whether the shock absorber rod to be detected has surface defects based on the global matching features between the shock absorber rod surface state multi-scale essential semantic representation feature vector and the shock absorber rod surface state multi-scale detection semantic feature vector.
[0030] In the above-mentioned machine vision-based new energy vehicle shock absorber rod detection system 100, the shock absorber rod reference image retrieval module 110 is used to extract a set of shock absorber rod surface state reference images marked as qualified from the background database. It should be understood that the shock absorber rod surface state reference image marked as qualified represents the standard or expected "normal" state of the shock absorber rod surface state, provides a benchmark and reference for defect detection, and is the key basis for determining whether the shock absorber rod has surface defects. By comparing and analyzing the shock absorber rod surface state image to be detected with the reference image marked as qualified, it is helpful to quickly and accurately determine whether the shock absorber rod to be detected has abnormalities or defects. In addition, in the technical solution of the present application, by retrieving a rich and diverse set of shock absorber rod surface state reference images, it is helpful to adapt to different environments, lighting conditions and changes in shock absorber rod design, and enhance the adaptability and robustness of the detection system, so that it can accurately detect surface defects of the shock absorber rod under various circumstances.
[0031] In the above-mentioned machine vision-based new energy vehicle shock absorber rod detection system 100, the reference image essential feature extraction module 120 is used to extract multi-scale semantic features and screen essential features of the set of shock absorber rod surface state reference images marked as qualified to obtain a multi-scale essential semantic representation feature vector of the shock absorber rod surface state. Figure 3 FIG. 1 is a block diagram of a reference image essential feature extraction module in a new energy vehicle shock absorber rod detection system based on machine vision according to an embodiment of the present application. Figure 3 As shown, the reference image essential feature extraction module 120 includes: a surface state feature extraction unit 121, which is used to pass each shock absorber rod surface state reference image marked as qualified in the set of shock absorber rod surface state reference images marked as qualified through a surface state feature extractor based on a DenseNet model to obtain a set of shock absorber rod surface state reference feature maps; a multi-scale semantic feature extraction unit 122, which is used to pass each shock absorber rod surface state reference feature map in the set of shock absorber rod surface state reference feature maps through a multi-scale semantic feature extractor to obtain a set of shock absorber rod surface state multi-scale semantic feature vectors; an essentialized feature screening unit 123, which is used to pass the set of shock absorber rod surface state multi-scale semantic feature vectors through a feature screening essentialized network based on a gating mechanism to obtain the shock absorber rod surface state multi-scale essential semantic representation feature vector.
[0032] Specifically, the surface state feature extraction unit 121 is used to pass each of the shock absorber rod surface state reference images marked as qualified in the set of shock absorber rod surface state reference images marked as qualified through a surface state feature extractor based on the DenseNet model to obtain a set of shock absorber rod surface state reference feature maps. It should be understood that, considering the complexity and diversity of the surface state of the shock absorber rod, in the technical solution of the present application, the DenseNet model is used as a surface state feature extractor to process the set of shock absorber rod surface state reference images marked as qualified, so as to make full use of the advantages of the DenseNet model in image feature extraction and mine the shock absorber rod surface state information in each shock absorber rod surface state reference image. Those skilled in the art should know that the DenseNet model constructs a densely connected convolutional network so that the output of each layer can be used as the input of the subsequent layer, which effectively improves the reusability of features and the efficiency of information transmission. That is to say, the surface state feature extractor based on the DenseNet model can effectively extract more robust and generalized shock absorber rod surface state features through its dense connection and feature reuse mechanism, so as to fully reflect the details and texture information of the shock absorber rod surface, thereby providing strong data support for subsequent defect detection.
[0033] Specifically, the multi-scale semantic feature extraction unit 122 is used to pass each shock absorber rod surface state reference feature map in the set of the shock absorber rod surface state reference feature map through a multi-scale semantic feature extractor to obtain a set of multi-scale semantic feature vectors of the shock absorber rod surface state. It should be understood that since the surface defects of the shock absorber rod often have different scales and manifestations, such as tiny scratches, larger wear areas, etc., a single scale feature extraction may be difficult to fully capture the surface defects of the shock absorber rod. Therefore, in the technical solution of the present application, a multi-scale semantic feature extractor is further used to process the shock absorber rod surface state reference feature map to extract multi-scale semantic features of the shock absorber rod surface state. Specifically, the multi-scale semantic feature extractor mines the global semantic information and local semantic information of the shock absorber rod surface state through convolution operations of multiple scales, comprehensively describes the surface state of the shock absorber rod from both global and local levels, fully reflects the complexity and diversity of the surface state of the shock absorber rod, and helps to accurately identify defects of various types and scales in the subsequent defect detection process, thereby achieving accurate detection of shock absorber rod surface defects.
[0034] In a specific example of the present application, the multi-scale semantic feature extraction unit 122 includes: a multi-scale semantic encoding sub-unit, used to respectively extract the global semantic features and local semantic features of the shock absorber rod surface state reference feature map to obtain a global semantic feature vector of the shock absorber rod surface state and a local semantic feature vector of the shock absorber rod surface state; a multi-scale semantic fusion sub-unit, used to fuse the global semantic feature vector of the shock absorber rod surface state and the local semantic feature vector of the shock absorber rod surface state to obtain a multi-scale semantic feature vector of the shock absorber rod surface state.
[0035] More specifically, the multi-scale semantic encoding subunit is used to: upsample the shock absorber rod surface state reference feature map to obtain an upsampled shock absorber rod surface state reference feature map; perform global mean pooling processing on the upsampled shock absorber rod surface state reference feature map to obtain an upsampled shock absorber rod surface state reference semantic channel feature vector; perform point convolution encoding on the upsampled shock absorber rod surface state reference semantic channel feature vector to obtain the shock absorber rod surface state global semantic feature vector; perform point convolution encoding on the upsampled shock absorber rod surface state reference feature map to obtain a channel-modulated upsampled shock absorber rod surface state reference feature map; perform two-dimensional convolution encoding on the channel-modulated upsampled shock absorber rod surface state reference feature map to obtain the shock absorber rod surface state local semantic feature vector.
[0036] Specifically, the essentialized feature screening unit 123 is used to pass the set of multi-scale semantic feature vectors of the surface state of the shock absorber rod through a feature screening essentialized network based on a gating mechanism to obtain the multi-scale essential semantic representation feature vector of the surface state of the shock absorber rod. It should be understood that the set of multi-scale semantic feature vectors of the surface state of the shock absorber rod respectively reflects the semantic information of each image in the set of reference images of the surface state of the shock absorber rod. However, due to the differences in factors such as lighting conditions and shooting angles that may exist between the reference images of the surface state of the shock absorber rod, the multi-scale essential semantic representation feature vector of the surface state of the shock absorber rod may contain some redundant or irrelevant information, thereby interfering with subsequent defect detection. Therefore, in the technical solution of the present application, a feature screening essentialized network based on a gating mechanism is further used to screen and integrate the set of multi-scale semantic feature vectors of the surface state of the shock absorber rod to obtain a more essential and refined multi-scale essential semantic representation feature vector of the surface state of the shock absorber rod. Specifically, the feature screening essentialization network based on the gating mechanism introduces a gating mechanism to dynamically select and weight the set of multi-scale semantic feature vectors of the surface state of the shock absorber rod, so that important features are enhanced and redundant or irrelevant features are suppressed, so as to achieve effective screening and essentialization of the set of multi-scale semantic feature vectors of the surface state of the shock absorber rod, thereby further reducing noise and redundancy in the data and improving the accuracy and robustness of defect detection.
[0037] In a specific example of the present application, the essentialized feature screening unit 123 includes: a semantic difference measurement subunit, which is used to calculate the semantic difference coefficient of each shock absorber rod surface state multi-scale semantic feature vector in the set of the shock absorber rod surface state multi-scale semantic feature vector relative to other shock absorber rod surface state multi-scale semantic feature vectors using the following semantic difference measurement formula to obtain a set of semantic difference coefficients, wherein the semantic difference measurement formula is: ;in, is the first in the set of multi-scale semantic feature vectors of the surface state of the shock absorber rod. A multi-scale semantic feature vector of the shock absorber surface state, It is A multi-scale semantic feature vector of the shock absorber surface state, Indicates the feature vector The eigenvalues of the positions, is the length of the multi-scale semantic feature vector of the surface state of the shock absorber rod, Represents the logarithmic function operation with base 2, The value of is the number of multi-scale semantic feature vectors of the surface state of the shock absorber rod minus one, For the The invention relates to a method for obtaining a plurality of shock absorber rods having a plurality of surface states and a plurality of multi-scale semantic feature vectors of the surface states of the shock absorber rods; a normalization subunit, used for normalizing the set of the semantic difference coefficients to obtain a set of normalized semantic difference coefficients; a masking subunit, used for using a gating unit to mask the set of the normalized semantic difference coefficients to obtain a set of masked semantic difference coefficients; and a semantic feature screening subunit, used for using the set of masked semantic difference coefficients as a weight to calculate the weighted sum of the set of multi-scale semantic feature vectors of the surface states of the shock absorber rods to obtain a multi-scale essential semantic representation feature vector of the surface states of the shock absorber rods.
[0038] In the above-mentioned machine vision-based new energy vehicle shock absorber rod detection system 100, the shock absorber rod image acquisition module 130 to be detected is used to obtain the surface state image of the shock absorber rod to be detected by the camera. It should be understood that in practical applications, for the surface defect detection of the shock absorber rod of the new energy vehicle, the defect characterization of the shock absorber rod can be obtained by collecting the surface state image of the shock absorber rod to be detected. Therefore, in the technical solution of the present application, the surface state image of the shock absorber rod to be detected of the new energy vehicle is collected in real time by the camera to provide direct visual information of the surface of the shock absorber rod to be detected, reflecting the actual state of the surface of the shock absorber rod, thereby realizing real-time and online detection of the shock absorber rod of the new energy vehicle to improve the detection efficiency and automation.
[0039] In the above-mentioned machine vision-based new energy vehicle shock absorber rod detection system 100, the image semantic feature extraction module 140 to be detected is used to extract multi-scale semantic features from the surface state image of the shock absorber rod to be detected to obtain a multi-scale detection semantic feature vector of the surface state of the shock absorber rod. Figure 4 FIG. 1 is a block diagram of a semantic feature extraction module of an image to be detected in a new energy vehicle shock absorber rod detection system based on machine vision according to an embodiment of the present application. Figure 4 As shown, the semantic feature extraction module 140 of the image to be detected includes: a detail enhancement unit 141, which is used to perform contrast normalization and generalized Gaussian fitting on the surface state image of the shock absorber rod of the new energy vehicle to be detected to obtain a detail enhanced surface state image; a surface state multi-scale encoding unit 142, which is used to pass the detail enhanced surface state image through the surface state feature extractor based on the DenseNet model and the multi-scale semantic feature extractor to obtain the multi-scale detection semantic feature vector of the surface state of the shock absorber rod.
[0040] Specifically, the detail enhancement unit 141 is used to perform contrast normalization and generalized Gaussian fitting on the surface state image of the shock absorber rod of the new energy vehicle to be detected to obtain a detail enhanced surface state image. It should be understood that due to the influence of the actual shooting environment and shooting equipment, the surface state image of the shock absorber rod of the new energy vehicle to be detected may have problems such as uneven illumination and poor contrast, which will have a certain impact on the subsequent defect detection. Therefore, in the technical solution of the present application, in order to improve the image quality, the surface state image of the shock absorber rod of the new energy vehicle to be detected is first subjected to contrast normalization processing to eliminate the influence of uneven illumination on the image quality, so that each area in the image has a similar contrast. Then, the normalized image is subjected to detail enhancement processing by generalized Gaussian fitting, and the brightness distribution of the image is modeled and adjusted to make the brightness distribution of the image more uniform and reasonable, so as to further improve the visual effect of the image, highlight the details and texture information of the surface of the shock absorber rod, such as tiny cracks, scratches, etc., so that the potential defect information of the shock absorber rod is more obvious and easy to identify in the image, thereby improving the accuracy and reliability of subsequent defect detection.
[0041] Specifically, the surface state multi-scale encoding unit 142 is used to pass the detail enhanced surface state image through the surface state feature extractor based on the DenseNet model and the multi-scale semantic feature extractor to obtain the multi-scale detection semantic feature vector of the shock absorber rod surface state. That is, for the detail enhanced surface state image, the same feature extraction method as the reference image is used to extract the multi-scale semantic features of the surface state of the shock absorber rod to be detected. Specifically, firstly, the detail enhanced surface state image is subjected to feature extraction by the surface state feature extractor based on the DenseNet model, so as to utilize the advantages of the DenseNet model in image feature extraction and extract a robust and generalized feature representation of the surface state of the shock absorber rod. Then, the feature representation extracted by the DenseNet model is input into the multi-scale semantic feature extractor to mine the global semantic information and local semantic information of the surface state of the shock absorber rod to be detected, so as to more comprehensively describe the surface state of the shock absorber rod. In this way, the surface state image of the shock absorber rod to be detected is converted into a feature representation similar to the reference image, so as to facilitate subsequent feature matching and defect detection.
[0042] In the above-mentioned machine vision-based new energy vehicle shock absorber rod detection system 100, the global matching metric module 150 is used to determine whether the shock absorber rod of the new energy vehicle to be detected has surface defects based on the global matching features between the multi-scale essential semantic representation feature vector of the shock absorber rod surface state and the multi-scale detection semantic feature vector of the shock absorber rod surface state. It should be understood that the multi-scale essential semantic representation feature vector of the shock absorber rod surface state and the multi-scale detection semantic feature vector of the shock absorber rod surface state respectively represent the surface state features of the qualified shock absorber rod and the shock absorber rod to be detected. By matching and analyzing the two, it can be determined whether the surface state of the shock absorber rod to be detected is abnormal or defective.
[0043] Figure 5 FIG. 1 is a block diagram of a global matching metric module in a new energy vehicle shock absorber rod detection system based on machine vision according to an embodiment of the present application. Figure 5 As shown, the global matching metric module 150 includes: a global matching coefficient calculation unit 151, which is used to calculate the global matching coefficient between the multi-scale essential semantic representation feature vector of the shock absorber surface state and the multi-scale detection semantic feature vector of the shock absorber surface state; and a surface defect diagnosis unit 152, which is used to determine whether the shock absorber rod of the new energy vehicle to be detected has surface defects based on the comparison between the global matching coefficient and a predetermined threshold.
[0044] Specifically, the global matching coefficient calculation unit 151 is used to calculate the global matching coefficient between the multi-scale essential semantic representation feature vector of the surface state of the shock absorber rod and the multi-scale detection semantic feature vector of the surface state of the shock absorber rod. That is, in the technical solution of the present application, the semantic similarity between the multi-scale essential semantic representation feature vector of the surface state of the shock absorber rod and the multi-scale detection semantic feature vector of the surface state of the shock absorber rod is measured by introducing the global matching coefficient. Specifically, by multiplying the transposed vector of the multi-scale essential semantic representation feature vector of the surface state of the shock absorber rod by the multi-scale detection semantic feature vector of the surface state of the shock absorber rod, the consistency of the multi-scale essential semantic representation feature vector of the surface state of the shock absorber rod and the multi-scale detection semantic feature vector of the surface state of the shock absorber rod in the same direction can be measured, and the similarity between the two can be quantified. It should be understood that the length of a vector (or modulus) is the square root of the sum of the squares of all elements of the vector, which reflects the size of the vector. In other words, the length of a vector can represent the richness of the information contained in the feature vector. By dividing the semantic similarity between the reference image and the detection image by the product of the vector length of the multi-scale essential semantic representation feature vector of the surface state of the shock absorber rod and the vector length of the multi-scale detection semantic feature vector of the surface state of the shock absorber rod, the similarity can be standardized to eliminate the influence of vectors of different lengths. In this way, the global matching coefficient provides a quantitative index that can directly reflect the degree of closeness between the surface state of the shock absorber rod to be detected and the known standard or reference state, thereby intuitively indicating whether the surface state of the shock absorber rod to be detected meets the requirements, which is convenient for rapid and accurate shock absorber rod defect detection.
[0045] In a specific example of the present application, the global matching coefficient calculation unit 151 is used to: calculate the transposed vector of the multi-scale essential semantic representation feature vector of the shock absorber surface state multi-scale detection semantic feature vector of the shock absorber surface state to obtain the reference image-detection image semantic similarity; calculate the reference image-detection image semantic similarity divided by the vector length of the multi-scale essential semantic representation feature vector of the shock absorber surface state and the vector length of the multi-scale detection semantic feature vector of the shock absorber surface state to obtain the global matching coefficient. That is, the global matching coefficient between the multi-scale essential semantic representation feature vector of the shock absorber surface state and the multi-scale detection semantic feature vector of the shock absorber surface state is calculated using the following global semantic matching formula, wherein the global semantic matching formula is: ;in, is the multi-scale essential semantic representation feature vector of the surface state of the shock absorber rod, is the multi-scale detection semantic feature vector of the shock absorber rod surface state, is the global matching coefficient, represents the transpose of the eigenvector, Represents the length of the feature vector.
[0046] Specifically, the surface defect diagnosis unit 152 is used to determine whether the shock absorber rod of the new energy vehicle to be tested has surface defects based on the comparison between the global matching coefficient and the predetermined threshold. That is, by setting a suitable global matching coefficient threshold, automatic detection of surface defects of the shock absorber rod can be achieved. Specifically, when the global matching coefficient is lower than the predetermined threshold, it indicates that there is a significant difference between the surface state of the shock absorber rod to be tested and the surface state of the qualified shock absorber rod, then it is considered that the shock absorber rod to be tested has surface defects and needs further processing and analysis. In this way, automatic identification and judgment of surface defects of the shock absorber rod can be achieved, so as to improve the detection efficiency and accuracy of the shock absorber rod and reduce the need for manual intervention.
[0047] It should be understood that before using the above neural network model, it needs to be trained. That is, in the machine vision-based new energy vehicle shock absorber rod detection system of the present application, it also includes a training module for training the surface state feature extractor based on the DenseNet model, the multi-scale semantic feature extractor and the feature screening essentialization network based on the gating mechanism.
[0048] Figure 6 FIG. 1 is a block diagram of a training module in a new energy vehicle shock absorber rod detection system based on machine vision according to an embodiment of the present application. Figure 6As shown, the training module 200 includes: a training data acquisition unit 210, which is used to acquire training data, wherein the training data includes a surface state image of a shock absorber rod of a new energy vehicle to be tested collected by a camera, a set of reference images of the surface state of the shock absorber rod of the training vehicle to be tested and marked as qualified extracted from a background database, and a true value of a global matching coefficient; a training reference image surface feature extraction unit 220, which is used to extract the surface state feature of the shock absorber rod of the training vehicle to be tested and marked as qualified by the surface state feature extraction unit based on the DenseNet model. The training reference image multi-scale semantic analysis unit 230 is used to pass each of the training shock absorber rod surface state reference feature maps in the set of training shock absorber rod surface state reference feature maps through the multi-scale semantic feature extractor to obtain a set of training shock absorber rod surface state multi-scale semantic feature vectors; the training reference image feature screening unit 240 is used to pass the set of training shock absorber rod surface state multi-scale semantic feature vectors through the feature screening essentialization network based on the gating mechanism to obtain a training shock absorber rod surface state multi-scale essential semantic representation feature vector; the training detection image fine The detail enhancement unit 250 is used to perform contrast normalization and generalized Gaussian fitting on the surface state image of the training new energy vehicle shock absorber rod to be detected to obtain a training detail enhanced surface state image; the training detection image multi-scale semantic analysis unit 260 is used to pass the training detail enhanced surface state image through the surface state feature extractor based on the DenseNet model and the multi-scale semantic feature extractor to obtain a training shock absorber rod surface state multi-scale detection semantic feature vector; the global matching coefficient loss calculation unit 270 is used to calculate the training shock absorber rod surface state multi-scale essential semantic representation feature vector and the training shock absorber rod surface state multi-scale essential semantic representation feature vector. A training global matching coefficient between multi-scale detection semantic feature vectors of face states, and calculating the difference between the training global matching coefficient and the true value of the global matching coefficient as the global matching coefficient loss function value; a distribution loss calculation unit 280, used to calculate the distribution loss function value based on the training global matching coefficient; a model training unit 290, used to train the surface state feature extractor based on the DenseNet model, the multi-scale semantic feature extractor and the feature screening essentialization network based on the gating mechanism with the weighted sum of the global matching coefficient loss function value and the distribution loss function value as the loss function value.
[0049] In an optimized embodiment of the present application, considering that the training shock absorber rod surface state multi-scale essential semantic representation feature vector and the training shock absorber rod surface state multi-scale detection semantic feature vector respectively express the full sample space domain essentialized multi-scale image semantic features of the set of training shock absorber rod surface state reference images and the single sample space domain multi-scale image semantic features of the training detail enhanced surface state image, in the process of mapping from the cross-sample space dimension to the image semantic matching domain based on the global matching coefficient, the difference in image semantic expression will cause noise outside the image semantic matching domain distribution, thereby reducing the calculation accuracy of the training global matching coefficient between the training shock absorber rod surface state multi-scale essential semantic representation feature vector and the training shock absorber rod surface state multi-scale detection semantic feature vector.
[0050] Therefore, during the model training process, at each iteration, for example, when gradient backpropagation is performed based on the difference between the true global matching coefficient between the training shock absorber rod surface state multi-scale essential semantic representation feature vector and the training shock absorber rod surface state multi-scale detection semantic feature vector and the training global matching coefficient, a distribution loss function based on the training global matching coefficient is further introduced. Specifically, the calculation process of the distribution loss function based on the training global matching coefficient is as follows: calculate each iteration by subtracting the training global matching coefficient to obtain the inverse value of the training global matching coefficient; determine the feature mean and feature variance of all feature values of the feature union of the training shock absorber rod surface state multi-scale essential semantic representation feature vector and the training shock absorber rod surface state multi-scale detection semantic feature vector; calculate the product of each feature value of the feature union of the training shock absorber rod surface state multi-scale essential semantic representation feature vector and the training shock absorber rod surface state multi-scale detection semantic feature vector in the form of class cross entropy and the training global matching coefficient, and then subtract the product of the inverse value of the training global matching coefficient and the feature mean to obtain the class cross entropy difference; The squares of the class cross entropy differences corresponding to each eigenvalue in the feature union of the eigenvectors are summed and divided by the size of the feature union of the training shock absorber rod surface state multi-scale essential semantic representation feature vector and the training shock absorber rod surface state multi-scale detection semantic feature vector to obtain a first loss term, wherein the size of the feature union of the training shock absorber rod surface state multi-scale essential semantic representation feature vector and the training shock absorber rod surface state multi-scale detection semantic feature vector is the sum of the lengths of the training shock absorber rod surface state multi-scale essential semantic representation feature vector and the training shock absorber rod surface state multi-scale detection semantic feature vector; the sum of the squares of the training global matching coefficient and the inverse of the training global matching coefficient is multiplied by the feature variance to obtain a second loss term; the second loss term is subtracted from the first loss term to obtain the distribution loss function value based on the training global matching coefficient.
[0051] Here, the distribution loss function based on the training global matching coefficient sets the approximate expected variance characteristics of the out-of-distribution noise of the feature union of the training shock absorber rod surface state multi-scale essential semantic representation feature vector and the training shock absorber rod surface state multi-scale detection semantic feature vector relative to the Bernoulli sampling instance of the feature set by using the matching probability distribution characteristics expressed by the training global matching coefficient and the inverse value of the training global matching coefficient based on the feature union of the training shock absorber rod surface state multi-scale essential semantic representation feature vector and the training shock absorber rod surface state multi-scale detection semantic feature vector in the image semantic matching domain.
[0052] Therefore, by using the distribution loss function based on the training global matching coefficient to measure the distribution feature noise based on the matching probability distribution characteristics expressed by the training global matching coefficient and the inverse value of the training global matching coefficient, the statistical noise distribution can be used to perform probability distribution mapping in the image domain matching domain based on each eigenvalue of the feature union of the training shock absorber rod surface state multi-scale essential semantic representation feature vector and the training shock absorber rod surface state multi-scale detection semantic feature vector. The possible sampling noise features can be degraded by using the probabilistic cross random selection characteristics of the inclusion cross entropy type composite difference calculation of the expected variance of the feature union of the training shock absorber rod surface state multi-scale essential semantic representation feature vector and the training shock absorber rod surface state multi-scale detection semantic feature vector, thereby realizing the distribution noise reduction of self-supervised learning in the process of updating the model parameters, so as to improve the calculation accuracy of the training global matching coefficient between the training shock absorber rod surface state multi-scale essential semantic representation feature vector and the training shock absorber rod surface state multi-scale detection semantic feature vector.
[0053] In summary, the new energy vehicle shock absorber rod detection system based on machine vision according to the embodiment of the present application is explained, which uses machine vision technology based on deep learning to detect surface defects of the shock absorber rod of the new energy vehicle, extracts multi-scale semantic features of the surface state image of the shock absorber rod, and intelligently determines whether the shock absorber rod of the new energy vehicle to be detected has surface defects based on the semantic feature difference between the surface state image of the shock absorber rod to be detected and the surface state reference image of the qualified shock absorber rod. In this way, the accuracy and efficiency of the shock absorber rod quality detection can be effectively improved, while reducing the false detection and missed detection rate caused by human factors.
[0054] Figure 7 FIG. 1 is a flow chart of a method for detecting a shock absorber rod of a new energy vehicle based on machine vision according to an embodiment of the present application. Figure 7 As shown, according to the embodiment of the present application, the new energy vehicle shock absorber rod detection method based on machine vision includes the following steps: S1, extracting a set of shock absorber rod surface state reference images marked as qualified from a background database; S2, performing multi-scale semantic feature extraction and essentialized feature screening on the set of shock absorber rod surface state reference images marked as qualified to obtain a shock absorber rod surface state multi-scale essential semantic representation feature vector; S3, acquiring a surface state image of the shock absorber rod to be detected by a camera; S4, performing multi-scale semantic feature extraction on the surface state image of the shock absorber rod to be detected to obtain a shock absorber rod surface state multi-scale detection semantic feature vector; S5, determining whether the shock absorber rod to be detected has surface defects based on the global matching feature between the shock absorber rod surface state multi-scale essential semantic representation feature vector and the shock absorber rod surface state multi-scale detection semantic feature vector.
[0055] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned new energy vehicle shock absorber rod detection method based on machine vision have been referred to above. Figures 1 to 6 The description of the new energy vehicle shock absorber rod detection system based on machine vision has been introduced in detail, and therefore, its repeated description will be omitted.
[0056] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.
[0057] In the above embodiments, the description of each embodiment has its own emphasis. For the parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical sub-unit, that is, it may be located in one place, or it may be distributed on multiple network sub-units. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment.
[0058] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference to a figure in a claim should not be considered as limiting the claim to which it relates.
[0059] In addition, it is obvious that the word "comprising" does not exclude other sub-units or steps, and the singular does not exclude the plural. Multiple sub-units stated in the system claim can also be implemented by one sub-unit through software or hardware.
[0060] Finally, it should be noted that the above description has been given for the purpose of illustration and description. In addition, the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.
Claims
1. A new energy vehicle shock absorber rod detection system based on machine vision, characterized in that: include: A shock absorber rod reference image retrieval module is used to extract a set of shock absorber rod surface state reference images marked as qualified from a background database; A reference image essential feature extraction module is used to perform multi-scale semantic feature extraction and essential feature screening on the set of reference images of the surface state of the shock absorber rod marked as qualified to obtain a multi-scale essential semantic representation feature vector of the surface state of the shock absorber rod; The image acquisition module of the shock absorber rod to be detected is used to obtain the surface state image of the shock absorber rod of the new energy vehicle to be detected collected by the camera; The semantic feature extraction module of the image to be detected is used to extract multi-scale semantic features of the surface state image of the shock absorber rod of the new energy vehicle to be detected to obtain a multi-scale detection semantic feature vector of the surface state of the shock absorber rod; A global matching metric module, used to determine whether the shock absorber rod of the new energy vehicle to be inspected has surface defects based on the global matching features between the multi-scale essential semantic representation feature vector of the surface state of the shock absorber rod and the multi-scale detection semantic feature vector of the surface state of the shock absorber rod; Wherein, the reference image essential feature extraction module includes: A surface state feature extraction unit, used for respectively passing each of the shock absorber rod surface state reference images marked as qualified in the set of shock absorber rod surface state reference images marked as qualified through a surface state feature extractor based on a DenseNet model to obtain a set of shock absorber rod surface state reference feature maps; A multi-scale semantic feature extraction unit, used for respectively passing each of the shock absorber rod surface state reference feature maps in the set of the shock absorber rod surface state reference feature maps through a multi-scale semantic feature extractor to obtain a set of shock absorber rod surface state multi-scale semantic feature vectors; An essentialized feature screening unit is used to pass the set of multi-scale semantic feature vectors of the surface state of the shock absorber rod through a feature screening essentialized network based on a gating mechanism to obtain a multi-scale essential semantic representation feature vector of the surface state of the shock absorber rod; Wherein, the essential feature screening unit includes: The semantic difference measurement subunit is used to calculate the semantic difference coefficient of each shock absorber rod surface state multi-scale semantic feature vector in the set of shock absorber rod surface state multi-scale semantic feature vectors relative to other shock absorber rod surface state multi-scale semantic feature vectors in accordance with the following semantic difference measurement formula to obtain a set of semantic difference coefficients, wherein the semantic difference measurement formula is: ;in, is the first in the set of multi-scale semantic feature vectors of the surface state of the shock absorber rod. A multi-scale semantic feature vector of the shock absorber surface state, It is A multi-scale semantic feature vector of the shock absorber surface state, Indicates the feature vector The eigenvalues of the positions, is the length of the multi-scale semantic feature vector of the surface state of the shock absorber rod, Represents the logarithmic function operation with base 2, The value of is the number of multi-scale semantic feature vectors of the surface state of the shock absorber rod minus one, For the said The semantic difference coefficient of the multi-scale semantic feature vector of the shock absorber surface state; A normalization subunit, configured to perform normalization processing on the set of semantic difference coefficients to obtain a set of normalized semantic difference coefficients; A masking subunit, configured to perform masking processing on the set of normalized semantic difference coefficients using a gating unit to obtain a set of masked semantic difference coefficients; The semantic feature screening subunit is used to calculate the weighted sum of the set of multi-scale semantic feature vectors of the surface state of the shock absorber rod using the set of masked semantic difference coefficients as weights to obtain the multi-scale essential semantic representation feature vector of the surface state of the shock absorber rod.
2. The machine vision-based new energy vehicle shock absorber rod detection system according to claim 1 is characterized in that: The multi-scale semantic feature extraction unit comprises: A multi-scale semantic encoding subunit, used for respectively extracting global semantic features and local semantic features of the reference feature map of the surface state of the shock absorber rod to obtain a global semantic feature vector of the surface state of the shock absorber rod and a local semantic feature vector of the surface state of the shock absorber rod; The multi-scale semantic fusion subunit is used to fuse the global semantic feature vector of the surface state of the shock absorber rod and the local semantic feature vector of the surface state of the shock absorber rod to obtain the multi-scale semantic feature vector of the surface state of the shock absorber rod.
3. The machine vision-based new energy vehicle shock absorber rod detection system according to claim 2 is characterized in that: The multi-scale semantic encoding subunit is used to: Upsampling the shock absorber rod surface state reference feature map to obtain an upsampled shock absorber rod surface state reference feature map; Performing global mean pooling processing on the up-sampled shock absorber rod surface state reference feature map to obtain an up-sampled shock absorber rod surface state reference semantic channel feature vector; Performing point convolution encoding on the upsampled shock absorber rod surface state reference semantic channel feature vector to obtain the shock absorber rod surface state global semantic feature vector; Performing point convolution encoding on the up-sampled shock absorber rod surface state reference feature map to obtain a channel modulated up-sampled shock absorber rod surface state reference feature map; Two-dimensional convolution coding is performed on the channel-modulated up-sampled shock absorber rod surface state reference feature map to obtain the shock absorber rod surface state local semantic feature vector.
4. The machine vision-based new energy vehicle shock absorber rod detection system according to claim 3 is characterized in that: The semantic feature extraction module of the image to be detected includes: A detail enhancement unit, used for performing contrast normalization and generalized Gaussian fitting on the surface state image of the shock absorber rod of the new energy vehicle to be detected to obtain a detail enhanced surface state image; The surface state multi-scale encoding unit is used to pass the detail-enhanced surface state image through the surface state feature extractor based on the DenseNet model and the multi-scale semantic feature extractor to obtain the multi-scale detection semantic feature vector of the shock absorber rod surface state.
5. The machine vision-based new energy vehicle shock absorber rod detection system according to claim 4 is characterized in that: The global matching metric module comprises: A global matching coefficient calculation unit, used to calculate a global matching coefficient between the multi-scale essential semantic representation feature vector of the surface state of the shock absorber rod and the multi-scale detection semantic feature vector of the surface state of the shock absorber rod; The surface defect diagnosis unit is used to determine whether the shock absorber rod of the new energy vehicle to be inspected has surface defects based on the comparison between the global matching coefficient and a predetermined threshold.
6. The machine vision-based new energy vehicle shock absorber rod detection system according to claim 5 is characterized in that: The global matching coefficient calculation unit is used to: Calculate the transposed vector of the multi-scale essential semantic representation feature vector of the surface state of the shock absorber rod and multiply it by the multi-scale detection semantic feature vector of the surface state of the shock absorber rod to obtain the semantic similarity between the reference image and the detection image; The global matching coefficient is obtained by calculating the semantic similarity between the reference image and the detection image and dividing the product of the vector length of the multi-scale essential semantic representation feature vector of the shock absorber surface state and the vector length of the multi-scale detection semantic feature vector of the shock absorber surface state.
7. The machine vision-based new energy vehicle shock absorber rod detection system according to claim 6 is characterized in that: It also includes a training module for training the surface state feature extractor based on the DenseNet model, the multi-scale semantic feature extractor, and the feature screening essentialization network based on the gating mechanism; Wherein, the training module includes: A training data acquisition unit, used to acquire training data, wherein the training data includes a surface state image of a shock absorber rod of a new energy vehicle to be tested collected by a camera, a set of reference images of the surface state of the shock absorber rod of the training vehicle to be tested that are marked as qualified and extracted from a background database, and a true value of a global matching coefficient; A training reference image surface feature extraction unit, used for passing each of the training shock absorber rod surface state reference images marked as qualified in the set of training shock absorber rod surface state reference images marked as qualified through the surface state feature extractor based on the DenseNet model to obtain a set of training shock absorber rod surface state reference feature maps; A training reference image multi-scale semantic analysis unit, used for respectively passing each training shock absorber rod surface state reference feature map in the set of training shock absorber rod surface state reference feature maps through the multi-scale semantic feature extractor to obtain a set of training shock absorber rod surface state multi-scale semantic feature vectors; A training reference image feature screening unit, used for passing the set of multi-scale semantic feature vectors of the surface state of the training shock absorber rod through the feature screening essentialization network based on the gating mechanism to obtain a multi-scale essential semantic representation feature vector of the surface state of the training shock absorber rod; A training detection image detail enhancement unit is used to perform contrast normalization and generalized Gaussian fitting on the surface state image of the shock absorber rod of the new energy vehicle to be detected in the training to obtain a training detail enhanced surface state image; A training detection image multi-scale semantic analysis unit, used for passing the training detail enhanced surface state image through the surface state feature extractor based on the DenseNet model and the multi-scale semantic feature extractor to obtain a training shock absorber rod surface state multi-scale detection semantic feature vector; A global matching coefficient loss calculation unit is used to calculate the training global matching coefficient between the training shock absorber rod surface state multi-scale essential semantic representation feature vector and the training shock absorber rod surface state multi-scale detection semantic feature vector, and calculate the difference between the training global matching coefficient and the true value of the global matching coefficient as the global matching coefficient loss function value; A distribution loss calculation unit, used to calculate a distribution loss function value based on the training global matching coefficient; A model training unit is used to train the surface state feature extractor based on the DenseNet model, the multi-scale semantic feature extractor and the feature screening essentialization network based on the gating mechanism by taking the weighted sum of the global matching coefficient loss function value and the distribution loss function value as the loss function value.
8. A new energy vehicle shock absorber rod detection method based on machine vision, using the new energy vehicle shock absorber rod detection system based on machine vision according to claim 1, characterized in that: include: Extracting a set of reference images of the surface conditions of the shock absorber rods marked as qualified from the background database; Performing multi-scale semantic feature extraction and essential feature screening on the set of reference images of the surface state of the shock absorber rod marked as qualified to obtain a multi-scale essential semantic representation feature vector of the surface state of the shock absorber rod; Acquire a surface state image of the shock absorber rod of the new energy vehicle to be inspected, which is captured by a camera; Performing multi-scale semantic feature extraction on the surface state image of the shock absorber rod of the new energy vehicle to be detected to obtain a multi-scale detection semantic feature vector of the surface state of the shock absorber rod; Based on the global matching features between the multi-scale essential semantic representation feature vector of the shock absorber rod surface state and the multi-scale detection semantic feature vector of the shock absorber rod surface state, it is determined whether the shock absorber rod of the new energy vehicle to be inspected has surface defects.
