Visual inspection system and method for new energy vehicle manufacturing

Abstract

The present application relates to the field of visual inspection technology, and specifically discloses a visual inspection system and method for new energy vehicle manufacturing, which uses a set of reference images of lithium battery surfaces marked as having no appearance defects as reference data, and obtains the reference aggregate representation features of the lithium battery surface state through feature extraction, feature filtering and cluster analysis, and then obtains the semantic difference feature expression between the reference aggregate representation features of the lithium battery surface state and the representation features of the lithium battery surface state to be detected through a semantic difference metric network, and finally determines whether the lithium battery to be detected has surface defects based on the semantic difference feature expression. In this way, the excessive focus on the types of defects in traditional methods can be effectively avoided, thereby reducing the complexity of the model and improving the accuracy and efficiency of detection.

Description

Technical Field

[0001] The present application relates to the field of visual inspection technology, and more specifically, to a visual inspection system and method for new energy vehicle manufacturing. Background Art

[0002] In the field of new energy vehicle manufacturing, lithium batteries are core components, and their quality directly affects the performance and safety of the entire vehicle. Traditional lithium battery surface defect detection methods mainly rely on manual visual inspection, which is not only inefficient, but also easily affected by human factors, resulting in unstable detection results. With the development of computer vision technology, the use of machine vision for lithium battery surface defect detection has become a trend.

[0003] China's public patent CN117798087A provides a lithium battery visual sorting system based on appearance defect detection, which includes an image acquisition module, an image processing module, a defect detection module, a defect classification module and a sorting module, wherein the image acquisition module is responsible for acquiring the surface image of the lithium battery to be detected, the image processing module preprocesses and extracts features of the image, the defect detection module is used to analyze features and detect defects in appearance, the defect classification module classifies the lithium battery according to the detection results, and the sorting module sorts the lithium battery according to the classification results.

[0004] Although the technical solution provided by patent CN 117798087A can detect various types of appearance defects and ensure comprehensive quality control, many technical problems are found in the specific implementation of the technical solution.

[0005] First, although the system can detect many types of appearance defects, in practical applications, how to further improve the accuracy of defect detection is still a challenge. In particular, when processing images with complex backgrounds, how to effectively suppress noise and reduce false positives and negatives requires further research.

[0006] Secondly, the differences between the feature representations of various defect features at the image level are not obvious, which leads to the phenomenon of incorrect specification of appearance defect types when classifying multiple types of defects. In other words, the inter-class distance between various types of appearance defect image features decreases, which causes the class boundaries of defect types to collapse when determining the defect type based on the appearance defect image features, resulting in low accuracy of defect classification.

[0007] Therefore, an optimized visual inspection solution for new energy vehicle manufacturing is expected. Summary of the invention

[0008] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a visual inspection system and method for new energy vehicle manufacturing, which uses a set of reference images of lithium battery surfaces marked as having no appearance defects as reference data, and obtains the reference aggregate representation features of the surface state of the lithium battery through feature extraction, feature filtering and cluster analysis, and then obtains the semantic difference feature expression between the reference aggregate representation features of the surface state of the lithium battery and the surface state representation features of the lithium battery to be detected through a semantic difference metric network, and finally determines whether the lithium battery to be detected has surface defects based on the semantic difference feature expression. In this way, the excessive focus on the type of defects in traditional methods can be effectively avoided, thereby reducing the complexity of the model and improving the accuracy and efficiency of detection.

[0009] Accordingly, according to one aspect of the present application, a visual inspection system for new energy vehicle manufacturing is provided, which includes: a data acquisition module, which is used to acquire a surface image of a lithium battery to be inspected collected by a camera, and extract a set of lithium battery surface reference images marked as having no appearance defects from a database; a reference image feature extraction module, which is used to extract lithium battery surface features from each lithium battery surface reference image marked as having no appearance defects in the set of lithium battery surface reference images marked as having no appearance defects, so as to obtain a set of lithium battery surface state reference feature vectors; a reference feature selection optimization aggregation module, which is used to extract lithium battery surface features from each lithium battery surface reference image marked as having no appearance defects in the set of lithium battery surface reference images marked as having no appearance defects, so as to obtain a set of lithium battery surface state reference feature vectors; A set of surface state reference feature vectors is subjected to feature selection optimization aggregation to obtain a lithium battery surface state reference aggregation representation vector; a detection image feature extraction module is used to extract the lithium battery surface features of the surface image of the lithium battery to be detected to obtain the surface state feature vector of the lithium battery to be detected; a semantic difference analysis module is used to determine the defect detection result of the lithium battery to be detected based on the semantic difference between the lithium battery surface state reference aggregation representation vector and the surface state feature vector of the lithium battery to be detected; a sorting module is used to place the lithium battery to be detected in a defect group by a sorting robot in response to the defect detection result being a defect.

[0010] In the above-mentioned visual inspection system for new energy vehicle manufacturing, the reference image feature extraction module is used to: input each of the lithium battery surface reference images marked as having no appearance defects into a double-branch boundary information loss compensation lithium battery surface feature extraction network to obtain a set of reference feature vectors of the lithium battery surface state, wherein the double-branch boundary information loss compensation lithium battery surface feature extraction network includes a backbone network and a boundary feature extraction branch.

[0011] In the above-mentioned visual inspection system for new energy vehicle manufacturing, the reference feature selection optimization aggregation module includes: a feature selection unit, which is used to input the set of lithium battery surface state reference feature vectors into a feature selection optimization module based on feature intensity distribution to obtain a set of selected lithium battery surface state reference feature vectors; a feature aggregation unit, which is used to perform feature aggregation on the set of selected lithium battery surface state reference feature vectors to obtain the lithium battery surface state reference aggregation representation vector.

[0012] In the above-mentioned visual inspection system for new energy vehicle manufacturing, the feature selection unit includes: an energy distribution spectrum calculation subunit, which is used to calculate the characteristic energy distribution spectrum vector of each lithium battery surface state reference feature vector in the set of lithium battery surface state reference feature vectors to obtain a set of lithium battery surface state reference feature energy distribution spectrum vectors; an energy spectrum distribution center calculation subunit, which is used to calculate the positional mean vector of the set of lithium battery surface state reference feature energy distribution spectrum vectors as the energy spectrum distribution center vector; an energy distribution spectrum span calculation subunit, which is used to calculate the Mahalanobis distance between the energy spectrum distribution center vector and each lithium battery surface state reference feature energy distribution spectrum vector in the set of lithium battery surface state reference feature energy distribution spectrum vectors as the energy distribution spectrum span factor to obtain a set of energy distribution spectrum span factors; a feature screening subunit, which is used to select the lithium battery surface state reference feature vectors corresponding to the energy distribution spectrum span factors less than the preset threshold value based on the comparison between each energy distribution spectrum span factor in the set of energy distribution spectrum span factors and the preset threshold value to form the selected set of lithium battery surface state reference feature vectors.

[0013] In the above-mentioned visual inspection system for new energy vehicle manufacturing, the energy distribution spectrum calculation subunit includes: a feature distribution synergistic representation secondary subunit, which is used to calculate the feature distribution energy synergistic representation vector between the i-th lithium battery surface state reference feature vector and each other lithium battery surface state reference feature vector in the set of lithium battery surface state reference feature vectors by taking the maximum eigenvalue element by element to obtain a set of feature distribution energy synergistic representation vectors; a feature distribution energy synergistic factor calculation secondary subunit, which is used to calculate the feature distribution energy synergistic factor of each feature distribution energy synergistic representation vector in the set of feature distribution energy synergistic representation vectors to obtain a feature energy distribution spectrum vector of the i-th lithium battery surface state reference feature vector composed of multiple feature distribution energy synergistic factors.

[0014] In the above-mentioned visual inspection system for the manufacture of new energy vehicles, the secondary subunit for calculating the feature distribution energy synergy factor is used to: calculate the square of the difference between the maximum eigenvalue of the feature distribution energy synergy representation vector and its feature mean, the feature variance of the feature distribution energy synergy representation vector and the weighted sum of the bias terms, and then add the variance of the feature distribution energy synergy representation vector and the bias term and divide the sum by the weighted sum to obtain the feature distribution energy synergy factor.

[0015] In the above-mentioned visual inspection system for new energy vehicle manufacturing, the inspection image feature extraction module is used to: input the surface image of the lithium battery to be inspected into the double-branch boundary information loss compensation lithium battery surface feature extraction network including a backbone network and a boundary feature extraction branch to obtain the surface state feature vector of the lithium battery to be inspected.

[0016] In the above-mentioned visual inspection system for new energy vehicle manufacturing, the semantic difference analysis module includes: a semantic difference measurement unit, which is used to input the lithium battery surface state reference aggregation representation vector and the surface state feature vector of the lithium battery to be detected into a semantic difference measurement network to obtain a lithium battery detection-reference semantic difference representation vector; a defect detection result generation unit, which is used to input the lithium battery detection-reference semantic difference representation vector into a classifier-based defect detector to obtain the defect detection result, and the defect detection result is used to indicate whether there is a defect.

[0017] In the above-mentioned visual inspection system for new energy vehicle manufacturing, the semantic difference measurement unit is used to: calculate the position point addition, position point subtraction and position point multiplication between the lithium battery surface state reference aggregate representation vector and the surface state feature vector of the lithium battery to be detected to obtain the first semantic interaction result, the second semantic interaction result and the third semantic interaction result; after cascading the first semantic interaction result, the second semantic interaction result and the third semantic interaction result into a lithium battery detection-reference multi-scale semantic interaction feature vector, perform one-dimensional convolution processing and maximum pooling processing on it to obtain the lithium battery detection-reference semantic difference representation vector.

[0018] According to another aspect of the present application, a visual inspection method for new energy vehicle manufacturing is provided, which includes: acquiring a surface image of a lithium battery to be inspected captured by a camera, and extracting a set of lithium battery surface reference images marked as having no appearance defects from a database; performing lithium battery surface feature extraction on each lithium battery surface reference image marked as having no appearance defects in the set of lithium battery surface reference images marked as having no appearance defects to obtain a set of lithium battery surface state reference feature vectors; performing feature selection optimization aggregation on the set of lithium battery surface state reference feature vectors to obtain a lithium battery surface state reference aggregation representation vector; extracting lithium battery surface features of the surface image of the lithium battery to be inspected to obtain a lithium battery surface state feature vector to be inspected; determining a defect detection result of the lithium battery to be inspected based on the semantic difference between the lithium battery surface state reference aggregation representation vector and the lithium battery surface state feature vector to be inspected; in response to the defect detection result being a defect, placing the lithium battery to be inspected in a defect group by a sorting robot.

[0019] Compared with the prior art, the visual inspection system and method for new energy vehicle manufacturing provided by the present application uses a set of reference images of lithium battery surfaces marked as having no appearance defects as reference data, and obtains the reference aggregate representation features of the lithium battery surface state through feature extraction, feature filtering and cluster analysis, and then obtains the semantic difference feature expression between the reference aggregate representation features of the lithium battery surface state and the representation features of the lithium battery surface state to be detected through a semantic difference metric network, and finally determines whether the lithium battery to be detected has surface defects based on the semantic difference feature expression. In this way, the excessive focus on the types of defects in traditional methods can be effectively avoided, thereby reducing the complexity of the model and improving the accuracy and efficiency of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0021] Figure 1 It is a block diagram of a visual inspection system for new energy vehicle manufacturing according to an embodiment of the present application.

[0022] Figure 2 Schematic diagram of data flow of a visual inspection system for new energy vehicle manufacturing according to an embodiment of the present application.

[0023] Figure 3This is an application scenario diagram of a visual inspection system for new energy vehicle manufacturing according to an embodiment of the present application.

[0024] Figure 4 It is a block diagram of a reference feature selection optimization aggregation module in a visual inspection system for new energy vehicle manufacturing according to an embodiment of the present application.

[0025] Figure 5 It is a block diagram of a feature selection unit in a visual inspection system for new energy vehicle manufacturing according to an embodiment of the present application.

[0026] Figure 6 It is a block diagram of a semantic difference analysis module in a visual inspection system for new energy vehicle manufacturing according to an embodiment of the present application.

[0027] Figure 7 The present invention is a flowchart of a visual inspection method for manufacturing new energy vehicles according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] Below, the embodiments of the present application will be described in more detail in conjunction with the accompanying drawings, and the above and other purposes, features and advantages of the present application will become more apparent. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein.

[0029] The technical idea of ​​the prior art described in the above background technology is: extracting various types of defect features and performing multi-classification detection of defect types based on the various defect type features. Following this technical idea, the main improvement directions are: 1. Improving the accuracy of representing various types of defect features; 2. Improving the accuracy of multi-classification by optimizing the training strategy to reduce the distance between classes and increase the distance between classes.

[0030] However, in the actual industry, there are many types of defects, and it is difficult to fully address them through rules or feature encoding methods; even if the comprehensiveness and coverage of feature extraction are achieved through specific strategies, this will lead to technical problems such as overfitting of the model when classifying defects. In other words, if the technical ideas of existing technologies are developed, there will be obvious technical limits.

[0031] It is worth mentioning that in the technical solution of the present application, in the defect detection of lithium batteries, it is not necessary to confirm the specific type of defects, but only to confirm whether the lithium battery has defects. On the other hand, for lithium batteries, although the types of defects are diverse, the surface state of lithium batteries that meet the specifications is generally consistent. Based on this, the present application overcomes technical bias and proposes a new visual inspection solution for lithium batteries.

[0032] Specifically, the technical concept of the novel visual inspection scheme for lithium batteries described in this application is: using a set of reference images of lithium battery surfaces marked as having no appearance defects as reference data, and obtaining the reference aggregate representation features of the lithium battery surface state through feature extraction, feature filtering and cluster analysis, and then obtaining the semantic difference feature expression between the reference aggregate representation features of the lithium battery surface state and the representation features of the lithium battery surface state to be detected through a semantic difference metric network, and finally determining whether the lithium battery to be detected has surface defects based on the semantic difference feature expression. In this way, the excessive focus on the types of defects in traditional methods can be effectively avoided, thereby reducing the complexity of the model and improving the accuracy and efficiency of detection.

[0033] Figure 1 It is a block diagram of a visual inspection system for new energy vehicle manufacturing according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of a visual inspection system for new energy vehicle manufacturing according to an embodiment of the present application. Figure 3 FIG. 1 is an application scenario diagram of a visual inspection system for new energy vehicle manufacturing according to an embodiment of the present application. Figure 1 , Figure 2 and Figure 3 As shown, according to an embodiment of the present application, a visual inspection system 100 for new energy vehicle manufacturing includes: a data acquisition module 110, which is used to acquire a surface image of a lithium battery to be inspected collected by a camera, and extract a set of lithium battery surface reference images marked as having no appearance defects from a database; a reference image feature extraction module 120, which is used to extract lithium battery surface features from each lithium battery surface reference image marked as having no appearance defects in the set of lithium battery surface reference images marked as having no appearance defects, so as to obtain a set of lithium battery surface state reference feature vectors; a reference feature selection optimization aggregation module 130, which is used to extract lithium battery surface features from each lithium battery surface reference image marked as having no appearance defects in the set of lithium battery surface reference images marked as having no appearance defects, so as to obtain a set of lithium battery surface state reference feature vectors; A set of reference feature vectors is used for feature selection optimization aggregation to obtain a reference aggregation representation vector of the surface state of the lithium battery; a detection image feature extraction module 140 is used to extract the lithium battery surface features of the surface image of the lithium battery to be detected to obtain the surface state feature vector of the lithium battery to be detected; a semantic difference analysis module 150 is used to determine the defect detection result of the lithium battery to be detected based on the semantic difference between the lithium battery surface state reference aggregation representation vector and the surface state feature vector of the lithium battery to be detected; a sorting module 160 is used to place the lithium battery to be detected in a defect group by a sorting robot in response to the defect detection result being a defect.

[0034] In the above-mentioned visual inspection system 100 for manufacturing new energy vehicles, the data acquisition module 110 is used to acquire the surface image of the lithium battery to be inspected captured by the camera, and extract a set of reference images of the lithium battery surface marked as having no appearance defects from the database. It should be understood that the surface image of the lithium battery to be inspected acquired from the camera is real-time inspection data, while the set of reference images of the lithium battery surface without appearance defects extracted from the database is a product image that has been verified to be of qualified quality, representing an ideal lithium battery surface state. By using this as a reference benchmark for comparison with the surface image of the lithium battery to be inspected, the inconsistencies on the surface of the lithium battery to be inspected can be quickly and accurately identified, thereby achieving efficient detection of defects in the lithium battery to be inspected.

[0035] In the above-mentioned visual inspection system 100 for manufacturing new energy vehicles, the reference image feature extraction module 120 is used to extract lithium battery surface features from each lithium battery surface reference image marked as having no appearance defects in the set of lithium battery surface reference images marked as having no appearance defects to obtain a set of reference feature vectors of the lithium battery surface state. In a specific example of the present application, the processing method of extracting lithium battery surface features from each lithium battery surface reference image marked as having no appearance defects in the set of lithium battery surface reference images marked as having no appearance defects is to input each of the lithium battery surface reference images marked as having no appearance defects into a double-branch boundary information loss compensation lithium battery surface feature extraction network to obtain a set of reference feature vectors of the lithium battery surface state, wherein the double-branch boundary information loss compensation lithium battery surface feature extraction network includes a backbone network and a boundary feature extraction branch. It should be understood that, considering that in a deep network architecture, as the number of network layers increases, although more abundant image feature information can be extracted, it will also lead to the loss of boundary information. Therefore, in the technical solution of the present application, a dual-branch network structure including a backbone network and a boundary feature extraction branch is particularly used to extract the surface features of the lithium battery, so as to ensure that while extracting deep features, the shallow detail information of the image can be retained, thereby improving the accuracy of feature representation. Specifically, the backbone network adopts a large receptive field and a deep network architecture to extract global information such as color distribution and overall shape on the surface of the lithium battery; while the boundary feature extraction branch adopts a small receptive field and a shallow network architecture to extract edge detail information such as small scratches and cracks on the surface of the lithium battery. By fusing the feature outputs of the backbone network and the boundary feature extraction branch, the extraction of global information and local detail information can be effectively balanced, so that the obtained reference features of the surface state of the lithium battery are more comprehensive and accurate.

[0036] In the visual inspection system 100 for new energy vehicle manufacturing, the reference feature selection optimization aggregation module 130 is used to perform feature selection optimization aggregation on the set of reference feature vectors of the lithium battery surface state to obtain a reference aggregation representation vector of the lithium battery surface state. Accordingly, Figure 4 FIG. 1 is a block diagram of a reference feature selection optimization aggregation module in a visual inspection system for new energy vehicle manufacturing according to an embodiment of the present application. Figure 4 As shown, the reference feature selection optimization aggregation module 130 includes: a feature selection unit 131, which is used to input the set of reference feature vectors of the lithium battery surface state into a feature selection optimization module based on feature intensity distribution to obtain a set of selected reference feature vectors of the lithium battery surface state; a feature aggregation unit 132, which is used to perform feature aggregation on the set of selected reference feature vectors of the lithium battery surface state to obtain the reference aggregation representation vector of the lithium battery surface state.

[0037] Specifically, the feature selection unit 131 is used to input the set of reference feature vectors of the lithium battery surface state into a feature selection optimization module based on feature intensity distribution to obtain a set of selected reference feature vectors of the lithium battery surface state. It should be understood that, considering that the number of images contained in the set of reference images of the lithium battery surface may be very large, directly using it for subsequent image feature comparison and analysis will not only lead to high computational complexity, but also easily fall into overfitting. Therefore, the present application further introduces a feature selection optimization module based on feature intensity distribution to process the set of reference feature vectors of the lithium battery surface state, and screen out representative and discriminative feature representations from a large number of reference features to reduce the computational burden of subsequent processing and improve the accuracy of detection.

[0038] Specifically, the module first calculates the characteristic energy distribution spectrum vector of each lithium battery surface state reference feature vector in the set to reveal the distribution of each lithium battery surface state reference feature vector at different energy levels. Then, the positional mean of the characteristic energy distribution spectrum vector of each lithium battery surface state reference feature vector is further calculated to reveal the overall energy distribution trend of the set and obtain the center vector of the energy spectrum distribution. Then, using this as reference information, by calculating the difference between the center vector of the energy spectrum distribution and each characteristic energy distribution spectrum vector, the closeness of each lithium battery surface state reference feature vector to the overall distribution is evaluated, and the corresponding energy distribution spectrum span factor is generated. Then, based on the energy distribution spectrum span factor, the original set of the lithium battery surface state reference feature vectors is screened to remove the feature information that deviates from the overall distribution and retain the useful features that are close to the overall distribution, thereby obtaining a set of selected lithium battery surface state reference feature vectors.

[0039] Figure 5FIG. 1 is a block diagram of a feature selection unit in a visual inspection system for new energy vehicle manufacturing according to an embodiment of the present application. Figure 5 As shown, the feature selection unit 131 includes: an energy distribution spectrum calculation subunit 1311, which is used to calculate the characteristic energy distribution spectrum vector of each lithium battery surface state reference feature vector in the set of lithium battery surface state reference feature vectors to obtain a set of lithium battery surface state reference feature energy distribution spectrum vectors; an energy spectrum distribution center calculation subunit 1312, which is used to calculate the positional mean vector of the set of lithium battery surface state reference feature energy distribution spectrum vectors as the energy spectrum distribution center vector; an energy distribution spectrum span calculation subunit 1313, which is used to calculate the Mahalanobis distance between the energy spectrum distribution center vector and each lithium battery surface state reference feature energy distribution spectrum vector in the set of lithium battery surface state reference feature energy distribution spectrum vectors as the energy distribution spectrum span factor to obtain a set of energy distribution spectrum span factors; a feature screening subunit 1314, which is used to select the lithium battery surface state reference feature vectors corresponding to the energy distribution spectrum span factor less than the preset threshold value based on the comparison between each energy distribution spectrum span factor in the set of energy distribution spectrum span factors and the preset threshold value to form the selected set of lithium battery surface state reference feature vectors.

[0040] Specifically, the energy distribution spectrum calculation subunit 1311 includes: a characteristic distribution synergistic representation secondary subunit, which is used to calculate the characteristic distribution energy synergistic representation vector between the i-th lithium battery surface state reference feature vector and each other lithium battery surface state reference feature vector in the set of lithium battery surface state reference feature vectors by taking the maximum eigenvalue element by element to obtain a set of characteristic distribution energy synergistic representation vectors; a characteristic distribution energy synergistic factor calculation secondary subunit, which is used to calculate the characteristic distribution energy synergistic factor of each characteristic distribution energy synergistic representation vector in the set of characteristic distribution energy synergistic representation vectors to obtain a characteristic energy distribution spectrum vector of the i-th lithium battery surface state reference feature vector composed of multiple characteristic distribution energy synergistic factors.

[0041] More specifically, the secondary subunit for calculating the characteristic distribution energy synergy factor is used to: calculate the square of the difference between the maximum eigenvalue of the characteristic distribution energy synergy representation vector and its characteristic mean, the characteristic variance of the characteristic distribution energy synergy representation vector, and the weighted sum of the bias terms, and then add the variance of the characteristic distribution energy synergy representation vector and the bias term and divide the sum by the weighted sum to obtain the characteristic distribution energy synergy factor.

[0042] That is, the feature selection unit 131 is used to process the set of reference feature vectors of the lithium battery surface state using the following feature selection optimization formula to obtain the selected set of reference feature vectors of the lithium battery surface state, wherein the feature selection optimization formula is: ;in, and Respectively represent the first and A lithium battery surface state reference feature vector, It means taking the maximum eigenvalue of two eigenvectors element by element. Indicates the A lithium battery surface state reference feature vector and the The characteristic distribution energy synergistic representation vector between the reference characteristic vectors of the lithium battery surface state, , and represent the characteristic mean, characteristic variance and maximum eigenvalue of the characteristic distribution energy collaborative representation vector respectively, is the bias term, , , and Respectively represent the first The reference feature vector of the lithium battery surface state is compared with the first, second, and The characteristic distribution energy synergy factor between the reference characteristic vectors of the lithium battery surface state, It represents the first A reference characteristic energy distribution spectrum vector of the lithium battery surface state, Represents the set of reference characteristic energy distribution spectrum vectors of the lithium battery surface state, The value of is the number of reference feature vectors of the surface state of the lithium battery, represents the center vector of energy spectrum distribution, Indicates The covariance matrix between the reference characteristic energy distribution spectrum vector of the lithium battery surface state and the center vector of the energy spectrum distribution, represents the distance metric function, Indicates The energy distribution spectrum span factor, Indicates the preset threshold value, Represents a set of reference feature vectors of the surface state of the lithium battery after the selection.

[0043] Specifically, the feature aggregation unit 132 is used to perform feature aggregation on the set of selected lithium battery surface state reference feature vectors to obtain the reference aggregate representation vector of the lithium battery surface state. It should be understood that even in the reference image of the lithium battery surface without appearance defects, some surface state differences that are not related to defects may be presented due to factors such as shooting position deviation and changes in lighting conditions. In order to ensure the accuracy and robustness of the detection results, the present application further performs feature aggregation processing on the set of selected lithium battery surface state reference feature vectors to reduce the influence of random noise and non-essential differences, capture the typical features of the surface state of defect-free lithium batteries, and thus obtain the reference aggregate representation vector of the lithium battery surface state. In an embodiment of the present application, median pooling, clustering algorithm or principal component analysis and other methods can be used to perform feature aggregation processing on the set of selected lithium battery surface state reference feature vectors.

[0044] In the above-mentioned visual inspection system 100 for manufacturing new energy vehicles, the inspection image feature extraction module 140 is used to extract the lithium battery surface features of the surface image of the lithium battery to be inspected to obtain the surface state feature vector of the lithium battery to be inspected. In a specific example of the present application, the processing method of extracting the lithium battery surface features of the surface image of the lithium battery to be inspected is to input the surface image of the lithium battery to be inspected into the double-branch boundary information loss compensation lithium battery surface feature extraction network including the backbone network and the boundary feature extraction branch to obtain the surface state feature vector of the lithium battery to be inspected. That is, in the technical solution of the present application, the double-branch boundary information loss compensation lithium battery surface feature extraction network including the backbone network and the boundary feature extraction branch is also used to process the surface image of the lithium battery to be inspected, so as to effectively capture the global features and boundary detail features of the surface of the lithium battery to be inspected, so as to obtain a more comprehensive and accurate feature representation. It is worth mentioning that in the technical solution of the present application, by using the same network structure to extract features from the surface image of the lithium battery to be inspected and the reference image of the lithium battery surface without appearance defects, the comparability of feature representation can be ensured, thereby helping to improve the accuracy of subsequent defect detection.

[0045] In the above-mentioned visual inspection system 100 for new energy vehicle manufacturing, the semantic difference analysis module 150 is used to determine the defect detection result of the lithium battery to be inspected based on the semantic difference between the lithium battery surface state reference aggregate representation vector and the surface state feature vector of the lithium battery to be inspected. Figure 6 FIG. 1 is a block diagram of a semantic difference analysis module in a visual inspection system for new energy vehicle manufacturing according to an embodiment of the present application. Figure 6As shown, the semantic difference analysis module 150 includes: a semantic difference measurement unit 151, which is used to input the lithium battery surface state reference aggregation representation vector and the surface state feature vector of the lithium battery to be detected into a semantic difference measurement network to obtain a lithium battery detection-reference semantic difference representation vector; a defect detection result generation unit 152, which is used to input the lithium battery detection-reference semantic difference representation vector into a classifier-based defect detector to obtain the defect detection result, and the defect detection result is used to indicate whether there is a defect.

[0046] Specifically, the semantic difference measurement unit 151 is used to input the lithium battery surface state reference aggregate representation vector and the lithium battery surface state feature vector to be detected into the semantic difference measurement network to obtain the lithium battery detection-reference semantic difference representation vector. That is, the lithium battery surface state reference aggregate representation vector and the lithium battery surface state feature vector to be detected are further compared and analyzed through the semantic difference measurement network to reveal the semantic difference between the two, thereby providing an important basis for the surface defect detection of lithium batteries. Specifically, the semantic difference measurement network comprehensively represents the feature difference between the image to be detected and the reference image by performing multiple semantic interactions on the lithium battery surface state reference aggregate representation vector and the lithium battery surface state feature vector to be detected. Then, by performing convolution and pooling processing on the multiple semantic interaction results, the local correlation between the two is learned to generate a higher-level feature representation to obtain the lithium battery detection-reference semantic difference representation vector.

[0047] In a specific example of the present application, the semantic difference measurement unit 151 is used to: calculate the point-by-point addition, point-by-point subtraction and point-by-point multiplication between the lithium battery surface state reference aggregate representation vector and the surface state feature vector of the lithium battery to be detected to obtain a first semantic interaction result, a second semantic interaction result and a third semantic interaction result; after cascading the first semantic interaction result, the second semantic interaction result and the third semantic interaction result into a lithium battery detection-reference multi-scale semantic interaction feature vector, perform one-dimensional convolution processing and maximum pooling processing on it to obtain the lithium battery detection-reference semantic difference representation vector.

[0048] That is, the semantic difference measurement unit 151 is used to: process the lithium battery surface state reference aggregate representation vector and the surface state feature vector of the lithium battery to be detected using the following semantic difference measurement formula to obtain the lithium battery detection-reference semantic difference representation vector, wherein the semantic difference measurement formula is: ;in, Represents the surface state of the lithium battery reference aggregation representation vector, represents the surface state feature vector of the lithium battery to be detected, Indicates cascade operation, represents the maximum pooling operation, represents a one-dimensional convolution operation, represents dot product, Indicates point addition, Indicates point reduction, Represents the lithium battery detection-reference semantic difference representation vector.

[0049] Specifically, the defect detection result generating unit 152 is used to input the lithium battery detection-reference semantic difference representation vector into a classifier-based defect detector to obtain the defect detection result, and the defect detection result is used to indicate whether there is a defect. In the technical solution of the present application, the lithium battery detection-reference semantic difference representation vector contains difference information about the surface state of the lithium battery to be detected and the surface state of the reference defect-free lithium battery, and the classifier-based defect detector is trained using a large amount of labeled data, and can effectively learn and identify the feature patterns in the lithium battery detection-reference semantic difference representation vector, and output a binary classification result, namely "defects exist" or "defects do not exist", thereby realizing the automated detection of lithium battery surface defects.

[0050] In the technical solution of the present application, the lithium battery surface state reference aggregate representation vector is used to represent the feature clustering representation of the image semantic features of a set of lithium battery surface reference images marked as having no appearance defects after feature filtering and enhancement. However, considering that effective information components will be filtered out during feature filtering and the sample heterogeneity of each lithium battery surface state reference feature vector is not considered during feature aggregation, when the lithium battery surface state reference aggregate representation vector and the lithium battery surface state feature vector to be detected are input into the semantic difference metric network, the obtained lithium battery detection-reference semantic difference representation vector will have semantic metric dynamic structural differences due to differences in sample size, internal structural heterogeneity of samples, and erroneous filtering of effective components. Therefore, it is expected to improve the detail semantic aggregation expression effect of the lithium battery detection-reference semantic difference representation vector based on the semantic metric dynamic structural differences.

[0051] Accordingly, in a preferred embodiment of the present application, inputting the lithium battery detection-reference semantic difference representation vector into a classifier-based defect detector to obtain a defect detection result includes: calculating the absolute value sum of each eigenvalue of the lithium battery detection-reference semantic difference representation vector to obtain a first lithium battery detection-reference semantic difference and modulation value, and calculating the square root of the sum of squares of each eigenvalue of the lithium battery detection-reference semantic difference representation vector to obtain a second lithium battery detection-reference semantic difference and modulation value; which can be expressed as: ;in, is the lithium battery detection-reference semantic difference representation vector, represents the set of real numbers, The first The eigenvalues ​​at the positions, represents the number of eigenvalues ​​of the lithium battery detection-reference semantic difference representation vector, Indicates the first lithium battery detection-reference semantic difference and modulation value, Represents the second lithium battery detection-reference semantic difference and modulation value.

[0052] After point-wise subtraction of the lithium battery detection-reference semantic difference representation vector and the second lithium battery detection-reference semantic difference and modulation value, point-wise multiplication is performed with the number of eigenvalues ​​of the lithium battery detection-reference semantic difference representation vector and the reciprocal of the first lithium battery detection-reference semantic difference and modulation value, and the reciprocal of each eigenvalue is taken to obtain the first lithium battery detection-reference semantic difference phase conversion vector; it can be expressed as: ;in, is the lithium battery detection-reference semantic difference representation vector, Indicates the first lithium battery detection-reference semantic difference and modulation value, Indicates the second lithium battery detection-reference semantic difference and modulation value, represents the number of eigenvalues ​​of the lithium battery detection-reference semantic difference representation vector, It means point multiplication by position. It means subtracting by position. It means to calculate the inverse of each eigenvalue of the eigenvector. Represents the first lithium battery detection-reference semantic difference phase transition vector.

[0053] After point-wise subtraction of the lithium battery detection-reference semantic difference representation vector and the first lithium battery detection-reference semantic difference and modulation value, point-wise multiplication is performed with the square root of the number of eigenvalues ​​of the lithium battery detection-reference semantic difference representation vector and the reciprocal of the second lithium battery detection-reference semantic difference and modulation value, and the reciprocal of each eigenvalue is taken to obtain the second lithium battery detection-reference semantic difference phase conversion vector; it can be expressed as: ;in, is the lithium battery detection-reference semantic difference representation vector, Indicates the first lithium battery detection-reference semantic difference and modulation value, Indicates the second lithium battery detection-reference semantic difference and modulation value, represents the number of eigenvalues ​​of the lithium battery detection-reference semantic difference representation vector, It means point multiplication by position. It means subtracting by position. It means to calculate the inverse of each eigenvalue of the eigenvector. Represents the second lithium battery detection-reference semantic difference phase transition vector.

[0054] The first lithium battery detection-reference semantic difference phase conversion vector is point-subtracted from the second lithium battery detection-reference semantic difference phase conversion vector and the point product vector of the weighted hyperparameter to obtain an optimized lithium battery detection-reference semantic difference representation vector; the formula can be expressed as: ;in, It means point multiplication by position. It means subtracting by position. represents the first lithium battery detection-reference semantic difference phase transition vector, represents the second lithium battery detection-reference semantic difference phase conversion vector, represents the weighted hyperparameter, Representation optimized lithium battery detection-reference semantic difference representation vector.

[0055] The optimized lithium battery detection-reference semantic difference representation vector is input into a classifier-based defect detector to obtain a defect detection result.

[0056] That is, in the preferred example, the difference in the difference and modulation representation of the eigenvalue of the lithium battery detection-reference semantic difference representation vector relative to the feature set of the entire vector of the lithium battery detection-reference semantic difference representation vector is used as the semantic change intensity information, and a phase-like transformation corresponding to the position-based intensity modulation is performed through different and modulation representation forms, so that the aggregation enhancement of semantic change phase perception can enhance the axial aggregation receptive field along the feature aggregation direction by performing a spatial translation operation based on alternating stacking under the scale balance of the vector set of the lithium battery detection-reference semantic difference representation vector, thereby improving the perception effect of the aggregated semantics of the lithium battery detection-reference semantic difference representation vector for the detailed semantic changes, so as to improve the expression effect of the lithium battery detection-reference semantic difference representation vector, and improve the accuracy of the defect detection results obtained by passing it through a classifier-based defect detector.

[0057] In the above-mentioned visual inspection system 100 for new energy vehicle manufacturing, the sorting module 160 is used to place the lithium battery to be inspected into a defect group through a sorting robot in response to the defect detection result being defective. That is, for lithium batteries with a detection result of "defective", the sorting robot will automatically remove them from the production line and place them in a special defect group for further inspection and processing. In this way, the rapid screening and classification of defective lithium batteries is effectively achieved, thereby ensuring that only non-defective lithium batteries enter the subsequent production process.

[0058] In summary, according to the embodiment of the present application, a visual inspection system for new energy vehicle manufacturing is explained, which uses a set of reference images of lithium battery surfaces marked as having no appearance defects as reference data, and obtains the reference aggregate representation features of the lithium battery surface state through feature extraction, feature filtering and cluster analysis, and then obtains the semantic difference feature expression between the reference aggregate representation features of the lithium battery surface state and the representation features of the lithium battery surface state to be detected through a semantic difference metric network, and finally determines whether the lithium battery to be detected has surface defects based on the semantic difference feature expression. In this way, the excessive focus on the types of defects in traditional methods can be effectively avoided, thereby reducing the complexity of the model and improving the accuracy and efficiency of detection.

[0059] Figure 7 FIG. 1 is a flow chart of a visual inspection method for manufacturing new energy vehicles according to an embodiment of the present application. Figure 7 As shown, according to the embodiment of the present application, the visual inspection method for new energy vehicle manufacturing includes the following steps: S1, acquiring a surface image of a lithium battery to be inspected collected by a camera, and extracting a set of lithium battery surface reference images marked as having no appearance defects from a database; S2, performing lithium battery surface feature extraction on each lithium battery surface reference image marked as having no appearance defects in the set of lithium battery surface reference images marked as having no appearance defects to obtain a set of lithium battery surface state reference feature vectors; S3, performing feature selection optimization aggregation on the set of lithium battery surface state reference feature vectors to obtain a lithium battery surface state reference aggregation representation vector; S4, extracting lithium battery surface features of the surface image of the lithium battery to be inspected to obtain a lithium battery surface state feature vector to be inspected; S5, determining a defect detection result of the lithium battery to be inspected based on the semantic difference between the lithium battery surface state reference aggregation representation vector and the lithium battery surface state feature vector to be inspected; S6, in response to the defect detection result being a defect, placing the lithium battery to be inspected in a defect group by a sorting robot.

[0060] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned visual inspection method for new energy vehicle manufacturing have been referred to above. Figures 1 to 6 The description of the visual inspection system for new energy vehicle manufacturing has been introduced in detail, and therefore, its repeated description will be omitted.

[0061] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0062] In the above embodiments, the description of each embodiment has its own emphasis. For the parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0063] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference to a figure in a claim should not be considered as limiting the claim to which it relates.

[0064] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0065] Finally, it should be noted that the above description has been given for the purpose of illustration and description. In addition, the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A visual inspection system for new energy vehicle manufacturing, characterized in that: include: A data acquisition module, used to acquire a surface image of a lithium battery to be inspected captured by a camera, and extract a set of reference images of the surface of lithium batteries marked as having no appearance defects from a database; A reference image feature extraction module, used to extract lithium battery surface features from each lithium battery surface reference image marked as having no appearance defects in the set of lithium battery surface reference images marked as having no appearance defects, so as to obtain a set of lithium battery surface state reference feature vectors; A reference feature selection optimization aggregation module is used to perform feature selection optimization aggregation on the set of reference feature vectors of the lithium battery surface state to obtain a reference aggregation representation vector of the lithium battery surface state; A detection image feature extraction module, used to extract the lithium battery surface features of the surface image of the lithium battery to be detected to obtain a surface state feature vector of the lithium battery to be detected; A semantic difference analysis module, used to determine the defect detection result of the lithium battery to be detected based on the semantic difference between the reference aggregate representation vector of the lithium battery surface state and the feature vector of the surface state of the lithium battery to be detected; A sorting module, configured to place the lithium battery to be inspected into a defect group by a sorting robot in response to the defect detection result indicating that there is a defect; Wherein, the semantic difference analysis module includes: A semantic difference measurement unit, used for inputting the lithium battery surface state reference aggregation representation vector and the surface state feature vector of the lithium battery to be detected into a semantic difference measurement network to obtain a lithium battery detection-reference semantic difference representation vector; A defect detection result generating unit, used for inputting the lithium battery detection-reference semantic difference representation vector into a classifier-based defect detector to obtain the defect detection result, wherein the defect detection result is used to indicate whether there is a defect; Wherein, the semantic difference measurement unit is used to: Calculate the position point addition, position point subtraction and position point multiplication between the reference aggregate representation vector of the lithium battery surface state and the feature vector of the lithium battery surface state to be detected to obtain a first semantic interaction result, a second semantic interaction result and a third semantic interaction result; After the first semantic interaction result, the second semantic interaction result and the third semantic interaction result are cascaded into a lithium battery detection-reference multi-scale semantic interaction feature vector, one-dimensional convolution processing and maximum pooling processing are performed on them to obtain the lithium battery detection-reference semantic difference representation vector.

2. The visual inspection system for new energy vehicle manufacturing according to claim 1 is characterized in that: The reference image feature extraction module is used to: Each of the lithium battery surface reference images marked as having no appearance defects is input into a dual-branch boundary information loss compensation lithium battery surface feature extraction network to obtain a set of reference feature vectors of the lithium battery surface state, wherein the dual-branch boundary information loss compensation lithium battery surface feature extraction network comprises a backbone network and a boundary feature extraction branch.

3. The visual inspection system for new energy vehicle manufacturing according to claim 2 is characterized in that: The reference feature selection optimization aggregation module includes: A feature selection unit, used for inputting the set of reference feature vectors of the surface state of the lithium battery into a feature selection optimization module based on feature intensity distribution to obtain a set of selected reference feature vectors of the surface state of the lithium battery; A feature aggregation unit is used to perform feature aggregation on the set of selected lithium battery surface state reference feature vectors to obtain the lithium battery surface state reference aggregation representation vector.

4. The visual inspection system for new energy vehicle manufacturing according to claim 3 is characterized in that: The feature selection unit comprises: An energy distribution spectrum calculation subunit, used to calculate the characteristic energy distribution spectrum vector of each lithium battery surface state reference feature vector in the set of lithium battery surface state reference feature vectors to obtain a set of lithium battery surface state reference characteristic energy distribution spectrum vectors; An energy spectrum distribution center calculation subunit, used to calculate the position mean vector of the set of reference characteristic energy distribution spectrum vectors of the lithium battery surface state as the energy spectrum distribution center vector; An energy distribution spectrum span calculation subunit, used to calculate the Mahalanobis distance between the energy spectrum distribution center vector and each lithium battery surface state reference characteristic energy distribution spectrum vector in the set of lithium battery surface state reference characteristic energy distribution spectrum vectors as an energy distribution spectrum span factor to obtain a set of energy distribution spectrum span factors; A feature screening subunit is used to select the lithium battery surface state reference feature vectors corresponding to the energy distribution spectrum span factors less than the preset threshold value based on the comparison between each energy distribution spectrum span factor in the set of energy distribution spectrum span factors and the preset threshold value to form the set of selected lithium battery surface state reference feature vectors.

5. The visual inspection system for new energy vehicle manufacturing according to claim 4 is characterized in that: The energy distribution spectrum calculation subunit comprises: The feature distribution collaborative representation secondary subunit is used to calculate the feature distribution energy collaborative representation vector between the i-th lithium battery surface state reference feature vector and each other lithium battery surface state reference feature vector in the set of lithium battery surface state reference feature vectors in a manner of taking the maximum eigenvalue element by element to obtain a set of feature distribution energy collaborative representation vectors; The characteristic distribution energy synergy factor calculation secondary subunit is used to calculate the characteristic distribution energy synergy factor of each characteristic distribution energy synergy representation vector in the set of characteristic distribution energy synergy representation vectors to obtain the characteristic energy distribution spectrum vector of the i-th lithium battery surface state reference characteristic vector composed of multiple characteristic distribution energy synergy factors.

6. The visual inspection system for new energy vehicle manufacturing according to claim 5, characterized in that: The characteristic distribution energy synergy factor calculation secondary subunit is used to: The square of the difference between the maximum eigenvalue of the feature distribution energy synergistic representation vector and its eigenmean, the feature variance of the feature distribution energy synergistic representation vector, and the weighted sum of the bias terms are calculated, and then the variance of the feature distribution energy synergistic representation vector and the bias term are added and divided by the weighted sum to obtain the feature distribution energy synergy factor.

7. The visual inspection system for new energy vehicle manufacturing according to claim 6, characterized in that: The detection image feature extraction module is used to: The surface image of the lithium battery to be detected is input into the double-branch boundary information loss compensation lithium battery surface feature extraction network including a backbone network and a boundary feature extraction branch to obtain the surface state feature vector of the lithium battery to be detected.

8. A visual inspection method for new energy vehicle manufacturing, characterized in that: include: Acquire a surface image of a lithium battery to be inspected captured by a camera, and extract a set of lithium battery surface reference images marked as having no appearance defects from a database; Extracting lithium battery surface features from each lithium battery surface reference image marked as having no appearance defects in the set of lithium battery surface reference images marked as having no appearance defects to obtain a set of lithium battery surface state reference feature vectors; Performing feature selection optimization aggregation on the set of reference feature vectors of the lithium battery surface state to obtain a reference aggregate representation vector of the lithium battery surface state; Extracting the lithium battery surface features of the surface image of the lithium battery to be detected to obtain a surface state feature vector of the lithium battery to be detected; Determining a defect detection result of the lithium battery to be detected based on a semantic difference between the reference aggregate representation vector of the lithium battery surface state and the feature vector of the surface state of the lithium battery to be detected; In response to the defect detection result indicating that there is a defect, placing the lithium battery to be detected into a defect group by a sorting robot; Wherein, determining the defect detection result of the lithium battery to be detected based on the semantic difference between the reference aggregate representation vector of the lithium battery surface state and the feature vector of the surface state of the lithium battery to be detected includes: Inputting the lithium battery surface state reference aggregate representation vector and the surface state feature vector of the lithium battery to be detected into a semantic difference metric network to obtain a lithium battery detection-reference semantic difference representation vector; Inputting the lithium battery detection-reference semantic difference representation vector into a classifier-based defect detector to obtain the defect detection result, wherein the defect detection result is used to indicate whether there is a defect; The lithium battery surface state reference aggregate representation vector and the surface state feature vector of the lithium battery to be detected are input into a semantic difference metric network to obtain a lithium battery detection-reference semantic difference representation vector, including: Calculate the position point addition, position point subtraction and position point multiplication between the reference aggregate representation vector of the lithium battery surface state and the feature vector of the lithium battery surface state to be detected to obtain a first semantic interaction result, a second semantic interaction result and a third semantic interaction result; After the first semantic interaction result, the second semantic interaction result and the third semantic interaction result are cascaded into a lithium battery detection-reference multi-scale semantic interaction feature vector, one-dimensional convolution processing and maximum pooling processing are performed on them to obtain the lithium battery detection-reference semantic difference representation vector.

Images