Vehicle welding spot positioning method and device, electronic equipment and storage medium

The thickness and fine positioning of welding joints is solved through the neural network model, which is a problem of low positioning efficiency caused by the large number of welding joints on the white body, and improves the accuracy of welding joint detection and the safety performance of the whole vehicle.

CN118982580AActive Publication Date: 2024-11-19CHONGQING CHANGAN AUTOMOBILE CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411015819.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-26
Publication Date
2024-11-19
Estimated Expiration
2044-07-26

AI Technical Summary

Technical Problem

In the production and manufacturing of white body, there are many welding points, and the prior art is difficult to achieve efficient and accurate welding points positioning, resulting in low quality detection efficiency of welding points, difficult to identify small defects, and affecting the safety performance of the whole vehicle.

Method used

The neural network model is used to perform coarse positioning and fine positioning of welding joints. By obtaining vehicle images, feature extraction is performed first to obtain the welding joint images, and then accurately position them based on the shape of the welding joints, reducing the influence of environmental factors and improving positioning accuracy and efficiency .

Benefits of technology

It effectively improves the efficiency and accuracy of automatic positioning of welding joints, avoids the problems of accuracy and low efficiency caused by artificial positioning, enhances the reliability of welding joint detection, and ensures the safety performance of the entire vehicle.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118982580A_ABST
    Figure CN118982580A_ABST
Patent Text Reader

Abstract

The invention relates to a vehicle welding spot positioning method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a vehicle image; wherein the vehicle image is an image of a body in white containing a plurality of welding spots; performing feature extraction processing on the vehicle image according to a preset welding spot detection model to obtain a welding spot image corresponding to the vehicle image; wherein the preset welding spot detection model is a neural network model and is used for positioning welding spots in the vehicle image, and the welding spot image is used for representing the welding spots in the vehicle image; and performing welding spot detection processing on the welding spot image according to a preset welding spot shape to obtain position information of a welding spot in the welding spot image. Through coarse positioning and fine positioning, the accuracy of automatic positioning of the welding spots is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a method, a device, an electronic device and a storage medium for locating a welding point of a vehicle. Background Art

[0002] The body in white refers to the body of the vehicle that has been welded but not yet painted. In the production and manufacturing of the body in white, welding is a common connection method, among which spot welding is the most widely used process for welding the body in white. The quality of the welding spot has a crucial impact on the strength of the body in white and the safety performance of the whole vehicle.

[0003] It is necessary to conduct in-depth and accurate inspection of the quality of welds. However, there are many welds on the body-in-white, and the welds need to be located before quality inspection can be carried out. How to efficiently and accurately locate the welds on the body-in-white is a problem that needs to be solved urgently. Summary of the invention

[0004] The invention provides a method, a device, an electronic device and a storage medium for positioning a welding point of a vehicle, so as to improve the accuracy of positioning the welding point.

[0005] In a first aspect, the present invention provides a method for locating a welding point of a vehicle, comprising:

[0006] Acquire a vehicle image; wherein the vehicle image is an image of a body-in-white containing a plurality of welding spots;

[0007] According to a preset weld spot detection model, feature extraction processing is performed on the vehicle image to obtain a weld spot image corresponding to the vehicle image; wherein the preset weld spot detection model is a neural network model, which is used to locate weld spots in the vehicle image, and the weld spot image is used to characterize weld spots in the vehicle image;

[0008] According to the preset weld spot shape, weld spot detection processing is performed on the weld spot image to obtain position information of the weld spot in the weld spot image.

[0009] In a second aspect, the present invention provides a positioning device for a welding point of a vehicle, comprising:

[0010] An image acquisition unit, used to acquire a vehicle image; wherein the vehicle image is an image of a body-in-white containing a plurality of welding spots;

[0011] A coarse positioning unit, used to perform feature extraction processing on the vehicle image according to a preset weld detection model to obtain a weld image corresponding to the vehicle image; wherein the preset weld detection model is a neural network model, used to locate welds in the vehicle image, and the weld image is used to characterize the welds in the vehicle image;

[0012] The fine positioning unit is used to perform welding spot detection processing on the welding spot image according to a preset welding spot shape to obtain the position information of the welding spot in the welding spot image.

[0013] In a third aspect, the present invention provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0014] The memory stores computer-executable instructions;

[0015] The processor executes the computer-executable instructions stored in the memory to implement the method according to the first aspect.

[0016] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the first aspect.

[0017] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which implements the method described in the first aspect when executed by a processor.

[0018] The present invention provides a method, device, electronic device and storage medium for locating a vehicle welding point. By acquiring a vehicle image of a body-in-white, a preset welding point detection model is first used to roughly locate the welding points in the vehicle image, and a welding point image roughly characterizing each welding point is obtained. Then, based on the shape of the welding point, the welding point in the welding point image is finely located, and the characteristics of the welding point are accurately extracted to obtain the position of the welding point on the body-in-white. The problem of low welding point determination accuracy and efficiency caused by manual positioning is avoided. By performing positioning twice, the influence of environmental factors such as light is effectively reduced, and the efficiency and accuracy of automatic welding point positioning are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0020] Figure 1 A schematic diagram of a flow chart of a method for locating a welding point of a vehicle provided by an embodiment of the present invention;

[0021] Figure 2 A schematic diagram of a solder joint image provided by an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of a flow chart of a method for locating a welding point of a vehicle provided by an embodiment of the present invention;

[0023] Figure 4 A schematic diagram of the network structure of a solder joint detection model provided by an embodiment of the present invention;

[0024] Figure 5 A schematic diagram of the network structure of a solder joint detection model provided by an embodiment of the present invention;

[0025] Figure 6 A schematic diagram of a flow chart of a method for locating a welding point of a vehicle provided by an embodiment of the present invention;

[0026] Figure 7 A structural block diagram of a vehicle welding point positioning device provided by an embodiment of the present invention;

[0027] Figure 8 A structural block diagram of a vehicle welding point positioning device provided by an embodiment of the present invention;

[0028] Fig. 9 A structural block diagram of an electronic device provided by an embodiment of the present invention;

[0029] Fig.10 A structural block diagram of an electronic device provided by an embodiment of the present invention.

[0030] The above drawings have shown clear embodiments of the present invention, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present invention in any way, but to illustrate the concept of the present invention for those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0031] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0032] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0033] In the description of the present invention, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances. In addition, in the description of the present invention, unless otherwise specified, "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0034] It should be noted that due to space limitations, this specification does not exhaustively list all optional implementations. After reading this specification, those skilled in the art should be able to think that as long as the technical features do not contradict each other, any combination of technical features can constitute an optional implementation. The following is a detailed description of each embodiment.

[0035] In traffic accidents, most accidents are caused by the fracture of the body in white. The strength of the body in white plays an important role in the safety of passengers in the car. In the production and manufacturing of the body in white, the common connection method is welding, among which spot welding is the most widely used process for welding the body in white. The quality of the welds produced by spot welding has a crucial impact on the strength of the body in white and the safety performance of the whole vehicle. Therefore, it is necessary to conduct in-depth and accurate inspection of the quality of the welds.

[0036] There are probably more than 4,000 welds in a body-in-white, and the welds are widely distributed in key parts such as the front beam, floor, and roof of the body-in-white. Therefore, the quality of the welds directly affects the safety performance of the entire vehicle. At present, the main means of testing the quality of welds is manual inspection, which mainly includes appearance inspection and handheld flaw detector inspection. However, there are many limitations to the manual inspection of weld quality. For example, problems such as workshop light pollution and small weld size will bring difficulties to manual inspection. The large number of welds makes manual inspection inefficient and it is impossible to locate all welds. In addition, long-term repetitive work will make inspectors tired, and it is difficult to identify small defects in a short time, resulting in difficulty in assessing the defects of welds.

[0037] The present invention provides a method, device, electronic device and storage medium for locating a welding point on a vehicle, aiming to solve the above technical problems in the prior art.

[0038] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0039] Figure 1 FIG. 1 is a flow chart of a method for locating a welding spot of a vehicle provided according to an embodiment of the present invention. The method can be executed by a positioning processing device for a welding spot of a vehicle. Figure 1 As shown, the method comprises the following steps:

[0040] S101, obtaining a vehicle image; wherein the vehicle image is an image of a body-in-white containing a plurality of welding points.

[0041] For example, a vehicle body-in-white is welded with a plurality of welding spots, and the welding spots are distributed at various positions on the vehicle body-in-white. An image of the vehicle body-in-white is acquired as a vehicle image. That is, the vehicle image contains a plurality of welding spots. For example, the vehicle body-in-white can be photographed and collected by a camera to obtain the vehicle image.

[0042] S102. Perform feature extraction processing on the vehicle image according to a preset weld detection model to obtain a weld image corresponding to the vehicle image; wherein the preset weld detection model is a neural network model, which is used to locate welds in the vehicle image, and the weld image is used to characterize the welds in the vehicle image.

[0043] Exemplarily, a neural network model is pre-built and trained as a weld detection model. The weld detection model can be used to locate welds in a vehicle image and achieve rough positioning of welds in the vehicle image. The vehicle image can be input into a preset weld detection model, and according to the network structure of the weld detection model, feature extraction processing is performed on the vehicle image to extract the features of the welds in the vehicle image, and the weld image corresponding to the vehicle image is output according to the extracted features. The weld detected by the weld detection model can be represented in the weld image. In this embodiment, the weld detection model can adopt the neural network structure of CenterNet. For example, for the CenterNet neural network, ResNet18 can be selected as the feature extraction network of the weld detection model. A 7×7 convolution layer with a stride of 2 in ResNet18 can be changed into three 3×3 convolution layers, thereby reducing the number of parameters of the model and improving the receptive field of the model. The 1×1 convolution layer with a stride of 2 in ResNet18 can also be replaced with a 1×1 convolution layer with a stride of 1 and an average pooling layer with a stride of 2. In this embodiment, the model structure of the solder joint detection model is not specifically limited.

[0044] In this embodiment, images of different positions on the vehicle body in white are collected in advance as images to be trained. The images to be trained are divided into a training set, a validation set and a test set in a ratio of 7:1:2. Model parameters such as the feature extraction network structure, the learning rate and the number of training cycles of the weld spot detection model are pre-set. The images to be trained in the training set and the validation set are input into the initial neural network model for training and learning to obtain a predicted weld spot detection model, and the predicted model is generalized and verified using the images to be trained in the test set to obtain the final weld spot detection model.

[0045] Figure 2 is a schematic diagram of a welding spot image. The sizes of the welding spot image and the vehicle image may be the same. Figure 2 In the figure, the circles on the hood and roof can be represented as detected welds.

[0046] S103, performing welding spot detection processing on the welding spot image according to the preset welding spot shape, and obtaining the position information of the welding spot in the welding spot image.

[0047] Exemplarily, the shape of the solder joint may be circular or approximately circular. Based on the shape of the solder joint, the solder joints in the solder joint image are detected, all the solder joints in the solder joint image are found, and the position information of each solder joint is obtained. The position information may refer to the coordinate position of the solder joint in the solder joint image. For example, a preset detection algorithm may be used to detect a circular pattern in the solder joint image, and the position of the circular pattern is determined as the position of the solder joint. In this embodiment, the preset detection algorithm is not specifically limited.

[0048] In this embodiment, the welds have been roughly positioned by a preset weld detection model, and the weld image contains welds detected by the rough positioning as candidate welds. The target weld can be determined from multiple candidate welds by a preset detection algorithm to obtain the position information of the target weld. The target weld is the real weld on the body of the vehicle in white. For example, it can be detected whether the candidate weld is a preset weld shape. If so, the candidate weld is determined to be the target weld and the position information of the target weld is obtained; if not, the candidate weld is determined not to be the target weld.

[0049] In the embodiment of the present invention, by acquiring a vehicle image of the body-in-white, a preset weld point detection model is first used to roughly locate the weld points in the vehicle image, and a weld point image roughly characterizing each weld point is obtained. Then, based on the shape of the weld point, the weld point in the weld point image is finely located, and the features of the weld point are accurately extracted to obtain the position of the weld point on the body-in-white. The problem of low accuracy and efficiency of weld point determination caused by manual positioning is avoided. By performing positioning twice, the influence of environmental factors such as lighting is effectively reduced, and the efficiency and accuracy of automatic positioning of weld points are improved.

[0050] Figure 3A schematic flow chart of a method for locating a welding point on a vehicle provided in an embodiment of the present invention is an optional embodiment based on the above embodiment.

[0051] In this embodiment, according to a preset weld spot detection model, feature extraction processing is performed on the vehicle image to obtain a weld spot image corresponding to the vehicle image, including: preprocessing the vehicle image to obtain a preprocessed vehicle image; wherein the preprocessing is used to improve the image quality of the vehicle image; according to the preset weld spot detection model, feature extraction processing is performed on the preprocessed vehicle image to obtain a weld spot image corresponding to the preprocessed vehicle image.

[0052] like Figure 3 As shown, the method comprises the following steps:

[0053] S301, obtaining a vehicle image; wherein the vehicle image is an image of a body-in-white containing a plurality of welding points.

[0054] Exemplarily, this step may refer to the above-mentioned step S101 and will not be described in detail.

[0055] S302, preprocessing the vehicle image to obtain a preprocessed vehicle image; wherein the preprocessing is used to improve the image quality of the vehicle image.

[0056] Exemplarily, the vehicle image may be an image captured in a workshop environment. The light pollution in the workshop is serious, and there will be many noise points on the image, resulting in unclear outlines of the welding points, so the vehicle image needs to be preprocessed. For example, affected by the lighting, there are many light spots on the vehicle image, which are easily confused with the welding points, affecting the positioning accuracy of the welding points. The noise points on the vehicle image can be removed by preprocessing, and the real welding points can be retained. The preprocessing algorithm can be preset according to the actual shooting effect of the vehicle image. For example, if there are many noise points on the vehicle image, an algorithm for removing the noise points can be preset; if the pixels of the vehicle image are not clear, an algorithm for adjusting the resolution can be preset, etc. In this embodiment, the preset preprocessing algorithm is not specifically limited.

[0057] In this embodiment, the vehicle image is preprocessed to obtain a preprocessed vehicle image, including: based on a preset noise reduction algorithm, performing noise reduction filtering processing on the vehicle image to obtain a noise-reduced vehicle image; based on a preset filtering operator, performing contour enhancement processing on the noise-reduced vehicle image to obtain a preprocessed vehicle image.

[0058] Specifically, the vehicle image is preprocessed to enhance the features of the welding points in the image. A noise reduction algorithm can be pre-set, and the noise reduction algorithm is used to remove the noise points in the vehicle image. After obtaining the vehicle image, the vehicle image can be input into a preset noise reduction algorithm, and the vehicle image is subjected to noise reduction filtering processing by the preset noise reduction algorithm to remove the noise points in the vehicle image and obtain a noise-reduced vehicle image. For example, a non-local mean noise reduction algorithm can be used to perform noise reduction filtering processing on the vehicle image to remove image noise.

[0059] A filter operator may also be preset, and the filter operator may be used to represent an algorithm for image enhancement processing. After obtaining the vehicle image after noise reduction, the preset filter operator may be used to perform contour enhancement processing on the vehicle image after noise reduction, that is, to enhance the contour of the welding point in the image. The enhanced image is used as the preprocessed vehicle image. For example, the Laplacian operator may be used to perform enhancement processing on the image after noise reduction.

[0060] The beneficial effect of such a setting is that by preprocessing the vehicle image, the relevant features of the welding point in the vehicle image are improved, and the influence of environmental factors on the positioning accuracy is reduced, thereby improving the positioning accuracy of the subsequent neural network model for the welding point position.

[0061] S303 . Perform feature extraction processing on the preprocessed vehicle image according to a preset welding spot detection model to obtain a welding spot image corresponding to the preprocessed vehicle image.

[0062] For example, after obtaining the preprocessed vehicle image, the preprocessed vehicle image can be input into a preset weld spot detection model. The weld spot detection model performs feature extraction processing on the preprocessed vehicle image and outputs a weld spot image, which includes the weld spots obtained after rough positioning. Figure 4 Schematic diagram of the network structure of the solder joint detection model in this embodiment. Figure 4 In the solder joint detection model, the input layer, output layer, and four Basic Blocks are included. The four Basic Blocks are Stage0, Stage1, Stage2, Stage3, and Stage4. The first Basic Block, that is, the 7×7 convolution layer with a stride of 2 in Stage0, can be changed into three 3×3 convolution layers. The 1×1 convolution layers with a stride of 2 in the last four Basic Blocks, that is, Stage1, Stage2, Stage3, and Stage4, are replaced by a 1×1 convolution layer with a stride of 1 plus an average pooling layer with a stride of 2.

[0063] In this embodiment, the vehicle image is first preprocessed by noise reduction, sharpening enhancement, etc. to improve the image quality, and then the preprocessed image is sent to a trained weld detection model to obtain coarse positioning features, improve the accuracy of coarse positioning, and further improve the accuracy of subsequent fine positioning.

[0064] In this embodiment, the preset weld detection model includes a first convolutional network and a second convolutional network; according to the preset weld detection model, feature extraction processing is performed on the preprocessed vehicle image to obtain a weld image corresponding to the preprocessed vehicle image, including: inputting the preprocessed vehicle image into the preset weld detection model, and based on the first convolutional network, feature extraction processing is performed on the preprocessed vehicle image to obtain a first processed image; wherein the preset first convolutional network includes at least two convolutional layers; based on the second convolutional network, feature extraction processing is performed on the preprocessed vehicle image to obtain a second processed image; wherein the preset second convolutional network includes at least one pooling layer and at least two convolutional layers; according to the first processed image and the second processed image, a weld image is obtained.

[0065] Specifically, the weld detection model may include two image processing branches, and the two branches perform image processing respectively. For example, the weld detection model may include a first convolutional network and a second convolutional network, the first convolutional network represents one branch, and the second convolutional network represents another branch. The preprocessed vehicle image is input into the weld detection model, and the preprocessed vehicle image passes through the first convolutional network and the second convolutional network respectively. The first convolutional network can obtain a first processed image, and the second convolutional network can obtain a second processed image.

[0066] The first convolutional network may include multiple convolutional layers. For example, the preprocessed vehicle image first undergoes a 1×1 convolution operation in the first convolutional network, and then undergoes a convolution operation with a convolution kernel size of K to output a first processed image. In this embodiment, K may be a convolution kernel size of 3×3. The second convolutional network may include a pooling layer and a convolutional layer. For example, there may be one pooling layer and multiple convolutional layers. The preprocessed vehicle image may first undergo an average pooling operation in the pooling layer, and then undergo a convolution operation in multiple convolutional layers to output a second processed image.

[0067] The first processed image and the second processed image are combined to obtain a solder joint image. For example, the first processed image and the second processed image may be overlapped, and the overlapped image may be determined as the solder joint image. For another example, different weights may be set for the first processed image and the second processed image, and based on the preset weights, the feature vectors represented by the first processed image and the second processed image are weighted summed to obtain a feature vector of the solder joint image, and then the feature vector of the solder joint image is converted into the solder joint image.

[0068] The beneficial effect of such a setting is that different feature extraction processes are performed on the pre-processed vehicle image to obtain two processed images, and the two processed images are combined to obtain a welding point image, thereby improving the feature extraction accuracy of the welding point and further improving the positioning accuracy of the welding point.

[0069] In this embodiment, based on the first convolutional network, feature extraction processing is performed on the preprocessed vehicle image to obtain a first processed image, including: performing dimensionality reduction processing on the preprocessed vehicle image through the first convolutional layer in the first convolutional network to obtain a first reduced dimensionality feature vector; performing convolution processing on the first reduced dimensionality feature vector through the second convolutional layer in the preset first convolutional network to obtain the first processed image.

[0070] Specifically, the first convolutional network includes a first convolutional layer, which may be a 1×1 convolutional layer. After entering the first convolutional network, the preprocessed vehicle image is first subjected to dimensionality reduction processing by the first convolutional layer, that is, dimensionality reduction is performed by a 1×1 convolution operation. The feature vector obtained after dimensionality reduction is determined as the first dimensionality reduction feature vector.

[0071] The first convolution network may further include a second convolution layer, and the second convolution layer may be a convolution layer with a convolution kernel size of 3×3. After obtaining the first reduced-dimensionality feature vector, the first reduced-dimensionality feature vector may be input into the second convolution layer. The second convolution layer performs convolution processing on the first reduced-dimensionality feature vector to extract the features of the welding points in the image, and the second convolution layer may output a first processed image. That is, the first processed image is an image obtained by performing feature extraction processing on the preprocessed vehicle image through two different convolution layers.

[0072] The beneficial effect of such a setting is that dimensionality reduction is performed first and then convolution is performed, which facilitates the extraction of solder joint features and improves the accuracy of subsequent positioning of solder joints.

[0073] In this embodiment, based on the second convolutional network, feature extraction processing is performed on the preprocessed vehicle image to obtain a second processed image, including: performing dimensionality reduction processing on the preprocessed vehicle image through the third convolutional layer in the second convolutional network to obtain a second reduced dimensionality feature vector; performing feature extraction processing on the second reduced dimensionality feature vector through a self-correcting convolutional network preset in the second convolutional network to obtain a second processed image; wherein the self-correcting convolutional network includes at least one pooling layer and at least two convolutional layers.

[0074] Specifically, the second convolutional network may include a third convolutional layer, and the third convolutional layer may also be a 1×1 convolutional layer, that is, the third convolutional layer may be the same as the first convolutional layer. After the preprocessed vehicle image enters the second convolutional network, it is first subjected to dimensionality reduction processing by the third convolutional layer, that is, it is also subjected to dimensionality reduction by a 1×1 convolution operation. The feature vector obtained after dimensionality reduction is determined as a second dimensionality reduction feature vector. The second dimensionality reduction feature vector may be the same as the first dimensionality reduction feature vector.

[0075] The second convolutional network also includes a self-correcting convolutional network, which may include at least one pooling layer and at least two convolutional layers. After obtaining the second reduced-dimensionality feature vector, the second reduced-dimensionality feature vector can be input into the self-correcting convolutional network. The second reduced-dimensionality feature vector is subjected to feature extraction processing by the self-correcting convolutional network in the second convolutional network, and a second processed image is obtained by output. For example, the second reduced-dimensionality feature vector can be subjected to pooling and convolution operations by the self-correcting convolutional network to obtain a second processed image. In this embodiment, the network structure in the self-correcting convolutional network is not specifically limited.

[0076] The beneficial effect of this setting is that in the second convolutional network, dimensionality reduction is performed first and then convolution, and the convolution method is different from that of the first convolutional network, which can achieve comprehensive extraction of solder joint features, avoid feature omissions, improve the accuracy of solder joint detection, and further improve the accuracy of solder joint positioning.

[0077] In this embodiment, feature extraction processing is performed on the second reduced dimensionality feature vector through a self-correcting convolutional network in the second convolutional network to obtain a second processed image, including: performing an average pooling operation on the second reduced dimensionality feature vector through a pooling layer in the self-correcting convolutional network to obtain a first vector; performing convolution processing on the second reduced dimensionality feature vector through a fourth convolutional layer in the self-correcting convolutional network to obtain a second vector; performing convolution processing on the second reduced dimensionality feature vector through a fifth convolutional layer in the self-correcting convolutional network to obtain a third vector; and obtaining a second processed image based on the first vector, the second vector, and the third vector.

[0078] Specifically, the self-correcting convolutional network includes a pooling layer, a fourth convolutional layer, and a fifth convolutional layer, and the pooling layer, the fourth convolutional layer, and the fifth convolutional layer can represent three branches in the self-correcting convolutional network. That is, the second dimensionality reduction feature vector can be input into the pooling layer, the fourth convolutional layer, and the fifth convolutional layer, respectively, and the pooling layer, the fourth convolutional layer, and the fifth convolutional layer respectively perform feature extraction processing and output respective output data. In this embodiment, the processing order of these three branches is not specifically limited.

[0079] The second reduced-dimensionality feature vector is input into the pooling layer of the self-correcting convolutional network, and the second reduced-dimensionality feature vector is averaged pooled by the pooling layer, and the data output after the pooling operation is determined as the first vector; the second reduced-dimensionality feature vector is input into the fourth convolutional layer of the self-correcting convolutional network, and the second reduced-dimensionality feature vector is convolved by the fourth convolutional layer, and the second vector is output; the second reduced-dimensionality feature vector is input into the fifth convolutional layer of the self-correcting convolutional network, and the second reduced-dimensionality feature vector is convolved by the fifth convolutional layer, and the third vector is output. The fourth convolutional layer and the fifth convolutional layer can be convolutional layers with preset convolutional kernel sizes, for example, the preset convolutional kernel size can be 3×3.

[0080] The first vector, the second vector and the third vector are combined to output a second processed image by the second convolutional network. For example, the first vector, the second vector and the third vector may be added or multiplied to obtain the second processed image.

[0081] The beneficial effect of such a setting is that the second dimension reduction feature vector is processed in three branches, namely pooling, convolution, and convolution, which improves the comprehensiveness of feature extraction and thus improves the accuracy of solder joint positioning.

[0082] In this embodiment, an average pooling operation is performed on the second reduced-dimensionality feature vector through a pooling layer in a self-correcting convolutional network to obtain a first vector, including: performing an average pooling operation on the second reduced-dimensionality feature vector through a pooling layer in a self-correcting convolutional network to obtain an average pooling vector; and convolving the average pooling vector through a sixth convolutional layer in the self-correcting convolutional network to obtain the first vector.

[0083] Specifically, in a branch of the self-correcting convolutional network, the second dimension reduction feature vector is first averaged by the pooling layer, and the second dimension reduction feature vector is downsampled by R times by the average pooling operation to obtain an average pooled vector. For example, R can be 4. The self-correcting convolutional network can also include a sixth convolutional layer, and the sixth convolutional layer can be connected after the pooling layer. After obtaining the average pooled vector, the average pooled vector can be input into the sixth convolutional layer, and the average pooled vector is convolved by the sixth convolutional layer, and the data output by the sixth convolutional layer can be the first vector.

[0084] The beneficial effect of such a setting is that the extraction accuracy of the solder joint features is improved by performing pooling and convolution operations on the second dimensionality reduction feature vector.

[0085] In this embodiment, the average pooling vector is convolved through the sixth convolutional layer in the self-correcting convolutional network to obtain a first vector, including: performing a convolution operation on the average pooling vector according to a preset convolution kernel size of the sixth convolutional layer in the self-correcting convolutional network to obtain a feature vector extracted after the convolution operation; upsampling the feature vector extracted after the convolution operation through a preset bilinear interpolation algorithm to obtain the first vector.

[0086] Specifically, the convolution kernel size of the sixth convolution layer is preset, for example, the preset convolution sum size may be 3×3. A convolution operation with a convolution kernel size of 3×3 is performed on the average pooling vector, and the feature vector extracted after the convolution operation is output. A bilinear interpolation algorithm is preset, and according to the bilinear interpolation algorithm, the feature vector extracted after the convolution operation is upsampled by R times to obtain the first vector, and R may also be 4.

[0087] That is to say, in one branch of the self-correcting convolutional network, the second reduced-dimensionality feature vector is first subjected to an average pooling operation, and the second reduced-dimensionality feature vector is downsampled by R times to obtain an average pooling vector. Then, a convolution operation is performed on the average pooling vector, and then, based on the bilinear interpolation algorithm, the feature vector extracted after the convolution operation is upsampled by R times to obtain the first vector.

[0088] The beneficial effect of such a setting is that the extraction accuracy of solder joint features is improved, feature omission is avoided, and the accuracy of solder joint positioning is further improved.

[0089] In this embodiment, a second processed image is obtained based on the first vector, the second vector, and the third vector, including: summing the first vector and the second vector to obtain a feature vector after the summation; multiplying the feature vector after the summation with the third vector to obtain a feature vector after the multiplication; and performing a convolution operation on the feature vector after the multiplication to obtain the second processed image.

[0090] Specifically, after obtaining the first vector and the second vector, the first vector and the second vector may be combined. For example, the first vector and the second vector may be element-wise summed to obtain a feature vector after the summation. Alternatively, the first vector and the second vector may be element-wise summed first, and then the calculated vector may be processed using a preset Sigmoid (S-type) activation function, and the processed result may be used as the feature vector after the summation.

[0091] After obtaining the feature vector after the summation process and the third vector, the feature vector after the summation process and the third vector can be combined again. For example, the feature vector after the summation process and the third vector can be element-wise multiplied to calculate the feature vector after the multiplication process. The feature vector after the multiplication process is convolved again to output a second processed image. For example, a 3×3 convolution operation can be performed again.

[0092] The beneficial effect of such a setting is that the summation and multiplication between elements of the feature vector are performed to achieve the fusion between features, improve the determination accuracy of the solder joint image, and further improve the positioning accuracy of the solder joint.

[0093] After obtaining the first processed image and the second processed image, the first processed image and the second processed image may be merged to obtain a solder joint image. The first processed image and the second processed image may be added by the number of convolution layers to obtain a solder joint image. For example, the first processed image is represented as (C, H, W), and the second processed image is represented as (C, H, W), then the solder joint image may be represented as (2C, H, W). Where C represents the number of convolution layers, H represents the image height, and W represents the image width.

[0094] Figure 5 Schematic diagram of the network structure of the solder joint detection model in this embodiment. Figure 5 In the figure, the first convolutional network outputs a first processed image, the second convolutional network outputs a second processed image, and the output layer outputs a solder joint image.

[0095] S304: Performing welding spot detection processing on the welding spot image according to the preset welding spot shape to obtain the position information of the welding spot in the welding spot image.

[0096] Exemplarily, this step may refer to the above-mentioned step S103 and will not be described in detail.

[0097] In the embodiment of the present invention, by acquiring a vehicle image of the body-in-white, a preset weld point detection model is first used to roughly locate the weld points in the vehicle image, and a weld point image roughly characterizing each weld point is obtained. Then, based on the shape of the weld point, the weld point in the weld point image is finely located, and the features of the weld point are accurately extracted to obtain the position of the weld point on the body-in-white. The problem of low accuracy and efficiency of weld point determination caused by manual positioning is avoided. By performing positioning twice, the influence of environmental factors such as lighting is effectively reduced, and the efficiency and accuracy of automatic positioning of weld points are improved.

[0098] Figure 6 A schematic flow chart of a method for locating a welding point on a vehicle provided in an embodiment of the present invention is an optional embodiment based on the above embodiment.

[0099] In this embodiment, the preset weld shape is a circle; according to the preset weld shape, the weld image is subjected to weld detection processing to obtain the position information of the weld in the weld image, including: through a preset Hough circle detection algorithm, the weld image is subjected to positioning processing of the preset weld shape to obtain the position information of the weld in the weld image.

[0100] like Figure 6 As shown, the method comprises the following steps:

[0101] S601, obtaining a vehicle image; wherein the vehicle image is an image of a body-in-white containing a plurality of welding points.

[0102] Exemplarily, this step may refer to the above-mentioned step S101 and will not be described in detail.

[0103] S602. Perform feature extraction processing on the vehicle image according to a preset weld detection model to obtain a weld image corresponding to the vehicle image; wherein the preset weld detection model is a neural network model, which is used to locate welds in the vehicle image, and the weld image is used to characterize the welds in the vehicle image.

[0104] Exemplarily, this step may refer to the above-mentioned step S102 and will not be described in detail.

[0105] S603: Performing a positioning process of a preset solder joint shape on the solder joint image by using a preset Hough circle detection algorithm to obtain position information of the solder joint in the solder joint image.

[0106] Exemplarily, the preset solder joint shape is a circle, and the Hough circle detection algorithm can be pre-set to detect the circular solder joint. The Hough circle detection algorithm can detect and locate circular targets in a given image. Therefore, the circular solder joint in the solder joint image can be located by the Hough circle detection algorithm, thereby obtaining the position information of the solder joint. In this embodiment, the detection process of the Hough circle detection algorithm is not specifically limited. For example, when executing the Hough circle detection algorithm, an edge detection can be performed on the solder joint image first to obtain a binary image corresponding to the solder joint image, that is, a black and white image. The Sobel operator is used to calculate the binary image to calculate the neighborhood gradient values ​​of all pixels, and then the Hough circle detection algorithm is used to calculate the specific position of the solder joint in the binary image.

[0107] The method of this embodiment first removes noise points on the image and enhances the characteristic contours of the welding points by performing noise reduction and sharpening on the vehicle image; then the welding point image is sent to the improved CenterNet neural network for prediction to perform rough positioning of the body-in-white welding points; finally, the Hough circle detection algorithm is used for precise positioning to obtain the position information of the welding points.

[0108] In the embodiment of the present invention, by acquiring a vehicle image of the body-in-white, a preset weld point detection model is first used to roughly locate the weld points in the vehicle image, and a weld point image roughly characterizing each weld point is obtained. Then, based on the shape of the weld point, the weld point in the weld point image is finely located, and the features of the weld point are accurately extracted to obtain the position of the weld point on the body-in-white. The problem of low accuracy and efficiency of weld point determination caused by manual positioning is avoided. By performing positioning twice, the influence of environmental factors such as lighting is effectively reduced, and the efficiency and accuracy of automatic positioning of weld points are improved.

[0109] Figure 7 This is a structural block diagram of a vehicle welding point positioning device provided by an embodiment of the present invention. For ease of description, only the parts related to the embodiment of the present disclosure are shown. Figure 7 The vehicle welding point positioning device 700 includes: an image acquisition unit 701, a coarse positioning unit 702 and a fine positioning unit 703.

[0110] The image acquisition unit 701 is used to acquire a vehicle image; wherein the vehicle image is an image of a body-in-white having a plurality of welding spots;

[0111] A coarse positioning unit 702 is used to perform feature extraction processing on the vehicle image according to a preset weld detection model to obtain a weld image corresponding to the vehicle image; wherein the preset weld detection model is a neural network model, which is used to locate welds in the vehicle image, and the weld image is used to characterize welds in the vehicle image;

[0112] The fine positioning unit 703 is used to perform a welding spot detection process on the welding spot image according to a preset welding spot shape, so as to obtain the position information of the welding spot in the welding spot image.

[0113] Figure 8 A structural block diagram of a vehicle welding point positioning device provided by an embodiment of the present disclosure, such as Figure 8 As shown, the vehicle welding point positioning device 800 includes an image acquisition unit 801 , a coarse positioning unit 802 and a fine positioning unit 803 , wherein the coarse positioning unit 802 includes a preprocessing module 8021 and a feature extraction module 8022 .

[0114] A preprocessing module 8021 is used to preprocess the vehicle image to obtain a preprocessed vehicle image; wherein the preprocessing is used to improve the image quality of the vehicle image;

[0115] The feature extraction module 8022 is used to perform feature extraction processing on the preprocessed vehicle image according to a preset weld spot detection model to obtain a weld spot image corresponding to the preprocessed vehicle image.

[0116] In one example, the preprocessing module 8021 is specifically used for:

[0117] Based on a preset noise reduction algorithm, performing noise reduction filtering processing on the vehicle image to obtain a noise-reduced vehicle image;

[0118] Based on a preset filtering operator, contour enhancement processing is performed on the denoised vehicle image to obtain the preprocessed vehicle image.

[0119] In one example, the preset solder joint detection model includes a first convolutional network and a second convolutional network; the feature extraction module 8022 includes:

[0120] A first processing submodule, used for inputting the preprocessed vehicle image into a preset welding spot detection model, and performing feature extraction processing on the preprocessed vehicle image based on the first convolutional network to obtain a first processed image; wherein the preset first convolutional network includes at least two convolutional layers;

[0121] A second processing submodule, configured to perform feature extraction processing on the preprocessed vehicle image based on the second convolutional network to obtain a second processed image; wherein the preset second convolutional network includes at least one pooling layer and at least two convolutional layers;

[0122] An image combining module is used to obtain the welding spot image according to the first processed image and the second processed image.

[0123] In one example, the first processing submodule is specifically used to:

[0124] Performing dimensionality reduction processing on the preprocessed vehicle image through a first convolutional layer in the first convolutional network to obtain a first dimensionality reduction feature vector;

[0125] The first reduced-dimensional feature vector is convolved through the second convolution layer in the preset first convolution network to obtain the first processed image.

[0126] In one example, the second processing submodule is specifically used for:

[0127] Performing dimensionality reduction processing on the preprocessed vehicle image through the third convolution layer in the second convolution network to obtain a second dimensionality reduction feature vector;

[0128] The second reduced-dimensional feature vector is subjected to feature extraction processing by a self-correcting convolutional network in the preset second convolutional network to obtain the second processed image; wherein the self-correcting convolutional network includes at least one pooling layer and at least two convolutional layers.

[0129] In one example, the second processing submodule is specifically used for:

[0130] Performing an average pooling operation on the second reduced-dimensionality feature vector through a pooling layer in the self-correcting convolutional network to obtain a first vector;

[0131] Performing convolution processing on the second reduced-dimensionality feature vector through a fourth convolution layer in the self-correcting convolutional network to obtain a second vector;

[0132] Performing convolution processing on the second dimension reduction feature vector through a fifth convolution layer in the self-correcting convolutional network to obtain a third vector;

[0133] The second processed image is obtained according to the first vector, the second vector, and the third vector.

[0134] In one example, the second processing submodule is specifically used for:

[0135] Performing an average pooling operation on the second dimension reduction feature vector through a pooling layer in the self-correcting convolutional network to obtain an average pooling vector;

[0136] The average pooling vector is convolved through the sixth convolutional layer in the self-correcting convolutional network to obtain a first vector.

[0137] In one example, the second processing submodule is specifically used for:

[0138] Performing a convolution operation on the average pooling vector according to a preset convolution kernel size of the sixth convolution layer in the self-correcting convolutional network to obtain a feature vector extracted after the convolution operation;

[0139] The feature vector extracted after the convolution operation is upsampled by a preset bilinear interpolation algorithm to obtain the first vector.

[0140] In one example, the second processing submodule is specifically used for:

[0141] Performing a summation process on the first vector and the second vector to obtain a summed feature vector;

[0142] Multiplying the summed feature vector with the third vector to obtain a multiplied feature vector;

[0143] A convolution operation is performed on the feature vector after the multiplication process to obtain the second processed image.

[0144] In one example, the preset welding spot shape is a circle; the fine positioning unit 803 includes:

[0145] The Hough circle detection module is used to perform a positioning process of a preset solder joint shape on the solder joint image by using a preset Hough circle detection algorithm to obtain the position information of the solder joint in the solder joint image.

[0146] Fig. 9 A structural block diagram of an electronic device provided in an embodiment of the present application, such as Fig. 9 As shown, the electronic device includes: a memory 91 and a processor 92; the memory 91 is a memory for storing instructions executable by the processor 92.

[0147] The processor 92 is configured to execute the method provided in the above embodiment.

[0148] The electronic device further includes a receiver 93 and a transmitter 94. The receiver 93 is used to receive instructions and data sent by other devices, and the transmitter 94 is used to send instructions and data to external devices.

[0149] Fig.10 It is a block diagram of a terminal device according to an exemplary embodiment. The device may be a mobile phone, a computer, a digital broadcast terminal, a message transceiver, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

[0150] Device 1000 may include one or more of the following components: a processing component 1002 , a memory 1004 , a power component 1006 , a multimedia component 1008 , an audio component 1010 , an input / output (I / O) interface 1012 , a sensor component 1014 , and a communication component 1016 .

[0151] The processing component 1002 generally controls the overall operation of the device 900, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1002 may include one or more processors 1020 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 1002 may include one or more modules to facilitate the interaction between the processing component 1002 and other components. For example, the processing component 1002 may include a multimedia module to facilitate the interaction between the multimedia component 1008 and the processing component 1002.

[0152] Device 1000 may include one or more of the following components: a processing component 1002 , a memory 1004 , a power component 1006 , a multimedia component 1008 , an audio component 1010 , an input / output (I / O) interface 1012 , a sensor component 1014 , and a communication component 1016 .

[0153] The memory 1004 is configured to store various types of data to support the operation of the device 1000. Examples of such data include instructions for any application or method operating on the device 1000, contact data, phone book data, messages, pictures, videos, etc. The memory 1004 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0154] The power supply component 1006 provides power to the various components of the device 1000. The power supply component 1006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 1000.

[0155] The multimedia component 1008 includes a screen that provides an output interface between the device 1000 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1008 includes a front camera and / or a rear camera. When the device 1000 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0156] The audio component 1010 is configured to output and / or input audio signals. For example, the audio component 1010 includes a microphone (MIC), and when the device 1000 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1004 or sent via the communication component 1016. In some embodiments, the audio component 1010 also includes a speaker for outputting audio signals.

[0157] I / O interface 1012 provides an interface between processing component 1002 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0158] The sensor assembly 1014 includes one or more sensors for providing various aspects of status assessment for the device 1000. For example, the sensor assembly 1014 can detect the open / closed state of the device 1000, the relative positioning of components, such as the display and keypad of the device 900, the sensor assembly 914 can also detect the position change of the device 900 or a component of the device 900, the presence or absence of user contact with the device 900, the orientation or acceleration / deceleration of the device 900 and the temperature change of the device 900. The sensor assembly 1014 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 1014 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1014 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0159] The communication component 1016 is configured to facilitate wired or wireless communication between the device 1000 and other devices. The device 1000 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1016 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1016 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0160] In an exemplary embodiment, the apparatus 1000 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described methods.

[0161] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1004 including instructions, and the instructions can be executed by the processor 1020 of the device 1000 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0162] A non-temporary computer-readable storage medium, when instructions in the storage medium are executed by a processor of a terminal device, enables the terminal device to execute the vehicle welding point positioning method of the terminal device.

[0163] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0164] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for locating a welding point of a vehicle, characterized in that: include: Acquire a vehicle image; wherein the vehicle image is an image of a body-in-white containing a plurality of welding spots; According to a preset weld spot detection model, feature extraction processing is performed on the vehicle image to obtain a weld spot image corresponding to the vehicle image; wherein the preset weld spot detection model is a neural network model, which is used to locate weld spots in the vehicle image, and the weld spot image is used to characterize weld spots in the vehicle image; According to the preset weld shape, weld detection processing is performed on the weld image to obtain position information of the weld in the weld image.

2. The method according to claim 1, characterized in that According to a preset welding spot detection model, feature extraction processing is performed on the vehicle image to obtain a welding spot image corresponding to the vehicle image, including: Preprocessing the vehicle image to obtain a preprocessed vehicle image; wherein the preprocessing is used to improve the image quality of the vehicle image; According to a preset welding spot detection model, feature extraction processing is performed on the preprocessed vehicle image to obtain a welding spot image corresponding to the preprocessed vehicle image.

3. The method according to claim 2, characterized in that Preprocessing the vehicle image to obtain a preprocessed vehicle image includes: Based on a preset noise reduction algorithm, performing noise reduction filtering processing on the vehicle image to obtain a noise-reduced vehicle image; Based on a preset filtering operator, contour enhancement processing is performed on the denoised vehicle image to obtain the preprocessed vehicle image.

4. The method according to claim 2, characterized in that: The preset weld spot detection model includes a first convolutional network and a second convolutional network; according to the preset weld spot detection model, feature extraction processing is performed on the preprocessed vehicle image to obtain a weld spot image corresponding to the preprocessed vehicle image, including: Inputting the preprocessed vehicle image into a preset welding spot detection model, and performing feature extraction processing on the preprocessed vehicle image based on the first convolutional network to obtain a first processed image; wherein the preset first convolutional network includes at least two convolutional layers; Based on the second convolutional network, feature extraction processing is performed on the preprocessed vehicle image to obtain a second processed image; wherein the preset second convolutional network includes at least one pooling layer and at least two convolutional layers; The welding spot image is obtained according to the first processed image and the second processed image.

5. The method according to claim 4, characterized in that Based on the first convolutional network, feature extraction processing is performed on the preprocessed vehicle image to obtain a first processed image, including: Performing dimensionality reduction processing on the preprocessed vehicle image through a first convolutional layer in the first convolutional network to obtain a first dimensionality reduction feature vector; The first reduced-dimensional feature vector is convolved by the second convolution layer in the preset first convolution network to obtain the first processed image.

6. The method according to claim 5, characterized in that Based on the second convolutional network, feature extraction processing is performed on the preprocessed vehicle image to obtain a second processed image, including: Performing dimensionality reduction processing on the preprocessed vehicle image through the third convolution layer in the second convolution network to obtain a second dimensionality reduction feature vector; The second reduced-dimensional feature vector is subjected to feature extraction processing by a self-correcting convolutional network in the preset second convolutional network to obtain the second processed image; wherein the self-correcting convolutional network includes at least one pooling layer and at least two convolutional layers.

7. The method according to claim 6, characterized in that The method further comprises: performing feature extraction processing on the second dimension reduction feature vector by using a self-correcting convolutional network in the second convolutional network to obtain the second processed image, including: Performing an average pooling operation on the second reduced-dimensionality feature vector through a pooling layer in the self-correcting convolutional network to obtain a first vector; Performing convolution processing on the second reduced-dimensionality feature vector through a fourth convolution layer in the self-correcting convolutional network to obtain a second vector; Performing convolution processing on the second dimension reduction feature vector through a fifth convolution layer in the self-correcting convolutional network to obtain a third vector; The second processed image is obtained according to the first vector, the second vector, and the third vector.

8. The method according to claim 7, characterized in that The method further comprises: performing an average pooling operation on the second dimension reduction feature vector through a pooling layer in the self-correcting convolutional network to obtain a first vector, including: Performing an average pooling operation on the second dimension reduction feature vector through a pooling layer in the self-correcting convolutional network to obtain an average pooling vector; The average pooling vector is convolved through the sixth convolutional layer in the self-correcting convolutional network to obtain a first vector.

9. The method according to claim 8, characterized in that The average pooling vector is convolved by the sixth convolutional layer in the self-correcting convolutional network to obtain a first vector, including: Performing a convolution operation on the average pooling vector according to a preset convolution kernel size of the sixth convolution layer in the self-correcting convolutional network to obtain a feature vector extracted after the convolution operation; The feature vector extracted after the convolution operation is upsampled by a preset bilinear interpolation algorithm to obtain the first vector.

10. The method according to claim 7, characterized in that Obtaining the second processed image according to the first vector, the second vector, and the third vector includes: Performing a summation process on the first vector and the second vector to obtain a summed feature vector; Multiplying the summed feature vector with the third vector to obtain a multiplied feature vector; A convolution operation is performed on the feature vector after the multiplication process to obtain the second processed image.

11. The method according to any one of claims 1 to 10, characterized in that The preset welding spot shape is circular; According to a preset weld shape, performing weld detection processing on the weld image to obtain position information of the weld in the weld image includes: The preset Hough circle detection algorithm is used to perform positioning processing of the preset solder joint shape on the solder joint image to obtain position information of the solder joint in the solder joint image.

12. A positioning device for a vehicle welding point, characterized in that: include: An image acquisition unit, used to acquire a vehicle image; wherein the vehicle image is an image of a body-in-white containing a plurality of welding spots; A coarse positioning unit, used to perform feature extraction processing on the vehicle image according to a preset weld detection model to obtain a weld image corresponding to the vehicle image; wherein the preset weld detection model is a neural network model, used to locate welds in the vehicle image, and the weld image is used to characterize the welds in the vehicle image; The fine positioning unit is used to perform welding spot detection processing on the welding spot image according to a preset welding spot shape to obtain the position information of the welding spot in the welding spot image.

13. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1-11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 11 when executed by a processor.

15. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 11 when being executed by a processor.

Citation Information

Patent Citations

  • Convolutional neural network training method and welding spot defect detection method and device

    CN110930347A

  • License plate character segmentation method and device and storage medium

    CN114120306A

  • Scene text detection method, system and equipment based on deep convolutional neural network

    CN114724155A

  • Processing defect detection method of semiconductor device

    CN115018833A