Identification method and device for welding state of welding spot, storage medium and processor

Abstract

The invention discloses an identification method and device for the welding state of a welding spot, a storage medium and a processor. The method comprises the steps that in the process of welding a welding spot in a vehicle, multi-source data are collected, and the multi-source data are at least used for representing welding information of the welding spot; performing feature extraction on the multi-source data to obtain multiple pieces of feature data corresponding to the multi-source data; performing fusion processing on the plurality of feature data to obtain fused feature data; and the fusion feature data is recognized, a recognition result is obtained, and the recognition result is used for representing the welding state of the welding spot. The technical problem that the welding state of the welding spot cannot be accurately recognized is solved.

Description

Technical Field

[0001] The present application relates to the field of vehicles, and in particular, to a method, device, storage medium, and processor for identifying the welding status of a welding point. Background Art

[0002] Resistance spot welding (RSW) is widely used in vehicle manufacturing due to its high efficiency and low cost. This is particularly true during vehicle body assembly, where thousands of welds are required to ensure structural strength and safety. However, due to the diverse process parameters and complex welding environment, quality control of these welds presents a significant challenge.

[0003] In related technologies, the detection and evaluation of solder joint quality mainly rely on manual visual inspection or an automatic detection system based on fixed rules. This method has the technical problem of being unable to accurately identify the welding status of the solder joint.

[0004] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0005] The embodiments of the present application provide a method, device, storage medium, and processor for identifying the welding status of a weld point, so as to at least solve the technical problem of being unable to accurately identify the welding status of a weld point.

[0006] According to one aspect of an embodiment of the present application, a method for identifying the welding status of a weld point is provided. The method may include: collecting multi-source data during welding of weld points in a vehicle, wherein the multi-source data is used at least to characterize welding information of the weld points; performing feature extraction on the multi-source data to obtain multiple feature data corresponding to the multi-source data; performing fusion processing on the multiple feature data to obtain fused feature data; and identifying the fused feature data to obtain an identification result, wherein the identification result is used to characterize the welding status of the weld point.

[0007] Optionally, the multi-source data includes at least: image data containing the welds, and time series data corresponding to the welds, wherein the image data is used to at least characterize the welding positions of the welds, and the time series data is used to at least characterize the dynamic characteristics of the welds.

[0008] Optionally, feature extraction is performed on multi-source data respectively to obtain multiple feature data corresponding to the multi-source data, including: using a weld defect classification model to extract features from image data to obtain feature data corresponding to the image data, wherein the weld defect classification model is trained based on historical image data, and the historical image data is used to characterize the welding status of historical abnormal welds in vehicles in abnormal welding states; using a multi-channel time series model to extract features from time series data to obtain feature data corresponding to the time series data, wherein the multi-channel time series model is trained based on historical time series data, and the historical time series data is used to characterize the dynamic characteristics of historical abnormal welds.

[0009] Optionally, multiple feature data are fused to obtain fused feature data, including: retrieving weight data corresponding to different types of feature data to obtain weight data corresponding to the feature data; and fusing multiple feature data according to the weight data to obtain fused feature data.

[0010] Optionally, the method may also include: obtaining historical multi-source data, and historical standard recognition results corresponding to the historical multi-source data, wherein the historical multi-source data is at least used to characterize the dynamic characteristics of historical abnormal welds in the vehicle whose welding state is an abnormal welding state; performing feature extraction on the historical multi-source data to obtain multiple historical feature data corresponding to the historical multi-source data; determining the historical recognition results corresponding to the historical feature data; based on the historical recognition results and the historical standard recognition results, determining the weight data corresponding to different types of historical feature data; based on the weight data corresponding to the historical feature data, retrieving the weight data corresponding to different types of feature data.

[0011] Optionally, based on the weight data corresponding to the historical feature data, weight data corresponding to different types of feature data are retrieved, including: in response to the type of the feature data and the type of the historical feature data being the same, the weight data corresponding to the historical feature data is determined as the weight data corresponding to the feature data.

[0012] According to another aspect of an embodiment of the present application, a device for identifying the welding status of a welding point is also provided. The device may include: an acquisition unit, configured to acquire multi-source data during welding of welding points in a vehicle, wherein the multi-source data is at least used to characterize the welding information of the welding points; an extraction unit, configured to perform feature extraction on the multi-source data respectively to obtain a plurality of feature data corresponding to the multi-source data; a fusion unit, configured to perform fusion processing on the plurality of feature data to obtain fused feature data; and an identification unit, configured to identify the fused feature data to obtain an identification result, wherein the identification result is used to characterize the welding status of the welding point.

[0013] According to another aspect of an embodiment of the present application, a computer-readable storage medium is also provided, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the method for identifying the welding status of the weld point of the embodiment of the present application.

[0014] According to another aspect of an embodiment of the present application, a processor is further provided, which is used to run a program, wherein when the program is run by the processor, the method for identifying the welding state of a weld point according to an embodiment of the present application is executed.

[0015] According to another aspect of an embodiment of the present application, a program product is further provided. The program product includes computer instructions, wherein when the computer instructions are executed by a processor, the method for identifying the welding state of a welding point according to an embodiment of the present application is implemented.

[0016] In an embodiment of the present application, during the welding process of a weld point in a vehicle, multi-source data is collected, wherein the multi-source data is used to at least characterize welding information of the weld point; feature extraction is performed on the multi-source data to obtain multiple feature data corresponding to the multi-source data; the multiple feature data are fused to obtain fused feature data; and the fused feature data is identified to obtain a recognition result, wherein the recognition result is used to characterize the welding state of the weld point. That is, in an embodiment of the present application, multi-source data of the vehicle during the welding process is acquired, wherein the multi-source data includes different types of vehicle-related data; feature extraction is performed on the multi-source data to obtain multiple feature data corresponding to the multi-source data; the multiple feature data are fused to obtain fused feature data; and the fused feature data is identified to obtain a recognition result. Based on the recognition result, the welding state of the weld point can be determined, thereby achieving the technical effect of accurately identifying the welding state of the weld point and solving the technical problem of being unable to accurately identify the welding state of the weld point. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 is a flow chart of a method for identifying the welding state of a welding point according to an embodiment of the present application;

[0019] Figure 2 This is a flow chart of a comprehensive evaluation method for vehicle body weld quality based on multi-source heterogeneous information fusion according to an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of a device for identifying the welding status of a welding point according to an embodiment of the present application;

[0021] Figure 4 is a structural block diagram of a computer terminal according to an embodiment of the present application;

[0022] Figure 5 The figure is a block diagram of an electronic device for a method for identifying the welding status of a welding point according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] According to an embodiment of the present application, an embodiment of a method for identifying the welding status of a weld point is provided. The steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0026] In this embodiment, a method for identifying the welding status of a welding spot is proposed. The method obtains multi-source data of a vehicle during a welding process, where the multi-source data includes different types of vehicle-related data. Feature extraction is performed on the multi-source data to obtain multiple feature data corresponding to the multi-source data. The multiple feature data are fused to obtain fused feature data. The fused feature data is identified to obtain an identification result. The welding status of the welding spot can be determined based on the identification result, thereby achieving a technical effect of accurately identifying the welding status of the welding spot and solving the technical problem of being unable to accurately identify the welding status of the welding spot.

[0027] Figure 1 FIG. 1 is a flow chart of a method for identifying the welding state of a welding spot according to an embodiment of the present application. Figure 1 As shown, the method may include the following steps:

[0028] Step S102 : collecting multi-source data during welding of welding points in a vehicle, wherein the multi-source data is at least used to represent welding information of the welding points.

[0029] In the technical solution provided in step S102 of the present application, the aforementioned welds may be welds in a vehicle, welds for components welded in a vehicle, or electric welds in a vehicle, for example, welds on a vehicle body. The aforementioned multi-source data may be multimodal data, including tabular data, image data, signal data, and point cloud data, and may also be referred to as weld multi-source heterogeneous data. The aforementioned multi-source data may include at least time series data and image data, and may be data collected by various sensors, such as current time series data collected by a current sensor, voltage time series data collected by a voltage sensor, image data collected by a camera, and image data monitored by an infrared sensor. The aforementioned welding information may include at least information such as the welding status, welding position, and dynamic characteristics of the weld. It should be noted that this is merely an example, and no specific limitations are placed on the content of the time series data, the representation of the welding information, or the collection method.

[0030] Optionally, the aforementioned time series data can be an intrinsic process signal, which can be used to determine the dynamic resistance, motor displacement, current and voltage corresponding to the weld, and can be used to characterize the welding state of the target weld in the vehicle during the welding process. It can be data generated by the target weld during the welding process, or it can be parameters that change over time during the welding process, such as welding current, voltage, welding pressure, and weld temperature. It should be noted that this is only an example, and there is no specific limitation on the content of the time series data. The aforementioned image data can be an auxiliary measurement signal, which can be an image of the weld appearance captured by an image acquisition device such as a camera, infrared detection equipment, or visual monitoring equipment. It can be a single static photo or continuous video frames, for example, a high-definition image of a weld showing cracks or pores on the weld surface.

[0031] Optionally, during the welding process of a weld in a vehicle, multi-source data about the weld can be collected in real time or from historical records. This multi-source data can include information reflecting the welding status of the weld, such as parameters such as welding current, voltage, weld temperature, welding time, and welding pressure, as well as possible visual image data, such as an image or video recording of the weld.

[0032] For example, when welding a vehicle body weld, the system can record the welding current changes, weld spot images, welding area temperature and sound spectrum during the welding process in real time. These data together constitute a multi-source data set.

[0033] Step S104 : extracting features from the multi-source data respectively to obtain a plurality of feature data corresponding to the multi-source data.

[0034] In the technical solution provided in step S104 above, different neural network models can be used to extract features from corresponding data in the multi-source data to obtain multiple feature data corresponding to the multi-source data. The feature data can be evidence bodies and can include data such as current features, voltage features, welding time features, and welding pressure features corresponding to time series data, as well as data such as size features, shape features, color features, and texture features corresponding to image data. It should be noted that this is merely an example and does not impose any specific restrictions on the type of feature data.

[0035] Optionally, a current sensor can be used to collect current time series data, a voltage sensor can be used to collect voltage time series data, a displacement sensor can be used to collect time series data on solder joint position changes, and a visual sensor can be used to collect image data corresponding to the solder joint. It should be noted that the multi-source data collection method here is only an example and is not a specific limitation.

[0036] For example, time series data and solder joint image data can be collected, and a solder joint defect classification model can be used to extract features from the solder joint image data to obtain feature data corresponding to the solder joint image data. Simultaneously, a multi-channel time series data model can be used to extract features from the time series data to obtain feature data corresponding to the time series data. The solder joint defect classification model can be a lightweight neural network model pre-constructed based on historical solder joint image data. The multi-channel time series data model can be a neural network model pre-constructed based on historical time series data. It should be noted that this is merely an example, and no specific limitation is placed on the feature extraction model.

[0037] Step S106: performing fusion processing on the multiple feature data to obtain fused feature data.

[0038] In the technical solution provided in step S106 of the present application, multiple feature data may be fused to obtain fused feature data. The fused feature data may be feature-layer fused feature data, or information obtained by comprehensively processing multiple feature data from different sensors or data sources.

[0039] Optionally, multiple feature data are fused by linear fusion, nonlinear fusion, decision-level fusion, or weighted summation to obtain fused features. It should be noted that this is only an example and does not specifically limit the fusion method.

[0040] For example, suppose the quality of a weld in a vehicle is analyzed, and the current features corresponding to the current time series data, the temperature features corresponding to the temperature data, and the image features corresponding to the image data are obtained from multiple source data. The above features are fused using a deep learning model (such as a neural network that fuses multimodal information). The deep learning model can first map these features into a unified feature space, and then capture the relationship between the features through a series of nonlinear transformations (such as using convolutional layers, fully connected layers, and activation functions), and finally generate a fused feature vector. For example, the fused feature data may include the comprehensive energy input of the weld (taking into account the current and temperature features), the appearance quality score of the weld (based on image features), the acoustic stability index of the weld (from audio features), etc. The fused feature vector obtained after fusion contains key information about the weld status and can serve as the basis for subsequent intelligent analysis or classification. It should be noted that the above fusion method is only an example and is not specifically limited here. As long as the method of fusing multiple feature data to obtain fused feature data is within the scope of protection of this application, it should be included.

[0041] In this embodiment, different types of information are extracted through different neural networks, and the extracted feature information is then spliced ​​and fused at the feature layer. Finally, the fused feature information is taken as a whole to obtain the recognition result of the weld point, which can be used to determine the welding status of the weld point.

[0042] Step S108 , identifying the fused feature data to obtain an identification result, wherein the identification result is used to characterize the welding state of the welding point.

[0043] In the technical solution provided in step S108 of this application, a recognition model can be used to identify the fused features to obtain a recognition result. This recognition result can be text information, image information, or digital information, and can be used to characterize the welding state of the weld. It should be noted that this is merely an example, and the form of the recognition result is not specifically limited. The above-mentioned welding state can include a normal welding state and an abnormal welding state.

[0044] Optionally, the welding state of the weld can be determined based on the recognition result. If the welding state is abnormal, the abnormal welding type corresponding to the welding state can be determined based on the recognition result. The abnormal welding type can include abnormal types such as cold welds and weld penetration. It should be noted that this is for illustrative purposes only and does not specifically limit the abnormal type.

[0045] In this embodiment, different neural networks are used to extract features from different types of data in multi-source data to obtain multiple feature data. The extracted feature information is then automatically weighted and fused by the multimodal features at the feature layer using an attention mechanism during neural network training to obtain fused feature data. The fused feature data can be used as a whole to perform recognition on the fused feature data to obtain a recognition result.

[0046] For example, we can obtain detail feature maps and semantic feature maps at different stages, establish an association model between any point on the detail feature map and the semantic feature map, and aggregate the features on the semantic feature map under the guidance of the association model as a supplement to the feature point on the detail feature map, thereby fusing the information on the semantic feature map to the detail feature map, and further use the same operation to fuse the information on the detail feature map to the semantic feature map, so as to achieve mutual fusion of feature maps from different stages. At the same time, combined with the long short-term memory network (LSTM) and the squeeze-and-excitation (SE) attention mechanism (such as TTAO-SE), the mutual attention mechanism can effectively associate long-term and short-term features in time series data, and realize efficient interaction of cross-level features.

[0047] In this embodiment, the Dempster-Shafer evidence theory (DS) can also be used to combine rule decision fusion processes. In an intelligent welding quality detection system, the above method can be used to integrate the evaluation results of different sensors on the weld state to obtain a more robust and reliable conclusion.

[0048] Optionally, fusing multiple feature data sets through the DS can include the following process: Based on the data sources collected by each sensor, the solder joint state is evaluated to generate a basic probability assignment (BPA) corresponding to each data source. Each BPA is a confidence distribution assigned to multiple possible states. It is assumed that the information provided by different data sources is independent, meaning that the evaluation of one source does not directly affect the evaluation of another. The Dempster combination rule is used to calculate the combined BPA results of multiple data sources. The combination rule involves mathematically synthesizing the support levels of different evidence sources. Its core is to calculate the joint confidence level of the intersection and non-intersection of the evidence. Furthermore, conflicting evidence may exist between different data sources. Therefore, the conflict coefficient in the combination rule can be used to quantify the degree of conflict between the evidence. If the conflict is severe, additional processing or weighting adjustment is required. Furthermore, the fused BPA is converted into a confidence function and a likelihood function. The most likely solder joint state is selected based on the maximum confidence level, or a threshold is set to determine whether there is sufficient evidence to support a particular state.

[0049] For example, suppose the collected multi-source data includes two data sources: image data from a visual sensor and sound time-series data from an acoustic sensor. The visual sensor assigns a confidence level of 0.9 to a "normal solder joint," 0.05 to a "weak solder joint," and 0.05 to other values. The acoustic sensor assigns a confidence level of 0.8 to a "normal solder joint," 0.1 to an "overheated solder joint," and 0.1 to other values. Decision fusion can be performed using the DS combination rule: visual sensor: \(\text{"normal solder joint"}: 0.9, \text{"weak solder joint"}: 0.05, \text{"overheated solder joint"}: 0.05\); acoustic sensor: \(\text{"normal solder joint"}: 0.8, \text{"overheated solder joint"}: 0.1, \text{"weak solder joint"}: 0.1\). The combination rule can be applied to calculate the combined BPA of the two sensor assessment results, taking into account possible conflicting evidence. The combined BPA is converted into a confidence function, which is then used to make the final solder joint status judgment. For example, if the combined confidence level is: "normal welding" 0.85, "cold welding" 0.1, and "overheating" 0.05, then the system will judge the solder joint status as "normal welding".

[0050] Optionally, the Dempster-Shafer combined rule decision fusion can effectively integrate the feature information extracted from multi-source data, and can make relatively reasonable decisions even in the face of highly conflicting, uncertain or fuzzy data, thereby improving the judgment accuracy and robustness of the intelligent welding detection system.

[0051] Through the above-mentioned steps S102 and S108 of the present application, during the process of welding a weld point in a vehicle, multi-source data is collected, wherein the multi-source data is used to at least characterize the welding information of the weld point; feature extraction is performed on the multi-source data to obtain multiple feature data corresponding to the multi-source data; the multiple feature data are fused to obtain fused feature data; and the fused feature data is identified to obtain a recognition result, wherein the recognition result is used to characterize the welding state of the weld point. In other words, in an embodiment of the present application, multi-source data of the vehicle during the welding process is obtained, wherein the multi-source data includes different types of vehicle-related data; feature extraction is performed on the multi-source data to obtain multiple feature data corresponding to the multi-source data; the multiple feature data are fused to obtain fused feature data; and the fused feature data is identified to obtain a recognition result. Based on the recognition result, the welding state of the weld point can be determined, thereby achieving the technical effect of accurately identifying the welding state of the weld point and solving the technical problem of being unable to accurately identify the welding state of the weld point.

[0052] The above method of this embodiment is further introduced below.

[0053] As an optional implementation, the multi-source data includes at least: image data of the welds, and time series data corresponding to the welds, the image data is used to at least characterize the welding positions of the welds, and the time series data is used to at least characterize the dynamic characteristics of the welds.

[0054] In this embodiment, the multi-source data may include at least image data of welds and time series data corresponding to the welds. The image data may be captured by a device such as a laser radar or camera and can be used to determine the location and weld morphology of the welds. The time series data can at least be used to characterize the dynamic characteristics of the welds. It should be noted that this is merely an example and does not impose any specific limitations on the content of the multi-source data.

[0055] As an optional implementation, feature extraction is performed on multi-source data respectively to obtain multiple feature data corresponding to the multi-source data, including: using a weld defect classification model to extract features from image data to obtain feature data corresponding to the image data, wherein the weld defect classification model is trained based on historical image data, and the historical image data is used to characterize the welding status of historical abnormal welds in vehicles in abnormal welding states; using a multi-channel time series model to extract features from time series data to obtain feature data corresponding to the time series data, wherein the multi-channel time series model is trained based on historical time series data, and the historical time series data is used to characterize the dynamic characteristics of historical abnormal welds.

[0056] In this embodiment, the solder joint defect classification model can be a model trained based on historical image data, and can be used to extract features from the image data to obtain feature data corresponding to the image data. The multi-channel time series model can be trained based on historical time series data, and can be used to extract features from the time series data to obtain feature data corresponding to the time series data.

[0057] Alternatively, a camera or visual sensor can be used to capture static images of the weld to obtain image information. Through image processing and analysis, key features related to the weld's appearance quality, such as its size, shape, color, and texture, can be extracted. Simultaneously, time series data can be collected, such as the changes in physical parameters such as current, voltage, and temperature during the welding process. This data contains rich information about the dynamic state of the weld, such as the peak current at the start and end of welding and the temperature trend throughout the welding process.

[0058] Optionally, a deep learning model (i.e., a weld defect classification model) can be pre-trained based on historical image data. This historical image data can include images of welds with various abnormal welding conditions that occur during vehicle manufacturing. By studying a large number of historical images of abnormal welds, the deep learning model learns the visual features that distinguish normal welds from historically abnormal welds. Furthermore, a multi-channel time series model can be pre-trained based on historical time series data. This historical time series data can be used to reflect the dynamic characteristics of various historically abnormal welds during the welding process, such as how current changes over time and the rates of temperature rise and fall.

[0059] Furthermore, after collecting image data, the current solder joint image can be input into a trained solder joint defect classification model. The solder joint defect classification model can perform multi-level feature extraction on the image, including edges, texture, and color at the bottom layer, and solder joint shape, size, and position at the top layer. Through this process, a set of feature data describing the appearance characteristics of the solder joint can be obtained. At the same time, the time series data collected during the welding process can be input into a multi-channel time series model. The multi-channel time series model can extract key features reflecting the dynamic characteristics of the solder joint by processing different channels in the time series signal (such as current, voltage, and temperature channels). For example, the multi-channel time series model can determine the current change trend, the time point when the temperature reaches its peak, the voltage fluctuation range, etc. These feature data can reveal abnormal behavior in the welding process. The obtained image feature data and time series feature data can be sent to the multi-source information fusion module, which can use Dempster-Shafer evidence theory, deep learning and other methods to comprehensively analyze these data to obtain fused feature data. The fused feature data can be input into the anomaly detection model to identify whether there are welding defects in the solder joints and further classify and determine the type of defects, such as cold solder joints, overheating, cracks, etc.

[0060] Optionally, through multi-source data feature extraction and fusion, the intelligent welding detection system can more comprehensively and accurately evaluate the quality of welds, thereby improving the reliability and effectiveness of detection, especially in complex and changeable vehicle manufacturing environments, achieving the technical effect of accurately identifying the welding status of welds, and solving the technical problem of being unable to accurately identify the welding status of welds.

[0061] As an optional implementation, step S106, fusing multiple feature data to obtain fused feature data, includes: retrieving weight data corresponding to different types of feature data to obtain weight data corresponding to the feature data; fusing multiple feature data according to the weight data to obtain fused feature data.

[0062] In this embodiment, the multi-source data may include different types of data, and different types of neural network models are used to extract different types of data to obtain different types of feature data. Since different types of feature data have different effects on the recognition results, in order to improve the accuracy of the recognition results, the weight data corresponding to different types of feature data can be determined in advance. After extracting features from the multi-source data and obtaining multiple feature data, the weight data corresponding to different types of feature data are determined. The weight data can be numbers such as 0.1 and 0.2, which can be used to characterize the importance of the feature data. It should be noted that this is only an example, and there is no specific restriction on the size of the weight data.

[0063] Optionally, after obtaining weight data corresponding to a plurality of feature data, weighted processing may be performed on the plurality of feature data according to the weight data to obtain fused feature data.

[0064] In this embodiment, these multimodal features are fused in the middle layer of the model, so that the correlation between the modalities can be utilized in the feature extraction stage to improve the expressiveness of the features.

[0065] Optionally, in multi-source data fusion, different types of feature data (such as image features and timing signal features) may contribute differently to determining solder joint quality. Therefore, the system needs to assign weights to each type of feature data, reflecting their importance in the final decision. These weights can be learned through algorithmic learning during the model training phase or set by domain experts based on experience. During the actual inspection process, the system will call on these pre-calculated weights to appropriately weight different feature data.

[0066] Optionally, before fusing multiple feature data, the weight parameters corresponding to each feature data can be determined, and the multiple feature data can be combined according to their respective weights. For example, each feature data is multiplied by its corresponding weight, and then the multiple weighted feature data are added together, and finally divided by the sum of the weights. Alternatively, the multiple feature data can be weighted and summed, and then the result of the summation can be converted into a probability distribution through a normalization technique (such as the softmax function) to ensure that the output fused feature data is numerically comparable and interpretable. After weighted fusion processing, a set of fused feature data can be obtained. This set of data integrates information from multiple types of features and more comprehensively reflects the quality status of the solder joint. The fused feature data can be used as input for subsequent decision analysis for anomaly detection or classification assessment of solder joint quality.

[0067] For example, assume that in an intelligent welding inspection system, there are two types of feature data: image feature data, with a weight of 0.6, describing the appearance of the weld; and timing signal feature data, with a weight of 0.4, reflecting changes in current, voltage, and other parameters during the welding process. To fuse the image and timing signal feature data, the following steps can be followed: First, extract feature data from the image and timing signal data, respectively. Then, using pre-set or calculated weights, with a weight of 0.6 for the image feature data and 0.4 for the timing signal feature data, multiply the image feature data by 0.6 and the timing signal feature data by 0.4, and then add them together to generate the fused feature data. Finally, the fused feature data can be input into a decision model to determine the presence of defects in the weld, as well as the type and severity of the defect.

[0068] In this embodiment, through weight allocation and weighted fusion, the system can reasonably integrate information according to the actual contribution of different feature data, thereby improving the accuracy and reliability of the detection results.

[0069] As an optional implementation, the method may also include: obtaining historical multi-source data, and historical standard identification results corresponding to the historical multi-source data, wherein the historical multi-source data is at least used to characterize the dynamic characteristics of historical abnormal welds in the vehicle whose welding state is an abnormal welding state; performing feature extraction on the historical multi-source data to obtain multiple historical feature data corresponding to the historical multi-source data; determining the historical identification results corresponding to the historical feature data; based on the historical identification results and the historical standard identification results, determining the weight data corresponding to different types of historical feature data; based on the weight data corresponding to the historical feature data, retrieving the weight data corresponding to different types of feature data.

[0070] In this embodiment, weight data corresponding to different types of historical feature data may be determined in advance based on historical multi-source data, and based on the weight data, weight data corresponding to the feature data may be determined.

[0071] Optionally, determining different types of historical feature data based on historical multi-source data may include the following steps: obtaining the historical multi-source data and historical standard recognition results corresponding to the historical multi-source data, wherein the historical standard recognition results can be used to characterize the welding conditions corresponding to historical welds in the historical multi-source data and can be used to determine the welding conditions of the welds in the historical multi-source data. Feature extraction can be performed on the historical multi-source data to obtain multiple historical feature data corresponding to the historical multi-source data. Historical recognition results corresponding to the historical feature data can be determined. Based on the differences between the historical recognition results and the historical standard recognition results, weight data corresponding to different types of historical feature data can be determined. For example, it can be set in advance that when the historical multi-source data includes image feature data and audio feature data, the weight data corresponding to the image feature data is 0.5, and the weight corresponding to the audio feature data is 0.5. Based on the weight data, the historical recognition results corresponding to the historical multi-source data are determined. If the historical recognition results are significantly different from the historical standard recognition results, the weight data corresponding to the image feature data and / or the weight data corresponding to the audio feature data can be adjusted until the similarity between the obtained historical recognition results and the historical standard recognition results is greater than the similarity threshold. In the above manner, the weight data corresponding to different types of feature data can be determined in different multi-source data cases.

[0072] Optionally, the weight data corresponding to the feature data of each dimension may also be determined through a neural network model.

[0073] As an optional implementation, based on the weight data corresponding to the historical feature data, the weight data corresponding to different types of feature data are retrieved, including: in response to the type of the feature data and the type of the historical feature data being the same, the weight data corresponding to the historical feature data is determined as the weight data corresponding to the feature data.

[0074] In this embodiment, if the historical multi-source data and the multi-source data contain the same data type, the weight data corresponding to the feature data in the historical multi-source data can be determined as the weight data corresponding to the feature data of that type in the multi-source data. In this way, the weight data corresponding to the feature data of different types of data in the multi-source data can be determined.

[0075] Figure 2 This is a flow chart of a comprehensive evaluation method for the quality of body welds based on multi-source heterogeneous information fusion according to an embodiment of the present application. Figure 2 As shown, the method may include the following steps:

[0076] Step S202 , collecting multi-source data of the welding point during the welding process.

[0077] In this embodiment, during the spot welding process in the vehicle, different types of sensors may be used to collect multi-source data.

[0078] Step S204 : Processing the solder joint image data in the multi-source data using the solder joint defect classification model.

[0079] In this embodiment, the weld defect classification model may be used to perform feature extraction on the weld image data in the multi-source data to obtain a first body of evidence, which may be feature data corresponding to the weld image data.

[0080] Step S206: Process the time series data in the multi-source data using the multi-channel time series data model.

[0081] In this embodiment, a multi-channel time series data model can be used to process the time series data in the multi-source data to obtain a second body of evidence. The second body of evidence can be feature data corresponding to the time series data.

[0082] Step S208 , performing feature fusion on the feature data corresponding to the time series data and the feature data corresponding to the solder joint image data.

[0083] In this embodiment, the feature data corresponding to the time series data and the feature data corresponding to the solder joint image data may be fused by combining rule decisions to obtain fused feature data.

[0084] Step S210: Identify the fused feature data.

[0085] In this embodiment, the fused feature data may be identified to obtain an identification result, which may be used to determine the welding quality of the welding spot.

[0086] In an embodiment of the present application, multi-source data of a vehicle during a welding process is obtained, the multi-source data including different types of vehicle-related data, feature extraction is performed on the multi-source data respectively to obtain multiple feature data corresponding to the multi-source data, the multiple feature data are fused to obtain fused feature data, the fused feature data is identified to obtain an identification result, and the welding state of the weld point can be determined based on the identification result, thereby achieving the technical effect of accurately identifying the welding state of the weld point and solving the technical problem of being unable to accurately identify the welding state of the weld point.

[0087] According to an embodiment of the present application, a device for identifying the welding state of a welding spot is also provided. It should be noted that the device for identifying the welding state of a welding spot of this embodiment can be used to execute the method for identifying the welding state of a welding spot of the embodiment of the present application.

[0088] Figure 3 FIG. 1 is a schematic diagram of a device for identifying the welding state of a welding spot according to an embodiment of the present application. Figure 3 As shown, the device 30 for identifying the welding state of a welding spot may include: a collection unit 302 , an extraction unit 304 , a fusion unit 306 and an identification unit 308 .

[0089] The collecting unit 302 is used to collect multi-source data during the process of welding the welding points in the vehicle, wherein the multi-source data is used to at least represent the welding information of the welding points.

[0090] The extraction unit 304 is configured to perform feature extraction on the multi-source data to obtain a plurality of feature data corresponding to the multi-source data.

[0091] The fusion unit 306 is used to perform fusion processing on multiple feature data to obtain fused feature data.

[0092] The recognition unit 308 is used to recognize the fused feature data to obtain a recognition result, wherein the recognition result is used to characterize the welding state of the welding point.

[0093] The device for identifying the welding status of welding points of this embodiment collects multi-source data through an acquisition unit during the process of welding welding points in a vehicle, wherein the multi-source data is at least used to characterize the welding information of the welding points; performs feature extraction on the multi-source data through an extraction unit to obtain multiple feature data corresponding to the multi-source data; performs fusion processing on the multiple feature data through a fusion unit to obtain fused feature data; and identifies the fused feature data through an identification unit to obtain an identification result, wherein the identification result is used to characterize the welding status of the welding points, thereby achieving the technical effect of accurately identifying the welding status of the welding points and solving the technical problem of being unable to accurately identify the welding status of the welding points.

[0094] The embodiment of the present application can provide a computer terminal, which can be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal can also be replaced by a terminal device such as a mobile terminal.

[0095] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of a computer network.

[0096] In this embodiment, the computer terminal can execute the program code of the following steps in the method for identifying the welding status of a welding spot: in the process of welding the welding spots in the vehicle, collecting multi-source data, wherein the multi-source data is at least used to characterize the welding information of the welding spot; performing feature extraction on the multi-source data respectively to obtain multiple feature data corresponding to the multi-source data; performing fusion processing on the multiple feature data to obtain fused feature data; and identifying the fused feature data to obtain an identification result, wherein the identification result is used to characterize the welding status of the welding spot.

[0097] Optionally, Figure 4 is a structural block diagram of a computer terminal according to an embodiment of the present application, such as Figure 4 As shown, the computer terminal 408 may include: one or more (only one is shown in the figure) processors 402 , a memory 404 and a transmission device 406 .

[0098] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the method and device for identifying the welding state of a solder joint in the embodiments of the present application. The processor executes the software programs and modules stored in the memory to perform various functional applications and data processing, thereby implementing the above-mentioned method for identifying the welding state of a solder joint. The memory can include high-speed random access memory and can also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory can further include memory remotely located relative to the processor, and these remote memories can be connected to the computer terminal 408 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0099] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: in the process of welding welding points in the vehicle, collect multi-source data, wherein the multi-source data is at least used to characterize the welding information of the welding points; perform feature extraction on the multi-source data respectively to obtain multiple feature data corresponding to the multi-source data; perform fusion processing on the multiple feature data to obtain fused feature data; and identify the fused feature data to obtain an identification result, wherein the identification result is used to characterize the welding status of the welding points.

[0100] It can be understood by those skilled in the art that Figure 4 The structure shown is for illustration only, and the computer terminal 408 may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID for short), a PAD, or other terminal devices. Figure 4 It does not limit the structure of the computer terminal 408. For example, the computer terminal 408 may also include Figure 4 More or fewer components (such as network interfaces, display devices, etc.) shown in, or with Figure 4 Different configurations shown.

[0101] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0102] According to an embodiment of the present application, a computer-readable storage medium is further provided. The computer-readable storage medium includes a stored program, wherein the program executes the method for identifying the welding state of a welding point in the embodiment.

[0103] Optionally, in this embodiment, the computer-readable storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0104] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: in the process of welding welding points in a vehicle, collecting multi-source data, wherein the multi-source data is at least used to characterize welding information of the welding points; performing feature extraction on the multi-source data respectively to obtain multiple feature data corresponding to the multi-source data; performing fusion processing on the multiple feature data to obtain fused feature data; and identifying the fused feature data to obtain an identification result, wherein the identification result is used to characterize the welding state of the welding points.

[0105] Optionally, the computer-readable storage medium may also execute program code for the following steps: performing feature extraction on image data using a weld defect classification model to obtain feature data corresponding to the image data, wherein the weld defect classification model is trained based on historical image data, and the historical image data is used to characterize the welding status of historical abnormal welds in vehicles in abnormal welding states; performing feature extraction on time series data using a multi-channel time series model to obtain feature data corresponding to the time series data, wherein the multi-channel time series model is trained based on historical time series data, and the historical time series data is used to characterize the dynamic characteristics of historical abnormal welds.

[0106] Optionally, the computer-readable storage medium may also execute program code for the following steps: retrieving weight data corresponding to different types of feature data to obtain weight data corresponding to the feature data; and fusing multiple feature data according to the weight data to obtain fused feature data.

[0107] Optionally, the computer-readable storage medium may also execute program code for the following steps: obtaining historical multi-source data and historical standard identification results corresponding to the historical multi-source data, wherein the historical multi-source data is at least used to characterize the dynamic characteristics of historical abnormal welds in a vehicle whose welding state is an abnormal welding state; performing feature extraction on the historical multi-source data to obtain a plurality of historical feature data corresponding to the historical multi-source data; determining the historical identification results corresponding to the historical feature data; determining weight data corresponding to different types of historical feature data based on the historical identification results and the historical standard identification results; and retrieving weight data corresponding to different types of feature data based on the weight data corresponding to the historical feature data.

[0108] Optionally, the computer-readable storage medium may further execute program code for the following steps: in response to the type of the feature data being the same as the type of the historical feature data, determining the weight data corresponding to the historical feature data as the weight data corresponding to the feature data.

[0109] In this embodiment, multi-source data of the vehicle during the welding process is obtained, and the multi-source data includes different types of vehicle-related data. Feature extraction is performed on the multi-source data to obtain multiple feature data corresponding to the multi-source data. The multiple feature data are fused to obtain fused feature data. The fused feature data is identified to obtain an identification result. The welding state of the weld point can be determined based on the identification result, thereby achieving the technical effect of accurately identifying the welding state of the weld point and solving the technical problem of being unable to accurately identify the welding state of the weld point.

[0110] According to an embodiment of the present application, a processor is further provided, which is used to run a program, wherein when the program is run by the processor, the method for identifying the welding state of the welding point in the embodiment is executed.

[0111] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of a computer network.

[0112] In this embodiment, the computer terminal can execute the program code of the following steps in the method for identifying the welding status of a welding spot: in the process of welding the welding spots in the vehicle, collecting multi-source data, wherein the multi-source data is at least used to characterize the welding information of the welding spot; performing feature extraction on the multi-source data respectively to obtain multiple feature data corresponding to the multi-source data; performing fusion processing on the multiple feature data to obtain fused feature data; and identifying the fused feature data to obtain an identification result, wherein the identification result is used to characterize the welding status of the welding spot.

[0113] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the method and device for identifying the welding state of a solder joint in the embodiments of the present application. The processor executes the software programs and modules stored in the memory to perform various functional applications and data processing, thereby implementing the above-mentioned method for identifying the welding state of a solder joint. The memory can include a high-speed random access memory and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory can further include a memory remotely located relative to the processor, and these remote memories can be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0114] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: in the process of welding welding points in the vehicle, collect multi-source data, wherein the multi-source data is at least used to characterize the welding information of the welding points; perform feature extraction on the multi-source data respectively to obtain multiple feature data corresponding to the multi-source data; perform fusion processing on the multiple feature data to obtain fused feature data; and identify the fused feature data to obtain an identification result, wherein the identification result is used to characterize the welding status of the welding points.

[0115] Optionally, the processor may also execute the program code of the following steps: performing feature extraction on the image data using a weld defect classification model to obtain feature data corresponding to the image data, wherein the weld defect classification model is trained based on historical image data, and the historical image data is used to characterize the welding status of historical abnormal welds in vehicles in abnormal welding states; performing feature extraction on the time series data using a multi-channel time series model to obtain feature data corresponding to the time series data, wherein the multi-channel time series model is trained based on historical time series data, and the historical time series data is used to characterize the dynamic characteristics of historical abnormal welds.

[0116] Optionally, the processor may also execute the following program code: retrieve weight data corresponding to different types of feature data to obtain weight data corresponding to the feature data; and fuse multiple feature data according to the weight data to obtain fused feature data.

[0117] Optionally, the processor may also execute the program code of the following steps: obtaining historical multi-source data and historical standard recognition results corresponding to the historical multi-source data, wherein the historical multi-source data is at least used to characterize the dynamic characteristics of historical abnormal welds in the vehicle whose welding state is an abnormal welding state; performing feature extraction on the historical multi-source data to obtain a plurality of historical feature data corresponding to the historical multi-source data; determining the historical recognition results corresponding to the historical feature data; determining the weight data corresponding to different types of historical feature data based on the historical recognition results and the historical standard recognition results; and retrieving the weight data corresponding to different types of feature data based on the weight data corresponding to the historical feature data.

[0118] Optionally, the processor may further execute program code of the following steps: in response to the type of the feature data being the same as the type of the historical feature data, determining the weight data corresponding to the historical feature data as the weight data corresponding to the feature data.

[0119] Optionally, the processor may further execute program code for the following steps: collecting noise data, wherein the noise data is used to characterize the characteristics of at least one abnormal weld point, where the abnormal weld point is an abnormal weld point whose welding state in the vehicle is an abnormal welding state; inputting the noise data into an image generation model to obtain a target abnormal weld point image, wherein the image generation model is used to generate an abnormal weld point image set based on the noise data, wherein the abnormal weld point image set includes at least one abnormal weld point image, and the abnormal weld point image includes at least one abnormal weld point; determining the similarity between the abnormal weld point image and a historical abnormal weld point image, wherein the historical abnormal weld point image includes at least one historical abnormal weld point, and the historical abnormal weld point is a historical abnormal weld point whose welding state in the vehicle was an abnormal welding state at a historical moment; and based on the similarity, screening out at least one target abnormal weld point image from the abnormal weld point image set.

[0120] By using an embodiment of the present application, multi-source data of a vehicle during a welding process is obtained, where the multi-source data includes different types of vehicle-related data. Feature extraction is performed on the multi-source data to obtain multiple feature data corresponding to the multi-source data. The multiple feature data are fused to obtain fused feature data. The fused feature data is identified to obtain an identification result. Based on the identification result, the welding state of the weld point can be determined, thereby achieving the technical effect of accurately identifying the welding state of the weld point and solving the technical problem of being unable to accurately identify the welding state of the weld point.

[0121] According to an embodiment of the present application, a computer program product is further provided. The computer program product includes computer instructions, wherein when the computer instructions are executed by a processor, the method for identifying the welding state of a welding point in the embodiment is implemented.

[0122] An embodiment of the present application may provide an electronic device, which may include a memory and a processor.

[0123] Figure 5 This is a block diagram of an electronic device for a method for identifying the welding state of a solder joint according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0124] like Figure 5As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the device 500 can also be stored in the RAM 503. The computing unit 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0125] Various components in device 500 are connected to I / O interface 505, including: an input unit 506, such as a keyboard, mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a magnetic disk, optical disk, etc.; and a communication unit 509, such as a network card, modem, wireless communication transceiver, etc. The communication unit 509 allows device 500 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0126] The computing unit 501 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the data verification method. For example, in some embodiments, the data verification method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the data verification method described above can be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to execute the data verification method in any other appropriate manner (for example, by means of firmware).

[0127] According to an embodiment of the present application, a method for identifying the welding status of a weld spot is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0128] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0129] The program code for implementing the methods of the present application can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow charts and / or block diagrams are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0130] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0131] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or an LCD (liquid crystal display, detector) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0132] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0133] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0134] The above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0135] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0136] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0137] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected to achieve the purpose of the present embodiment according to actual needs.

[0138] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0139] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a read-only memory, a random access memory, a mobile hard disk, a magnetic disk or an optical disk.

[0140] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for identifying the welding state of a welding spot, characterized in that: include: During welding of a welding spot in a vehicle, multi-source data is collected, wherein the multi-source data is at least used to characterize welding information of the welding spot; Performing feature extraction on the multi-source data respectively to obtain a plurality of feature data corresponding to the multi-source data; Performing fusion processing on the plurality of feature data to obtain fused feature data; The fused feature data is identified to obtain an identification result, wherein the identification result is used to characterize the welding state of the welding point.

2. The method according to claim 1, characterized in that The multi-source data includes at least: image data containing the solder joint, and time series data corresponding to the solder joint, wherein the image data is used to at least characterize the soldering position of the solder joint, and the time series data is used to at least characterize the dynamic characteristics of the solder joint.

3. The method according to claim 2, characterized in that The extracting features from the multi-source data to obtain a plurality of feature data corresponding to the multi-source data includes: performing feature extraction on the image data using a weld defect classification model to obtain feature data corresponding to the image data, wherein the weld defect classification model is trained based on historical image data, and the historical image data is used to characterize the welding status of historical abnormal welds in the vehicle in the abnormal welding state; A multi-channel time series model is used to extract features from the time series data to obtain feature data corresponding to the time series data, wherein the multi-channel time series model is trained based on historical time series data, and the historical time series data is used to characterize the dynamic characteristics of the historical abnormal solder joints.

4. The method according to claim 1, wherein The fusing the plurality of feature data to obtain fused feature data includes: Retrieving weight data corresponding to different types of feature data to obtain the weight data corresponding to the feature data; According to the weight data, a plurality of the feature data are fused to obtain the fused feature data.

5. The method according to claim 4, characterized in that The method further comprises: Acquiring historical multi-source data and historical standard recognition results corresponding to the historical multi-source data, wherein the historical multi-source data is at least used to characterize dynamic characteristics of historical abnormal welds in the vehicle where the welding state is an abnormal welding state; Performing feature extraction on the historical multi-source data to obtain a plurality of historical feature data corresponding to the historical multi-source data; Determining a historical recognition result corresponding to the historical feature data; Determining weight data corresponding to the different types of historical feature data based on the historical recognition result and the historical standard recognition result; Based on the weight data corresponding to the historical feature data, weight data corresponding to different types of feature data are retrieved.

6. The method according to claim 5, characterized in that The step of retrieving weight data corresponding to different types of feature data based on the weight data corresponding to the historical feature data includes: In response to the fact that the type of the feature data is the same as the type of the historical feature data, the weight data corresponding to the historical feature data is determined as the weight data corresponding to the feature data.

7. A device for identifying the welding state of a welding spot, characterized in that: include: A collection unit, configured to collect multi-source data during welding of a welding spot in a vehicle, wherein the multi-source data is at least used to characterize welding information of the welding spot; an extraction unit, configured to extract features from the multi-source data respectively to obtain a plurality of feature data corresponding to the multi-source data; a fusion unit, configured to fuse the plurality of feature data to obtain fused feature data; The recognition unit is used to recognize the fused feature data to obtain a recognition result, wherein the recognition result is used to characterize the welding state of the welding point.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 6.

9. A processor, characterized in that: The processor is configured to run a program, wherein the program, when run by the processor, executes the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions, which implement the method according to any one of claims 1 to 6 when executed by a processor.

Images