Melting welding training and assessment system based on AI multi-action recognition

Abstract

The present invention discloses a melting welding training and assessment system based on AI multi-action recognition, which relates to the field of welding assessment technology and includes an infrared positioning module, an AI multi-action recognition module, and a warning module. The infrared positioning module is used to locate a welding head by using infrared detection technology. The AI ​​multi-action recognition module is used to scan the welding head after locating the welding head. The warning module is used to judge the probability of normal welding of the welding head and to warn the assessor by sound. The infrared positioning module includes an infrared signal transmitter, a signal receiver, and a data processing unit. The infrared signal transmitter and the signal receiver are both electrically connected to the data processing unit. The infrared signal transmitter is used to transmit an infrared signal to an assessment area. The signal receiver is used to receive the infrared signal reflected by a target. The present invention has the characteristic of accurate judgment.

Description

Technical Field

[0001] The present invention relates to the technical field of welding assessment, and in particular to a melting welding training and assessment system based on AI multi-action recognition. Background Art

[0002] The AI ​​multi-action recognition system uses computer vision and machine learning technology to identify multiple actions during welding in real time. This system captures video data of the welding process through cameras or other sensors, and uses algorithms to analyze and classify each action of the welder to help evaluate and improve welding technology. It is used in the unified welding assessment of welders.

[0003] The stability of the welder's welding head movement during the welding process has always been one of the important assessment indicators, which will significantly affect the smoothness of the weld and the uniformity of the weldment of the finished welding product. During the welding process, a small rebound may occur when the welding head contacts the weld, which is usually caused by the heat-affected zone or the flexibility of the material. This small rebound movement may be misjudged as abnormal movement caused by incorrect force, resulting in erroneous deductions, but if no warning is given, it may cause serious problems for subsequent welding. Existing visual judgment methods can only make judgments immediately after identifying such movements, and the judgment is inaccurate. Therefore, it is very necessary to design a melting welding training and assessment system based on AI multi-action recognition with accurate judgment. Summary of the invention

[0004] The purpose of the present invention is to provide a melting welding training and assessment system based on AI multi-action recognition to solve the problems raised in the above-mentioned background technology.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: a melting welding training and assessment system based on AI multi-action recognition, comprising an infrared positioning module, an AI multi-action recognition module, and a warning module. The infrared positioning module is used to locate the welding head through infrared detection technology, the AI ​​multi-action recognition module is used to scan the welding head after locating it, and the warning module is used to judge the probability of normal welding of the welding head and to warn the assessor by sound.

[0006] According to the above technical solution, the infrared positioning module includes an infrared signal transmitter, a signal receiver, and a data processing unit. The infrared signal transmitter and the signal receiver are both electrically connected to the data processing unit. The infrared signal transmitter is used to transmit infrared signals to the assessment area. These signals will be reflected by the target welding head. The signal receiver is used to receive the infrared signal reflected by the target. The data processing unit is used to parse the received signal, determine the detected welding head reflection signal, extract the distance, height and azimuth position information of the welding head, and perform real-time positioning of the welding head.

[0007] The AI ​​multi-action recognition module includes a camera control unit, an image collector, a motion recognition algorithm module, and a data statistics module. The camera control unit is electrically connected to the image collector and the data processing unit. The motion recognition algorithm module is electrically connected to the data statistics module. The camera control unit is used to control the switch and rotation of the camera for scanning. The image collector is used to capture video image data in real time. The motion recognition algorithm module is used to analyze the scanned image using a deep learning method to identify the welding action and movement state of the welding head. The data statistics module is used to count the rebound amplitude and behavior pattern of the welding head on the weldment.

[0008] The warning module includes a warning strategy module, a sound transmitter, a probability calculation module, a sound direction control unit, and a sound warning unit. The probability calculation module is electrically connected to the data statistics module, the warning strategy module is electrically connected to the sound transmitter and the probability calculation module, the sound direction control unit is electrically connected to the sound transmitter, and the sound warning unit is electrically connected to the warning strategy module. The warning strategy module is used to take warning measures according to the position information and image information of the welding head. The sound transmitter is used to play the sound and turn to the target examiner to produce auditory discomfort. The probability calculation module is used to evaluate the probability of incorrect force application of the welding head according to the rebound amplitude of the welding head on the weldment and the frequency of each rebound in the assessment. The sound direction control unit is used to selectively control the sound transmitter to align with the direction of the examiner who has made an erroneous action. The sound warning unit is used to use sound warning when the rebound amplitude of the welding head on the weldment exceeds a set value.

[0009] According to the above technical solution, the working method of the system includes the following steps:

[0010] S1. Use infrared detection technology to determine the position of the welding head in the assessment range in real time, establish a spatial rectangular coordinate system, and use Indicates the position coordinates of the welding head;

[0011] S2. When the moving height of a welding head is lower than the critical height, the image collector is controlled to face the target welding head and scan, the scanned image is analyzed in real time, and the moving action of the welding head is identified;

[0012] S3. After the welding head contacts the weld and rebounds slightly, the welding head behavior is analyzed through the movement and position change of the welding head to determine whether it is normal welding or incorrect force application, and decide whether to issue a warning signal;

[0013] S4. Real-time statistics of the rebound amplitude of the welding head on the weldment, combined with the average number of small-amplitude rebound behaviors in each assessment, judge the probability of incorrect force application of the welding head. When the rebound amplitude exceeds the set value, the sound transmitter will start to warn the assessor in a directional manner;

[0014] S5. When the rebound amplitude of the welding head tested by multiple people exceeds the set value at the same time, the weldment should be improved to avoid defects in the heat-affected zone and material flexibility.

[0015] According to the above technical solution, in S3, the specific process of analyzing the behavior of the welding head through the change of the moving action and position is as follows:

[0016] S3-1, the position change characteristics of infrared detection are as follows: The height of the axis coordinate suddenly stops changing the position coordinate, and Axis and The axis coordinates keep changing back and forth, that is, the position change characteristics of the welding head are welding pause and welding adjustment, which means that it is judged as normal welding at this time and no warning is issued;

[0017] S3-2, if the infrared detection position change characteristic is that the welding head is The axis coordinates keep changing back and forth, and the movement of the welding head is judged to be an abnormal movement caused by incorrect force application, which means that it is judged as incorrect force application. At this time, the sound transmitter is controlled to align its direction with the position of the welding head and emit a sound warning.

[0018] According to the above technical solution, in S4, the method for determining whether the rebound amplitude exceeds the set value is:

[0019] S4-1. At a fixed interval between each assessment, the image collector is used to scan the assessment area above the weldment to identify the welding head in the image. The small rebound behavior of each identified welding head will be marked in the form of a bounding box, and the height of these bounding boxes will be counted to obtain the amplitude of the small rebound. ,in The number of scans for each assessment;

[0020] S4-2. When judging the probability of incorrect force application of the welding head, clearly define the number of times the corresponding rebound amplitude exceeds the set value in one assessment. , the corresponding rebound amplitude exceeds the total amplitude of the set value The assessment time is , then the probability of incorrect force application by the welding head is ,in is the weight coefficient of rebound amplitude and assessment time. The setting value of rebound amplitude is ,when , the sound transmitter will be activated.

[0021] According to the above technical solution, in S4-2, when the welding head is judged to be normal welding, the plane coordinates of the landing point of each welding head when it rebounds on the weldment are recorded. ,in The number of landing points is the time when the welding head leaves the landing point on the weldment. , let the number of times the welding head leaves the landing point on the weldment be ,Right now , and take the plane coordinates of each landing point as the center and the radius as When a welding head enters the circle of the landing point again, the time for the welding head to leave the landing point on the weldment is , the probability of incorrect force application by the welding head will be reduced, and the probability will be corrected, that is, the probability becomes ,in The time factor for a single assessor to repeat welding within the same range.

[0022] According to the above technical solution, in S4-2, when it is determined that the welding head is incorrectly applying force, the plane coordinates of the landing points of each welding head on the weldment are recorded. , and take the plane coordinates of each landing point as the center and the radius as When the welding head frequently enters the circle where the landing point is located again, it means that the weldment of this plane coordinate has defects in the heat-affected zone and material flexibility. The number of times the welding head enters the circle where a certain plane coordinate is located is recorded as , that is, the probability becomes ,in The coefficient of incorrect force application of the welding head that occurs when multiple examiners repeatedly apply the force at a certain coordinate.

[0023] According to the above technical solution, in S5, when the welding head rebound amplitude exceeds the set value multiple times within the assessment range, the average plane distance between the coordinates of the landing points of each welding head on the weldment is calculated. , and the number of drop point coordinates , if the ratio of the number of point coordinates to the average plane distance ,when ≥ When , it indicates that there are defects in the heat-affected zone and material flexibility in the weldment structure. The abnormal rebound amplitude of the weldment exceeds the threshold value of the set value.

[0024] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: after the welding head contacts the weld and rebounds slightly, the behavior of the welding head is analyzed by the subsequent movement of the welding head and the change in position to determine whether it is normal welding or the possibility of erroneous force application. At the same time, by repeatedly counting the coordinate points where the welding head contacts the weld and rebounds slightly, the probability of normal welding and the probability of erroneous force application are corrected, making the judgment result more accurate, and structural defects are judged for welds where the welding head rebounds more than once in an amplitude exceeding the set value, to avoid as much as possible the problems affecting the assessment results caused by the heat affected zone or material flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0026] Figure 1 It is a schematic diagram of the overall module structure of the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] See also Figure 1 The present invention provides a technical solution: a melting welding training and assessment system based on AI multi-action recognition, including an infrared positioning module, an AI multi-action recognition module, and a warning module. The infrared positioning module is used to locate the welding head through infrared detection technology, the AI ​​multi-action recognition module is used to scan the welding head after locating it, and the warning module is used to determine the probability of normal welding of the welding head and use sound to remind the assessor to warn;

[0029] The infrared positioning module includes an infrared signal transmitter, a signal receiver, and a data processing unit. The infrared signal transmitter and the signal receiver are both electrically connected to the data processing unit. The infrared signal transmitter is used to transmit infrared signals to the assessment area. These signals will be reflected by the target welding head. The signal receiver is used to receive the infrared signals reflected by the target. The data processing unit is used to analyze the received signals, determine the detected welding head reflection signal, extract the distance, height and azimuth position information of the welding head, and locate the welding head in real time.

[0030] The AI ​​multi-action recognition module includes a camera control unit, an image collector, an action recognition algorithm module, and a data statistics module. The camera control unit is electrically connected to the image collector and the data processing unit. The action recognition algorithm module is electrically connected to the data statistics module. The camera control unit is used to control the switch and rotation of the camera for scanning. The image collector is used to capture video image data in real time. The action recognition algorithm module is used to analyze the scanned image using a deep learning method to identify the welding action and movement state of the welding head. The data statistics module is used to count the rebound amplitude and behavior pattern of the welding head on the weldment.

[0031] The warning module includes a warning strategy module, a sound transmitter, a probability calculation module, a sound direction control unit, and a sound warning unit. The probability calculation module is electrically connected to the data statistics module, the warning strategy module is electrically connected to the sound transmitter and the probability calculation module, the sound direction control unit is electrically connected to the sound transmitter, and the sound warning unit is electrically connected to the warning strategy module. The warning strategy module is used to take warning measures according to the position information and image information of the welding head. The sound transmitter is used to play the sound and turn to the target assessor to produce auditory discomfort. The probability calculation module is used to evaluate the probability of incorrect force application of the welding head according to the rebound amplitude of the welding head on the weldment and the frequency of each assessment rebound. The sound direction control unit is used to selectively control the sound transmitter to align with the direction of the assessor who has made an erroneous action. The sound warning unit is used to use sound warning when the rebound amplitude of the welding head on the weldment exceeds a set value;

[0032] The working method of the system includes the following steps:

[0033] S1. Use infrared detection technology to determine the position of the welding head in the assessment range in real time, establish a spatial rectangular coordinate system, and use Indicates the position coordinates of the welding head;

[0034] S2. When the moving height of a welding head is lower than the critical height, the image collector is controlled to face the target welding head and scan, the scanned image is analyzed in real time, and the moving action of the welding head is identified;

[0035] S3. After the welding head contacts the weld and rebounds slightly, the welding head behavior is analyzed through the movement and position change of the welding head to determine whether it is normal welding or incorrect force application, and decide whether to issue a warning signal;

[0036] S4. Real-time statistics of the rebound amplitude of the welding head on the weldment, combined with the average number of small-amplitude rebound behaviors in each assessment, judge the probability of incorrect force application of the welding head. When the rebound amplitude exceeds the set value, the sound transmitter will start to warn the assessor in a directional manner;

[0037] S5. When the rebound amplitude of the welding head tested by multiple people exceeds the set value at the same time, the weldment should be improved to avoid defects in the heat-affected zone and material flexibility;

[0038] In S3, the specific process of analyzing the behavior of the welding head through the change of movement and position is as follows:

[0039] S3-1, the position change characteristics of infrared detection are as follows: The height of the axis coordinate suddenly stops changing the position coordinate, and Axis and The axis coordinates keep changing back and forth, that is, the position change characteristics of the welding head are welding pause and welding adjustment, which means that it is judged as normal welding at this time and no warning is issued;

[0040] S3-2, if the infrared detection position change characteristic is that the welding head is The axis coordinates keep changing back and forth, and the movement of the welding head is judged to be an abnormal movement caused by the wrong force application, which means that the force is judged to be wrong at this time. At this time, the sound transmitter is controlled to align its direction with the position of the welding head and emit a sound warning;

[0041] In S4, the method for determining whether the rebound amplitude exceeds the set value is:

[0042] S4-1. At a fixed interval between each assessment, the image collector is used to scan the assessment area above the weldment to identify the welding head in the image. The small rebound behavior of each identified welding head will be marked in the form of a bounding box, and the height of these bounding boxes will be counted to obtain the amplitude of the small rebound. ,in The number of scans for each assessment;

[0043] S4-2. When judging the probability of incorrect force application of the welding head, clearly define the number of times the corresponding rebound amplitude exceeds the set value in one assessment. , the corresponding rebound amplitude exceeds the total amplitude of the set value The assessment time is , then the probability of incorrect force application by the welding head is ,in is the weight coefficient of rebound amplitude and assessment time. The setting value of rebound amplitude is ,when The sound transmitter will start when

[0044] In S4-2, when the welding head is judged to be welding normally, the plane coordinates of the landing point of each welding head when it rebounds on the weldment are recorded. ,in The number of landing points is the time when the welding head leaves the landing point on the weldment. , let the number of times the welding head leaves the landing point on the weldment be ,Right now , and take the plane coordinates of each landing point as the center and the radius as When a welding head enters the circle of the landing point again, the time for the welding head to leave the landing point on the weldment is , the probability of incorrect force application by the welding head will be reduced, and the probability will be corrected, that is, the probability becomes ,in The time factor for a single assessor to repeat welding within the same range;

[0045] In S4-2, when it is determined that the welding head is applying force incorrectly, the plane coordinates of each welding head's landing point on the weldment are recorded. , and take the plane coordinates of each landing point as the center and the radius as When the welding head frequently enters the circle where the landing point is located again, it means that the weldment of this plane coordinate has defects in the heat-affected zone and material flexibility. The number of times the welding head enters the circle where a certain plane coordinate is located is recorded as , that is, the probability becomes ,in The coefficient of incorrect force application of the welding head when multiple examiners repeatedly apply the force at a certain coordinate;

[0046] In S5, when the welding head rebound amplitude exceeds the set value multiple times within the assessment range, the average plane distance between the coordinates of the landing points of each welding head on the weldment is calculated. , and the number of drop point coordinates , if the ratio of the number of point coordinates to the average plane distance ,when ≥ When , it indicates that there are defects in the heat-affected zone and material flexibility in the weldment structure. The abnormal rebound amplitude of the weldment exceeds the threshold value of the set value.

[0047] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0048] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. The melting welding training and assessment system based on AI multi-action recognition is characterized by: It includes an infrared positioning module, an AI multi-action recognition module, and a warning module. The infrared positioning module is used to locate the welding head through infrared detection technology. The AI ​​multi-action recognition module is used to scan the welding head after locating it. The warning module is used to determine the probability of normal welding of the welding head and use sound to remind the examiner to warn. The infrared positioning module includes an infrared signal transmitter, a signal receiver, and a data processing unit. The infrared signal transmitter and the signal receiver are both electrically connected to the data processing unit. The infrared signal transmitter is used to transmit infrared signals to the assessment area. These signals will be reflected by the target welding head. The signal receiver is used to receive the infrared signals reflected by the target. The data processing unit is used to analyze the received signals, determine the detected welding head reflection signal, extract the distance, height and azimuth position information of the welding head, and locate the welding head in real time; The AI ​​multi-action recognition module includes a camera control unit, an image collector, a motion recognition algorithm module, and a data statistics module. The camera control unit is electrically connected to the image collector and the data processing unit. The motion recognition algorithm module is electrically connected to the data statistics module. The camera control unit is used to control the switch and rotation of the camera for scanning. The image collector is used to capture video image data in real time. The motion recognition algorithm module is used to analyze the scanned image using a deep learning method to identify the welding action and movement state of the welding head. The data statistics module is used to count the rebound amplitude and behavior pattern of the welding head on the weldment. The warning module includes a warning strategy module, a sound transmitter, a probability calculation module, a sound direction control unit, and a sound warning unit. The probability calculation module is electrically connected to the data statistics module, the warning strategy module is electrically connected to the sound transmitter and the probability calculation module, the sound direction control unit is electrically connected to the sound transmitter, and the sound warning unit is electrically connected to the warning strategy module. The warning strategy module is used to take warning measures according to the position information and image information of the welding head. The sound transmitter is used to play the sound and turn to the target assessor to produce auditory discomfort. The probability calculation module is used to evaluate the probability of incorrect force application of the welding head according to the rebound amplitude of the welding head on the weldment and the frequency of each assessment rebound. The sound direction control unit is used to control the sound transmitter to align with the direction of the assessor who has made an erroneous action. The sound warning unit is used to use sound warning when the rebound amplitude of the welding head on the weldment exceeds a set value.

2. The melting welding training and assessment system based on AI multi-action recognition according to claim 1 is characterized by: The working method of the system includes the following steps: S1. Use infrared detection technology to determine the position of the welding head in the assessment range in real time, establish a spatial rectangular coordinate system, and use {x, y, z} to represent the position coordinates of the welding head; S2. When the moving height of a welding head is lower than the critical height, the image collector is controlled to face the target welding head and scan, the scanned image is analyzed in real time, and the moving action of the welding head is identified; S3. After the welding head contacts the weld and rebounds slightly, the welding head behavior is analyzed through the movement and position change of the welding head to determine whether it is normal welding or incorrect force application, and decide whether to issue a warning signal; S4. Real-time statistics of the rebound amplitude of the welding head on the weldment, combined with the average number of small-amplitude rebound behaviors in each assessment, judge the probability of incorrect force application of the welding head. When the rebound amplitude exceeds the set value, the sound transmitter will start to warn the assessor in a directional manner; S5. When the rebound amplitude of the welding head tested by multiple people exceeds the set value at the same time, the weldment should be improved.

3. The melting welding training and assessment system based on AI multi-action recognition according to claim 2 is characterized by: In S3, the specific process of analyzing the behavior of the welding head through the change of the moving action and position is as follows: S3-1, the position change feature of infrared detection is that the welding head suddenly stops changing its position coordinate at a low height of the z-axis coordinate, and the x-axis and y-axis coordinates keep changing back and forth, that is, the position change feature of the welding head is welding pause and welding adjustment, which means that it is judged as normal welding at this time, and no warning is issued at this time; S3-2. If the position change characteristic of the infrared detection is that the welding head changes back and forth continuously in the z-axis coordinate, it is judged that the movement of the welding head is an abnormal movement caused by incorrect force application, which means that it is judged as incorrect force application at this time. At this time, the sound transmitter is controlled to align its direction with the position of the welding head and emit a sound warning.

4. The melting welding training and assessment system based on AI multi-action recognition according to claim 3 is characterized by: In S4, the method for determining whether the rebound amplitude exceeds the set value is: S4-1. At a fixed interval between each assessment, the image collector is used to scan the assessment area above the weldment to identify the welding head in the image. The small rebound behavior of each identified welding head will be marked in the form of a bounding box. The height of these bounding boxes is counted to obtain the amplitude of the small rebound {f1,f2,…f i }, where i is the number of scans per assessment; S4-2. When judging the probability of incorrect force application of the welding head, the number of times k when the corresponding rebound amplitude exceeds the set value in one test is determined, and the total amplitude ∑f of the corresponding rebound amplitude exceeding the set value is obtained. k , the test time is t, then the probability of incorrect force application by the welding head is Among them, α is the weight coefficient of rebound amplitude and assessment time, and the setting value of rebound amplitude is f0. , the sound transmitter will be activated.

5. The melting welding training and assessment system based on AI multi-action recognition according to claim 4 is characterized in that: In the above S4-2, when it is determined that the welding head is in normal welding, the plane coordinates of the landing point {(x1, y1), (x2, y2), ..., (x m ,y m )}, where m is the number of landing points, and the time {T1, T2, …, T m }, let the number of times the welding head leaves the landing point on the weldment be n, that is, n∈[1~m], and draw a circle with a radius of r with the plane coordinates of each landing point as the center. When the welding head enters the circle of the landing point again, let the time when the welding head leaves the landing point on the weldment be T n , the probability of incorrect force application by the welding head will be reduced, and the probability will be corrected, that is, the probability becomes Where β is the time coefficient for a single assessor to repeat welding within the same range.

6. The melting welding training and assessment system based on AI multi-action recognition according to claim 5 is characterized by: In the above S4-2, when it is determined that the welding head is applying force incorrectly, the plane coordinates {(x1, y1), (x2, y2), ..., (x m ,y m )}, and a circle with a radius of r is drawn with each landing point plane coordinate as the center. When the welding head frequently enters the circle range where the landing point is located again, it means that the weldment of this plane coordinate has defects in the heat affected zone and material flexibility. The number of times the welding head enters the circle range where a certain plane coordinate is located is recorded as W m , that is, the probability becomes Where γ is the coefficient of incorrect force application of the welding head caused by multiple examiners repeatedly applying force at a certain coordinate.

7. The melting welding training and assessment system based on AI multi-action recognition according to claim 6 is characterized by: In S5, when the welding head rebound amplitude exceeds the set value for multiple times within the test range, the average plane distance ΔH between the landing point coordinates of each welding head on the weldment and the number of landing point coordinates g are calculated. If the ratio of the number of landing point coordinates to the average plane distance is When F ≥ F0, it indicates that there are defects in the heat-affected zone and material flexibility in the weldment structure, and F0 is the threshold at which the abnormal rebound amplitude of the weldment exceeds the set value.

Images