A method for identifying and determining fire operation violations based on machine vision

Abstract

This invention discloses a machine vision-based method for identifying and judging violations during hot work operations, belonging to the field of hot work operation recognition technology. Specifically, it includes: acquiring infrared thermal imaging data and optical image data of the hot work area to obtain the surface temperature distribution and outline of the hot work tool; for the infrared thermal imaging data, extracting the temperature gradient change characteristics of the heated area of ​​the hot work tool, and generating a dynamic deformation trajectory model based on the temperature gradient change characteristics; for the optical image data, extracting the outline offset of the hot work tool after thermal deformation through outline difference analysis between consecutive frames, and establishing a time-series correspondence with the dynamic deformation trajectory model; when the correspondence between the outline offset of the hot work tool and its temperature gradient change characteristics deviates from a preset correlation threshold range, it is judged as a violation operation; this invention enhances the ability to judge complex violations.

Description

Technical Field

[0001] This invention relates to the field of hot work operation identification technology, and specifically to a method for identifying and judging hot work operation violations based on machine vision. Background Technology

[0002] Hot work operations (such as welding, cutting, and spraying) are common high-risk operations in industrial production. Improper operation or tool malfunctions can lead to fires, explosions, and other safety accidents. Traditional safety supervision relies mainly on manual inspections or simple temperature monitoring, which is insufficient for real-time identification of violations, especially abnormal conditions caused by heat deformation of hot work tools. With the development of machine vision technology, image analysis-based intelligent monitoring systems are increasingly being applied to hot work scenarios. However, due to the deformation characteristics of hot work tools at high temperatures, conventional vision algorithms are prone to misjudgments based on changes in tool shape, necessitating more accurate identification methods.

[0003] Currently, machine vision monitoring for hot work operations mainly relies on optical cameras or infrared thermal imaging devices working independently. For example, some solutions analyze the contour changes of welding torches or electrodes through optical image analysis and combine this with preset shape template matching to determine the tool status; other solutions rely solely on infrared thermal imaging data, using temperature thresholds to determine the presence of overheating risks. In addition, a few studies have attempted to integrate optical and infrared data for comprehensive monitoring of hot work operations.

[0004] However, existing visual monitoring of hot work operations ignores the interference of thermal deformation on visual recognition. When hot work tools (such as welding rods and spray guns) expand or bend at high temperatures, the contours in the optical images differ too much from the standard templates, making traditional shape matching algorithms prone to failure. Summary of the Invention

[0005] The purpose of this invention is to provide a machine vision-based method for identifying and judging violations during hot work operations, thereby solving the following technical problems:

[0006] When hot work tools are exposed to high temperatures, they may expand, bend, or undergo other deformations, resulting in a significant difference between the contour in the optical image and the standard template, which can cause traditional shape matching algorithms to fail.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] A machine vision-based method for identifying and judging violations during hot work operations includes the following steps:

[0009] Infrared thermal imaging data and optical image data of the hot work area are collected simultaneously by infrared thermal imaging equipment and optical zoom equipment. The infrared thermal imaging equipment is used to obtain the surface temperature distribution of the hot work tools, and the optical zoom equipment is used to obtain the outline of the hot work tools.

[0010] For infrared thermal imaging data, extract the temperature gradient change characteristics of the heated area of ​​the hot work tool, and generate a dynamic deformation trajectory model based on the temperature gradient change characteristics.

[0011] For optical image data, the contour offset of the hot tool after thermal deformation is extracted by contour difference analysis between consecutive frames, and a time series correspondence is established with the dynamic deformation trajectory model.

[0012] When the correspondence between the external contour offset of the hot work tool and its temperature gradient change characteristics deviates from the preset correlation threshold range, it is judged as a violation of operation.

[0013] As a further aspect of the present invention: the generation process of the dynamic deformation trajectory model is as follows:

[0014] Based on the temperature change rate of different areas on the surface of the hot work tool in the infrared thermal imaging data, areas where the temperature rise rate exceeds a preset critical value are marked as high-activity hot zones. Using the temperature diffusion direction of the high-activity hot zones as a reference, the extension path of the hot work tool's thermal deformation is simulated to generate a three-dimensional trajectory model that includes the time dimension.

[0015] As a further aspect of the present invention: the specific method for establishing the correspondence is as follows:

[0016] The offset direction of the hot tool's outline in the optical image is vector-superimposed with the temperature diffusion direction of the highly active hot zone in the dynamic deformation trajectory model. When the angle between the two directions is less than a preset angle, it is judged as normal thermal deformation; when the angle exceeds the preset angle or the offset direction is opposite to the temperature diffusion direction, it is judged as abnormal deformation.

[0017] As a further aspect of the present invention: the determination of abnormal deformation further includes:

[0018] If the offset of the outline of the hot work tool is analyzed over time, and the offset shows a non-monotonic increase or abrupt change within a continuous time window, and is accompanied by a step fluctuation in the temperature diffusion rate of the highly active hot zone, then it is determined to be a violation of operating procedures.

[0019] As a further aspect of the present invention: the processing of the optical image data includes:

[0020] The outline of the hot work tool after heat deformation is reconstructed in three dimensions to generate a three-dimensional model containing the curvature change of the tool surface. This three-dimensional model is then spatially compared with the standard model in the unheated state, and local areas with curvature differences exceeding a preset threshold are extracted as abnormal deformation areas.

[0021] As a further aspect of the present invention: the curvature change trend of the abnormal deformation region is spatially projected and matched with the temperature diffusion path in the dynamic deformation trajectory model. If the overlap between the region with the largest curvature change and the end position of the temperature diffusion path is less than a preset ratio, it is determined to be an illegal operation.

[0022] As a further aspect of the present invention: if an abnormal temperature rise is detected in the non-working area of ​​the hot work tool in the infrared thermal imaging data, and the temperature rise area simultaneously exhibits unexpected deformation in the optical image, and the deformation direction conflicts with the normal working force direction of the tool, then it is determined to be a violation of operating procedures.

[0023] As a further aspect of the present invention: for different types of hot work tools, a standard database of their heat deformation trajectories is established in advance, including the axial linear expansion trajectory model of welding rods and the radial radial deformation trajectory model of spray guns; in actual testing, the dynamic deformation trajectory model generated in real time is matched with the standard database, and if the matching degree is lower than a preset threshold, a violation alarm is triggered.

[0024] The beneficial effects of this invention are:

[0025] This invention solves the problem of image recognition algorithm failure caused by thermal deformation of hot work tools in existing technologies by synchronously acquiring data from infrared thermal imaging equipment and optical zoom equipment. The construction of a dynamic deformation trajectory model can accurately reflect the correlation between the temperature gradient change and deformation trend of the tool after heating. By performing vector superposition analysis of the outline offset and the temperature diffusion direction, it can effectively distinguish between normal thermal deformation and abnormal deformation caused by illegal operation, overcoming the limitations of traditional static image matching algorithms. At the same time, the pre-established standard database for different hot work tools enables the system to adapt to the differentiated characteristics such as the axial linear expansion of welding rods and the radial radial deformation of spray guns, which greatly improves the recognition accuracy. By analyzing the spatial projection matching relationship between abnormal deformation areas and temperature diffusion paths through time series analysis, the ability to judge complex illegal behaviors is further enhanced, and finally, high-precision intelligent recognition of illegal behaviors in hot work operations is achieved. Attached Figure Description

[0026] The invention will now be further described with reference to the accompanying drawings.

[0027] Figure 1 This is a flowchart illustrating the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] Please see Figure 1 As shown, this invention is a machine vision-based method for identifying and judging violations during hot work operations, comprising the following steps:

[0030] Data collection

[0031] Using both infrared thermal imaging and optical zoom equipment, data is simultaneously collected from the hot work area. The infrared thermal imaging focuses on acquiring the temperature distribution on the surface of the hot work tools. Taking welding equipment as an example, during normal welding, the temperature distribution of the soldering iron tip is uniform and maintained within a specific temperature range; however, if improper operations such as excessive welding time or excessive current occur, the temperature of the soldering iron tip will rise sharply, and the temperature distribution will become uneven. The optical zoom equipment is responsible for clear imaging, accurately acquiring the shape and contour of the hot work tools, providing basic data for subsequent analysis of changes in the shape and contour.

[0032] Infrared thermal imaging data processing

[0033] The key to analyzing the acquired infrared thermal imaging data lies in extracting the temperature gradient characteristics of the heated area of ​​the hot work tool. These temperature gradient characteristics directly reflect the rate of temperature change over time and space. During normal operation, the temperature gradient in the heated area of ​​the hot work tool changes gradually and follows a specific pattern. For example, with an oxy-fuel cutting torch, during normal cutting operations, the temperature near the nozzle rises rapidly, then gradually decreases with increasing distance from the nozzle, and the cooling rate remains relatively stable. Based on the analysis and processing of these temperature gradient characteristics, a dynamic deformation trajectory model is further generated. This model, using time as the dimension, displays the temperature change trend and corresponding deformation trajectory of the heated area of ​​the hot work tool at different times.

[0034] Optical image data processing

[0035] When processing optical image data, a contour difference analysis method between consecutive frames is employed. During hot work operations, changes in tool heating and operational conditions lead to alterations in the tool's shape and contour. By comparing consecutive frame optical images, the contour offset of the hot work tool after thermal deformation can be accurately extracted. For example, in the heating and bending of sheet metal, the shape and contour of the sheet metal gradually change as the heating time increases. Analyzing the contour differences in consecutive frame images yields the contour offset. Subsequently, a time-series correspondence is established between this contour offset and the previously generated dynamic deformation trajectory model, thereby comprehensively and dynamically monitoring the operational status of the hot work tool from two dimensions: temperature change and contour change.

[0036] Violation judgment

[0037] Based on the established correspondence, reasonable judgment rules are set. The preset correlation threshold range is derived from a large amount of experimental data and actual production experience. For example, multiple simulations of normal welding operations are conducted to statistically analyze the correspondence between the deviation of the welding tool's outline and the temperature gradient change characteristics during normal operation, thereby setting a reasonable threshold range. When the actual operation monitoring data exceeds this threshold range, such as excessive welding current causing the welding tool to overheat and deform beyond the normal range, it is judged as a violation of operating procedures.

[0038] In a preferred embodiment of the present invention, the generation process of the dynamic deformation trajectory model is as follows:

[0039] First, an in-depth analysis of the infrared thermal imaging data is conducted. During hot work operations, the temperature changes in different areas of the hot work tool surface vary significantly. By accurately calculating the rate of temperature change, the thermal activity state of each area can be clearly identified. Here, areas where the rate of temperature rise exceeds a pre-set critical value are marked as highly active thermal zones. For example, in arc welding operations, the area where the electrode contacts the workpiece generates a large amount of heat instantaneously, resulting in an extremely rapid rate of temperature rise, usually far exceeding the preset critical value, and thus being identified as a highly active thermal zone.

[0040] After identifying the highly active hot zone, its temperature diffusion direction is used as a key benchmark. The temperature diffusion direction reflects the heat transfer path on the surface of the hot work tool, which is crucial for simulating the extension path of the tool's thermal deformation. Using specialized algorithms and data analysis methods, the potential deformation trajectory of the tool is simulated along the temperature diffusion direction. Based on this, a time dimension is incorporated, integrating the temperature diffusion of the highly active hot zone at different times with the corresponding deformation extension paths, ultimately generating a three-dimensional trajectory model including the time dimension. This model can intuitively and dynamically display the temperature diffusion and deformation extension of the highly active hot zone over time during the hot work tool's heating process, providing crucial reference for subsequent analysis.

[0041] In another preferred embodiment of the present invention, the specific method for establishing the correspondence is as follows:

[0042] In optical images, the outline of a hot work tool shifts due to heat and operational processes. Simultaneously, the highly active hot zones in the dynamic deformation trajectory model exhibit specific temperature diffusion directions. Performing vector overlay analysis of these two factors provides a deeper understanding of the tool's deformation state.

[0043] In practice, a preset angle is used as a judgment benchmark. When the offset direction of the tool's outline in the optical image is vector-superimposed with the temperature diffusion direction of the highly active hot zone in the dynamic deformation trajectory model, if the angle between the two directions is less than the preset angle, it indicates that the tool's deformation and temperature diffusion exhibit a synergistic relationship consistent with normal operating patterns, and can be judged as normal thermal deformation. For example, in gas welding operations, as the flame continuously heats the workpiece, the workpiece's outline gradually shifts slowly towards the direction of flame temperature diffusion, and the angle between the offset direction and the temperature diffusion direction is small; this situation is considered normal thermal deformation.

[0044] Conversely, when the directional angle exceeds the preset angle, or when the offset direction of the tool's outline is completely opposite to the direction of temperature diffusion, it means that the tool's deformation deviates from the normal thermal deformation pattern, and this is judged as abnormal deformation. For example, in cases of localized overheating caused by improper operation, a part of the tool's outline may suddenly shift in the opposite direction to the direction of temperature diffusion, which falls under the category of abnormal deformation.

[0045] In a preferred embodiment of this invention, the determination of abnormal deformation further includes:

[0046] The determination of abnormal deformation has been further refined. In addition to considering the relationship between the direction of the outline offset and the direction of temperature diffusion, a time series analysis of the offset of the hot work tool's outline is also performed. By collecting and analyzing the offset data within a continuous time window, its changing trend is observed.

[0047] Under normal circumstances, within a continuous time window, if the hot work is performed correctly, the offset of the tool's outline will change relatively steadily over time. For example, during metal forging heating, the offset of the metal billet's outline will increase slowly and monotonically as the heating time increases. However, the situation is quite different when violations occur. If the offset does not increase monotonically within the continuous time window, such as fluctuating between increases and decreases, or undergoing abrupt changes with a large sudden shift, and simultaneously, the temperature diffusion rate in the highly active hot zone also fluctuates dramatically (i.e., the temperature diffusion rate suddenly increases or decreases rapidly), this complex situation indicates an abnormality in the hot work process, thus constituting a violation. For example, when using thermal cutting equipment, if the operator improperly adjusts the cutting parameters, it may cause a sharp fluctuation in the temperature of the cutting area, and at the same time, the offset of the cut material's outline will also change abnormally. In this case, this comprehensive judgment method can accurately identify the violation.

[0048] In another preferred embodiment of the present invention, the processing of the optical image data includes:

[0049] First, 3D reconstruction technology is used to reconstruct the outline of the hot workpiece after it has been deformed by heat. This process integrates and calculates multi-view optical image data to generate a three-dimensional model that accurately reflects the changes in the curvature of the tool's surface. For example, in the hot forging process of metal processing, the shape of the metal workpiece changes under high temperature, and the curvature of various parts of its surface also changes accordingly. 3D reconstruction technology can clearly present these changes.

[0050] After generating the 3D model, it is spatially compared with a standard model of the tool in its unheated state. The standard model records the precise shape and surface features of the tool under normal, unheated conditions. Using a professional spatial comparison algorithm, corresponding parts of the two models are meticulously compared, with a focus on extracting local areas where the curvature difference exceeds a pre-set threshold; these areas are identified as abnormal deformation regions. Taking a soldering iron as an example, the shape and surface curvature of the soldering iron tip are relatively stable during normal use. However, in the event of abnormal heating, the curvature of certain areas of the soldering iron tip may change significantly. By comparing it with the standard model, these abnormal regions can be quickly identified.

[0051] In a preferred embodiment, the curvature change trend of the abnormal deformation region is spatially projected and matched with the temperature diffusion path in the dynamic deformation trajectory model. First, the curvature change of the abnormal deformation region is analyzed in detail to identify the area with the largest curvature change, which is often where the tool undergoes the most significant thermal deformation. Simultaneously, in the dynamic deformation trajectory model, the end position of the temperature diffusion path is identified, as the end of the temperature diffusion path typically reflects the boundary conditions where heat is transferred to the tool edge or the area of ​​influence.

[0052] Subsequently, a spatial projection matching operation is performed, projecting the area of ​​maximum curvature change in the abnormal deformation region onto the space constructed by the dynamic deformation trajectory model and comparing it with the end position of the temperature diffusion path. If the overlap between the two is less than a pre-set ratio, it means that there is a significant deviation between the location of the abnormal deformation region and the expected range of temperature diffusion. For example, in flame cutting operations, if the overlap between the abnormal deformation location of the cutting area and the end of the flame temperature diffusion path is low, it indicates that the cutting process may have been interfered with by abnormal factors, and this can be judged as a violation of operating procedures.

[0053] In another preferred embodiment of the present invention, during the analysis of infrared thermal imaging data, the focus is on detecting whether abnormal temperature rises occur in the non-working areas of the hot work tool. Non-working areas generally refer to parts of the hot work tool that do not directly participate in heat generation or transfer, such as the outer casing of welding equipment. When an abnormal temperature rise is detected in a non-working area, further analysis using optical image data is required.

[0054] If an unexpected deformation is observed in the temperature rise area simultaneously in the optical image, and the direction of this deformation spatially conflicts with the normal force direction of the tool, it can be determined as a violation of operating procedures. For example, when using an electric grinder, the grinding head heats up and the force direction is clear during normal operation. If an abnormal temperature rise suddenly occurs in a certain area of ​​the grinder's casing, and at the same time, the optical image shows that this area has undergone deformation inconsistent with the normal force direction of the grinder, this is likely due to improper operation, such as excessive pressure or collision. In this case, the system will determine it as a violation of operating procedures.

[0055] In another preferred embodiment of the present invention, a standard database matching-based identification strategy is employed for different categories of hot work tools. In the preliminary preparation stage, a standard database of the thermal deformation trajectories of various hot work tools is pre-established through extensive experiments and data collection. For example, for slender hot work tools like welding rods, their thermal expansion after heating is primarily axial linear, thus establishing an axial linear expansion trajectory model for welding rods; while for spray guns, heat is transferred radially around the nozzle during operation, thereby constructing a radially radial deformation trajectory model for spray guns.

[0056] During actual testing, the system generates a dynamic deformation trajectory model of the hot work tool in real time and then performs pattern matching with the corresponding model in the standard database. A precise algorithm calculates the matching degree between the two. If the matching degree is lower than a pre-set threshold, it indicates that the current hot work tool's thermal deformation trajectory differs significantly from the standard, potentially indicating unauthorized operation. In this case, the system will trigger a violation alarm. For example, in actual welding operations, if the real-time deformation trajectory of the welding electrode has a low matching degree with the axial linear expansion trajectory model in the standard database, it may mean that parameters such as welding current and voltage are improperly set, leading to abnormal heating of the welding electrode. The system can then promptly issue an alarm to remind the operator.

[0057] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A method for identifying and judging violations of hot work operations based on machine vision, characterized in that, Includes the following steps: Infrared thermal imaging data and optical image data of the hot work area are collected simultaneously by infrared thermal imaging equipment and optical zoom equipment. The infrared thermal imaging equipment is used to obtain the surface temperature distribution of the hot work tools, and the optical zoom equipment is used to obtain the outline of the hot work tools. For infrared thermal imaging data, extract the temperature gradient change characteristics of the heated area of ​​the hot work tool, and generate a dynamic deformation trajectory model based on the temperature gradient change characteristics. For optical image data, the contour offset of the hot tool after thermal deformation is extracted by contour difference analysis between consecutive frames, and a time series correspondence is established with the dynamic deformation trajectory model. When the correspondence between the external contour offset of the hot work tool and its temperature gradient change characteristics deviates from the preset correlation threshold range, it is judged as a violation of operation. The generation process of the dynamic deformation trajectory model is as follows: Based on the temperature change rate of different areas on the surface of the hot work tool in the infrared thermal imaging data, the areas where the temperature rise rate exceeds the preset critical value are marked as high active heat zones. Based on the temperature diffusion direction of the high active heat zones, the extension path of the hot work tool under heat deformation is simulated to generate a three-dimensional trajectory model including the time dimension. The specific method for establishing the correspondence on the time series is as follows: The offset direction of the outline of the hot tool in the optical image is vector-superimposed with the temperature diffusion direction of the highly active hot zone in the dynamic deformation trajectory model. When the angle between the two directions is less than a preset angle, it is judged as normal thermal deformation. When the directional angle exceeds the preset angle or the offset direction is opposite to the temperature diffusion direction, it is judged as abnormal deformation; The determination of abnormal deformation also includes: If the offset of the outline of the hot work tool is analyzed over time, and the offset shows a non-monotonic increase or abrupt change within a continuous time window, and is accompanied by a step fluctuation in the temperature diffusion rate of the highly active hot zone, then it is determined to be a violation of operating procedures.

2. The method for identifying and judging violations of hot work operations based on machine vision according to claim 1, characterized in that, The processing of the optical image data includes: The outline of the hot work tool after heat deformation is reconstructed in three dimensions to generate a three-dimensional model containing the curvature change of the tool surface. This three-dimensional model is then spatially compared with the standard model in the unheated state, and local areas with curvature differences exceeding a preset threshold are extracted as abnormal deformation areas.

3. The method for identifying and judging violations of hot work operations based on machine vision according to claim 2, characterized in that, The curvature change trend of the abnormal deformation region is spatially projected and matched with the temperature diffusion path in the dynamic deformation trajectory model. If the overlap between the region with the largest curvature change and the end position of the temperature diffusion path is less than a preset ratio, it is judged as an illegal operation.

4. The method for identifying and judging violations of hot work operations based on machine vision according to claim 1, characterized in that, If an abnormal temperature rise is detected in the non-working area of ​​a hot work tool in infrared thermal imaging data, and this temperature rise area simultaneously exhibits unexpected deformation in the optical image, and the direction of deformation conflicts with the direction of force during normal tool operation, then it is determined to be a violation of operating procedures.

5. The method for identifying and judging violations of hot work operations based on machine vision according to claim 1, characterized in that, For different types of hot work tools, a standard database of their heat deformation trajectories is established in advance, including the axial linear expansion trajectory model of welding rods and the radial deformation trajectory model of spray guns. In actual testing, the dynamic deformation trajectory model generated in real time is matched with the standard database. If the matching degree is lower than the preset threshold, a violation alarm is triggered.

Images