AI analysis of safety warning methods for confined space operations

Abstract

A method for AI-based safety warning analysis of confined space operations includes: obtaining the dimensions of the inner wall of a reactor, cutting a flexible visual patch based on the dimensions, and triggering a visual frame sequence. Locally distorted regions are detected in the visible light image in the visual frame sequence, and pixel correction is performed on the detected locally distorted regions according to a calibration mapping table. A pre-trained U-Net semantic segmentation model is used to process the visible light image, outputting a corresponding binary mask image. Based on the binary mask image, a determination is made as to whether to switch the visible light image to a sonar image. Regional heat weights are calculated based on the binary edge mask of the visible light image, and candidate welding head regions are extracted through Otsu segmentation to reconstruct the three-dimensional space. The world coordinates of the welding head are compared with a preset trajectory. If the deviation exceeds a set threshold, a safety warning is triggered, enabling real-time, accurate, and intelligent warning of welding gun deviations in a globally confined space.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and chemical production technology, and more specifically, relates to an AI analysis method for safety early warning of confined space operations. Background Art

[0002] In the production process of chemical plants, repair welding or replacement of linings inside high-pressure reactors are typical hot work operations in confined spaces. Operators need to carry welding torches into the reactor body. Faced with flammable or toxic residual media, the space is only wide enough for one person to operate, and the field of vision is limited. There are many blind spots in manual inspections, and the positioning of welding torch trajectories relies on experience, which can easily cause welds to deviate from the pre-planned path. This not only affects the quality of the welds, but also increases the risk of uneven heating, leakage, and even explosions in equipment and pipelines. With the development of industrial automation and AI technology, the introduction of online monitoring and early warning systems based on the fusion of video and sonar has become a key means to improve hot work safety and welding accuracy in confined spaces. This system aims to use the splicing of visible light and ultrasonic sensors to implement all-round, real-time spatial positioning and deviation warning of welding torch trajectories, providing intelligent safety protection for welding operations in high-pressure reactors.

[0003] Currently, welding in confined spaces in chemical plants mostly relies on manual or fixed-mounted single camera monitoring, combined with intercom systems and lighting assistance to observe the welding gun trajectory. Common practices are as follows:

[0004] (1) Manual inspection and fixed cameras: Several surveillance cameras are installed on the outer wall of the cauldron to monitor the welding process through observation windows or intercoms. The camera angle and field of view are fixed, and there are blind spots; it is difficult for the human eye to distinguish subtle track deviations.

[0005] (2) Simple optical sensing and laser indication: Use laser to aim at the target to assist in positioning. If the laser point deviates from the preset straight line, an alarm will be issued. However, the laser is easily scattered and distorted in areas obscured by smoke, steam, or high-temperature refraction, and the alarm accuracy is insufficient.

[0006] (3) Local temperature / infrared temperature measurement: Infrared thermometers or thermal imaging cameras are introduced to monitor the temperature distribution in the weld area to determine whether the welding is normal. However, it is difficult to directly quantify the offset distance of the welding gun relative to the planned trajectory.

[0007] (4) Sonar-assisted positioning attempts: Ultrasonic scanning is introduced into welding monitoring to compensate for the obstruction of visible light. However, most of them are single-point detection and are not synchronized with the camera, making it difficult to form continuous trajectory mapping.

[0008] In summary, existing technologies mainly rely on manual comparison and single sensing that is easily affected by the environment. Fixed or single-point sensors are difficult to cover the entire circumference of the inner wall of the kettle, and cannot provide early warning of welding gun deviation from the trajectory, resulting in insufficient monitoring of blind spots in confined spaces. In addition, due to the "fisheye" distortion caused by the air refraction layer around the welding arc, the optical image has a large deviation in capturing the actual position of the welding gun, which is more likely to cause refraction distortion problems. Finally, in the existing technology, optics and sonar are independent of each other and do not form a synchronous fusion data stream. The accuracy and timeliness of the determination of trajectory deviation are limited, and sensor fusion is relatively imperfect. Therefore, it is difficult for existing technologies to provide real-time and accurate intelligent early warning of welding gun deviation from the trajectory in a globally confined space. Summary of the Invention

[0009] In order to address the deficiencies in the prior art, the purpose of the present invention is to solve the above-mentioned defects and further propose an AI analysis method for safety warning of confined space operations.

[0010] The present invention adopts the following technical solutions.

[0011] The first aspect of the present invention discloses an AI analysis method for safety early warning of confined space operations, the method comprising:

[0012] Obtaining the size of the inner wall curved surface of the reactor, and cutting the flexible visual patch based on the size of the inner wall curved surface of the reactor to trigger a visual frame sequence;

[0013] Detecting local distortion areas of the visible light image in the visual frame sequence, and performing pixel correction on the detected local distortion areas according to a calibration mapping table;

[0014] Calling a pre-trained U-Net semantic segmentation model to process the visible light image, outputting a corresponding binary mask map, and determining whether to switch the visible light image to a sonar image based on the binary mask map;

[0015] Calculating a regional heat weight based on a binary edge mask of the visible light image, and extracting a welding head candidate region through Otsu segmentation to reconstruct a three-dimensional space;

[0016] Based on the three-dimensional space, the world coordinates of the welding head are compared with the preset trajectory. If the deviation exceeds the set threshold, a safety warning is triggered.

[0017] Furthermore, the obtaining of the size of the curved surface of the inner wall of the reactor and cutting the flexible visual patch based on the size of the curved surface of the inner wall of the reactor to trigger the visual frame sequence includes:

[0018] Cutting the flexible visual patch based on the size of the inner wall curved surface of the reactor to obtain a plurality of patch units, and constructing a flexible visual patch unit set based on the plurality of patch units;

[0019] PI optical fiber film is selected as the patch unit, and a micro CMOS image sensor unit is embedded at equal intervals in the center of one side of each patch unit, and an ultrasonic transmitting and receiving unit for obtaining sonar images is set in the center of the other side;

[0020] Each micro CMOS image sensor unit is led to a data convergence box outside the reaction vessel through a conduit, so as to integrate multiple image sensor units in the data convergence box.

[0021] Furthermore, the obtaining of the size of the curved surface of the inner wall of the reactor and cutting the flexible visual patch based on the size of the curved surface of the inner wall of the reactor to trigger the visual frame sequence further includes:

[0022] Dividing the reactor into multiple monitoring areas according to the weld locations and the operation path, and assigning initial monitoring priorities to the image sensor units according to the hole spacing between each image sensor unit and the operation path;

[0023] Obtaining the horizontal coordinates of the image sensor units numbered sequentially in the circumferential direction on the reactor at the same time point, and the vertical coordinates of the binary detection results of the current orientation or tilt angle of the welding gun by each image sensor unit, and determining a binary response curve based on the horizontal and vertical coordinates;

[0024] The image sensor unit at the welding gun position is determined based on the binarized response curve to preferentially acquire the visible light image captured by the image sensor unit at the welding gun position, and the visual frame sequence is constructed in combination with the visible light images captured by other image sensor units.

[0025] Furthermore, the detecting of local distortion areas of the visible light image in the visual frame sequence and performing pixel correction on the detected local distortion areas according to the calibration mapping table includes:

[0026] Extracting a corresponding grayscale value image from the visible light image, and generating a corresponding baseline gradient map based on the grayscale value image;

[0027] The baseline gradient image is compared with the real-time gradient image, and when the comparison deviation value exceeds a preset distortion threshold, the current pixel is determined to be a distorted pixel to determine the local distortion area.

[0028] Furthermore, the detecting of local distortion areas of the visible light image in the visual frame sequence and performing pixel correction on the detected local distortion areas according to the calibration mapping table further includes:

[0029] Obtain a pre-built calibration mapping table, wherein the calibration mapping table consists of a key and a value, and the value is equal to the difference between a pixel point of the distortion image and a pixel point of the corresponding calibration reference image;

[0030] Based on the keys and values ​​in the calibration mapping table, the distortion degree of each distorted pixel is calculated, and an index value is determined according to the distortion degree and a real-time gradient map corresponding to the distortion map and the calibration reference map, so as to correct the distorted pixel according to the index value.

[0031] Furthermore, calling a pre-trained U-Net semantic segmentation model to process the visible light image, outputting a corresponding binary mask image, and determining whether to switch the visible light image to a sonar image based on the binary mask image includes:

[0032] Obtaining image label pairs of smoke occlusion in a welding scene, and using the image label pairs as training data to jointly train the U-Net semantic segmentation model using a cross entropy loss function and a Dice coefficient to obtain the pre-trained U-Net semantic segmentation model;

[0033] Inputting the visible light image into the pre-trained U-Net semantic segmentation model to output a binary mask image of the same size as the visible light image, wherein the first value in the binary mask image represents smoke pixels and the second value represents clear areas;

[0034] The area with the largest grayscale value is extracted from the visible light image as the welding head bright spot, and when the welding head bright spot output is the first value, it is determined that the welding head is blocked and the corresponding sonar image is switched at the same time.

[0035] Furthermore, the calculation of the regional heat weight based on the binary edge mask of the visible light image and the extraction of the welding head candidate area by Otsu segmentation to reconstruct the three-dimensional space include:

[0036] Constructing a binary edge mask of the visible light image, and calculating a regional heat weight of the visible light image or the corresponding sonar image based on the binary edge mask;

[0037] Based on the regional heat weight, Otsu segmentation is used to extract the welding head associated area to construct a welding head candidate area set, and the weighted centroid of the largest area in the welding head candidate area set is calculated as the two-dimensional image coordinates of the welding head in the current frame;

[0038] The three-dimensional coordinates are reconstructed based on the two-dimensional image coordinates by using the triangulation principle, combining the world coordinates of the welding head and the least squares estimation.

[0039] A second aspect of the present invention discloses an AI analysis confined space operation safety warning device, which is used to implement the AI ​​analysis confined space operation safety warning method described in any one of the first aspects. The device includes:

[0040] A visual frame triggering module is used to obtain the size of the inner wall curved surface of the reactor and cut the flexible visual patch based on the size of the inner wall curved surface of the reactor to trigger the visual frame sequence;

[0041] a distortion correction module, configured to detect local distortion areas of the visible light image in the visual frame sequence, and perform pixel correction on the detected local distortion areas according to a calibration mapping table;

[0042] An image switching module is used to call a pre-trained U-Net semantic segmentation model to process the visible light image, output a corresponding binary mask image, and determine whether to switch the visible light image to a sonar image based on the binary mask image;

[0043] A three-dimensional space reconstruction module is used to calculate the regional heat weight based on the binary edge mask of the visible light image and extract the welding head candidate area through Otsu segmentation to reconstruct the three-dimensional space;

[0044] The safety warning trigger module is used to compare the world coordinates of the welding head with a preset trajectory based on the three-dimensional space, and trigger a safety warning if the deviation exceeds a set threshold.

[0045] A third aspect of the present invention discloses a terminal, comprising a processor and a storage medium; the terminal is characterized in that:

[0046] The storage medium is used to store instructions;

[0047] The processor is configured to operate according to the instructions to execute the steps of the method of the first aspect.

[0048] A fourth aspect of the present invention discloses a computer-readable storage medium having a computer program stored thereon, wherein the program implements the steps of the method described in the first aspect when executed by a processor.

[0049] The beneficial effects of the present invention are that, compared with the prior art, the present invention has the following advantages:

[0050] (1) The present invention obtains the dimensions of the inner wall of the reactor and cuts the flexible visual patch based on the dimensions of the inner wall of the reactor to construct a patch unit set to trigger a visual frame sequence. Combined with the visual frame sequence constructed by the image sensor unit, the present invention achieves the acquisition of multi-directional visible light images inside the reactor. At the same time, based on the visible light image, the sonar image conversion is combined. Regardless of how the visible light image and sonar sensor signal are blocked, the welding head position can be dynamically and accurately located, thus solving the technical defects of limited field of view due to the small container space and the large number of blind spots in manual inspection.

[0051] (2) The present invention detects the local distorted area of ​​the visible light image in the visual frame sequence, corrects the pixels of the detected local distorted area according to the calibration mapping table, processes the visible light image in combination with the pre-trained U-Net semantic segmentation model, and outputs the corresponding binary mask image. On the basis of all-round monitoring of the visual frame in the reactor, the local distorted area can be accurately located and corrected. Combined with the U-Net semantic segmentation model, the influence of the distorted image on the positioning accuracy of the welding head is effectively avoided, and the timeliness and intelligence of the safety warning triggering are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a flowchart of the AI ​​analysis confined space operation safety warning method provided by the present invention. DETAILED DESCRIPTION

[0053] The present application will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present application.

[0054] like Figure 1 As shown, in one embodiment, an AI analysis method for safety warning of confined space operations includes the following steps:

[0055] Step S110 , obtaining the size of the curved surface of the inner wall of the reactor, and cutting the flexible visual patch based on the size of the curved surface of the inner wall of the reactor to trigger a visual frame sequence.

[0056] In some embodiments, the AI ​​analysis method for safety early warning of confined space operations provided by the present invention, step S110 specifically includes the following steps:

[0057] Step S111 , cutting the flexible patch based on the size of the inner wall curved surface of the reactor to obtain a plurality of patch units, and constructing a flexible visual patch unit set based on the plurality of patch units.

[0058] In step S112, a PI optical fiber film is selected as a patch unit, and a micro CMOS image sensor unit is embedded at equal intervals in the center of one side of each patch unit, and an ultrasonic transmitting and receiving unit for acquiring sonar images is set in the center of the other side.

[0059] Step S113 , guiding each micro CMOS image sensor unit to a data convergence box outside the reaction vessel through a conduit, so as to integrate multiple image sensor units in the data convergence box.

[0060] In some embodiments, the AI ​​analysis method for safety early warning of confined space operations provided by the present invention, step S110 specifically further includes the following steps:

[0061] In step S114 , the reactor is divided into a plurality of monitoring areas according to the weld positions and the operation paths, and the image sensor units are assigned initial monitoring priorities according to the hole spacing between each image sensor unit and the welding path.

[0062] Step S115 , obtaining the horizontal coordinates of the image sensor units numbered sequentially in the circumferential direction on the reactor at the same time point, and the vertical coordinates of the binary detection results of the current orientation or tilt angle of the welding gun by each image sensor unit, and determining a binary response curve based on the horizontal and vertical coordinates.

[0063] Step S116 , determining the image sensor unit at the welding gun position based on the binarized response curve, so as to preferentially acquire the visible light image captured by the image sensor unit at the welding gun position, and constructing a visual frame sequence by combining the visible light images captured by other image sensor units.

[0064] In a specific embodiment, the AI ​​analysis confined space operation safety warning method provided by the present invention is designed to monitor the actual motion trajectory of the welding gun during operation in real time and immediately issue a prompt when it deviates from the pre-planned welding path to ensure weld quality and equipment safety. The method includes steps 1 to 5:

[0065] Step 1: Redundant flexible vision patches layout and synchronous triggering.

[0066] The specific steps include:

[0067] Step 1.1: Based on the size of the inner wall surface of the reactor, the flexible patch is cut to obtain a flexible visual patch unit set.

[0068] Specifically, the flexible visual patch unit set ,in, Indicates the SMD units, is the number of patch units.

[0069] The flexible visual patch unit should be a high-temperature and corrosion-resistant fiber optic film, for example, a polyimide-based fiber optic film (PI fiber optic film) coated with polytetrafluoroethylene (PTFE). The patch unit needs to reserve a fixed flange of 5mm width at the top to form a mechanical lock to ensure that the patch fits tightly to the curved surface. On each patch unit, a micro CMOS image sensor unit is embedded at equal intervals (such as in the center of the patch). , each The micro-flexible cable is routed through a high-temperature-resistant conduit to a data convergence box outside the reactor, where multiple image sensor units are integrated. This unit also provides a unified electrical connection to the flexible patch's signal port. Furthermore, an ultrasonic transceiver module (30° beam angle, 1 MHz frequency) is affixed parallel to the back of the patch at the same location. The module and the corresponding image sensor unit are aligned and coaxial.

[0070] Step 1.2: Based on the flexible visual patch unit set, the visual frame sequence is synchronously triggered.

[0071] Specifically, the number of image sensor units and ultrasonic transmitting and receiving units is less than or equal to In this example, 3 can be selected. It is required that during the dynamic process of welding the inner wall of a closed container, the number of the selected sensor units should be adjusted in real time according to the position of the welding head, thereby obtaining a visual frame sequence: , in this sequence, Indicates the Frame image set, , Respectively represent the first image captured by the image sensor units No. 1, 2, and 3. When the visible light image is unavailable, the corresponding sonar image should be switched to. The ultrasonic transmitting and receiving units No. 1, 2, and 3 capture the Frame sonar image.

[0072] Before the operation begins, the reactor's inner wall should be divided into several monitoring zones based on the weld location and the work path. Initial monitoring priorities should be pre-assigned to all image sensor units (image sensor units and ultrasonic transmitter / receiver units) based on their spatial distance from the weld path. However, in actual operation, the four images selected for a single frame do not necessarily come from the four highest-priority sensor units. Instead, they should correspond to the sensor units indicated by the local peaks on the binary response curve. The binary response curve here refers to a point in time where each sensor unit is arranged in order according to its circumferential number on the reactor wall. The vertical axis represents the binary detection result of each sensor unit for the current welding torch position or tilt angle (where 1 indicates a valid response exceeding the set threshold, and 0 indicates no response). When a local maximum of 1 appears at a certain location on the binary response curve, it indicates that the sensor unit has detected the most accurate welding torch position. The image captured by this sensor unit should be prioritized for subsequent processing. In this way, no matter how the visible light image or sonar sensor signal is blocked, the position of the welding head can be dynamically and accurately located.

[0073] Step S120 , performing local distortion area detection on the visible light image in the visual frame sequence, and performing pixel correction on the detected local distortion area according to the calibration mapping table.

[0074] In some embodiments, the AI ​​analysis method for safety early warning of confined space operations provided by the present invention, step S120 specifically includes the following steps:

[0075] Step S121 : extracting a corresponding grayscale image from the visible light image, and generating a corresponding baseline gradient map according to the grayscale image.

[0076] In step S122 , the baseline gradient image is compared with the real-time gradient image, and when the comparison deviation value exceeds a preset distortion threshold, the current pixel is determined to be a distorted pixel to determine a local distortion area.

[0077] In some embodiments, the AI ​​analysis method for safety early warning of confined space operations provided by the present invention, step S120 specifically further includes the following steps:

[0078] Step S123: Obtain a pre-built calibration mapping table, wherein the calibration mapping table is composed of a key and a value, and the value is equal to the difference between the pixel point of the distortion image and the pixel point of the corresponding calibration reference image.

[0079] In step S124, the distortion degree of each distorted pixel is calculated based on the keys and values ​​in the calibration mapping table, and an index value is determined according to the distortion degree and the real-time gradient map corresponding to the distortion map and the calibration reference map, so as to correct the distorted pixel according to the index value.

[0080] In a specific embodiment, the present invention provides an AI-based safety warning method for confined space operations. Step 2 involves real-time correction of localized visible light thermal refraction. High-temperature airflow (especially around the welding torch tip) causes air vibrations. These areas of airflow vibrations flicker in the visible light image, appearing as an unstable, blurred image. Therefore, refractive index correction is required to accurately determine the welding torch's position.

[0081] It should be noted that all visible light images actually only process the ROI (Region of Interest) area, that is, the approximate area of ​​the predicted coordinate position of the current welding gun position. For example, a rectangular area of ​​approximately 50×50 pixels centered on the predicted coordinate position is extracted as the ROI area.

[0082] The specific steps include:

[0083] Step 2.1, local distortion area detection.

[0084] Specifically, first, based on visible light images , extract the corresponding grayscale value image , here May refer to the aforementioned Any one of Indicates the number of frames. Then, based on the grayscale image, obtain its corresponding baseline gradient map , the expression is:

[0085] ;

[0086] Where, is the Sobel operator, Indicates the number of frames, You can select the frame value within 0.1s. Assuming that the frequency of the collected visible light image is 24 / s, then It can be rounded up to 3. Represents the horizontal and vertical coordinates of the pixel points in the image, Represents the grayscale image corresponding to the k-th frame.

[0087] Then, the baseline gradient map is compared with the real-time gradient map Compare and if it is greater than the preset distortion threshold, the corresponding pixel is determined to be distorted. The expression is:

[0088] ;

[0089] ;

[0090] Where, is the actual distortion value, For the distortion threshold, you can choose 0.05.

[0091] Step 2.2: Load and index the calibration mapping table.

[0092] Specifically, we first need to build a calibration mapping table, which can be obtained from relevant materials. The actual steps are as follows:

[0093] (1) Take a flat metal plate and spray a fixed grid (e.g. 20 mm × 20 mm squares) on it so that each intersection can be clearly seen.

[0094] (2) Install the same image sensor unit and fiber patch layout as the actual patch in front of the board.

[0095] (3) Prepare a set of controllable hot air guns or electric heaters that can generate hot air flows of different temperature levels in the same area of ​​the metal plate.

[0096] (4) No heat is applied, and only the image sensor unit is allowed to normally photograph the grid plate to obtain a calibration reference image.

[0097] (5) Corresponding to different areas, the hot air gun is aimed at the metal plate, and multiple distortion maps can be obtained at different reference temperatures; each distortion map corresponds to a different temperature.

[0098] (6) The calibration mapping table is composed of keys and value The composition is expressed as:

[0099] ;

[0100] ;

[0101] Where, is the label of the distortion map, To calibrate the pixel coordinates of the reference image, for In the The corresponding pixel coordinates in the distortion map.

[0102] Afterwards, for each pixel that is judged to be distorted , calculate the corresponding distortion , the expression is:

[0103] ;

[0104] ;

[0105] ;

[0106] ;

[0107] Where, is judged to be distorted and A collection of connected pixels, Indicates the The grayscale image of the distortion map, when hour, represents the grayscale image of the calibration reference image. Therefore, Respectively represent The real-time gradient map corresponding to the distortion map and the calibration reference map, is the reference distortion.

[0108] Finally, select closest , and thus determine the index value. After finding the index value, let ,in, is the pixel after correction, The value corresponding to the index value.

[0109] In step S130 , a pre-trained U-Net semantic segmentation model is called to process the visible light image, output a corresponding binary mask image, and determine whether to switch the visible light image to a sonar image based on the binary mask image.

[0110] In some embodiments, the AI ​​analysis method for safety early warning of confined space operations provided by the present invention, step S130 specifically includes the following steps:

[0111] Step S131: Obtain image label pairs of smoke occlusion in the welding scene, and use the image label pairs as training data to jointly train the U-Net semantic segmentation model using the cross entropy loss function and the Dice coefficient to obtain a pre-trained U-Net semantic segmentation model.

[0112] In step S132 , the visible light image is input into a pre-trained U-Net semantic segmentation model to output a binary mask image of the same size as the visible light image. The first value in the binary mask image represents smoke pixels, and the second value represents clear areas.

[0113] Step S133: extracting the area with the largest grayscale value from the visible light image as the welding head bright spot, and when the welding head bright spot outputs a first value, determining that the welding head is blocked, and switching to the corresponding sonar image at the same time.

[0114] In a specific embodiment, the AI-based confined space operation safety warning method provided by the present invention includes step 3, which involves image segmentation-based smoke detection and infrared parallel frame generation. Step 3, based on a U-Net model, determines whether to switch the visible light image to the corresponding sonar image.

[0115] The specific steps include:

[0116] In step 3.1, the ROI image is input into the trained U-Net semantic segmentation model. The model outputs a binary mask of the same size, where "1" represents smoke pixels and "0" represents clear areas.

[0117] It should be noted that the training data for the U-Net semantic segmentation model comes from image label pairs obscured by smoke in welding scenarios. The training is obtained by jointly (additively) optimizing the cross-entropy loss function and the Dice coefficient. The training data must cover scenarios with different temperatures, lighting conditions, and smoke concentrations.

[0118] Step 3.2: Determine whether to switch to the corresponding sonar image.

[0119] Specifically, in the original ROI image, the area with the maximum grayscale value is extracted as a candidate for the welding head bright spot (high-temperature metal welding heads have the strongest reflectivity under visible light); if the bright spot position is determined by the U-Net model to be a "smoke pixel" (that is, the mask is 1), the welding head is considered to be blocked, and the corresponding sonar image should be switched, otherwise no switching is performed.

[0120] In step S140 , the regional heat weight is calculated based on the binary edge mask of the visible light image, and the welding head candidate region is extracted by Otsu segmentation to reconstruct the three-dimensional space.

[0121] In some embodiments, the AI ​​analysis confined space operation safety warning method provided by the present invention, step S140 specifically includes the following steps:

[0122] Step S141 : constructing a binary edge mask of the visible light image, and calculating the regional heat weight of the visible light image or the corresponding sonar image based on the binary edge mask.

[0123] In step S142, based on the regional heat weight, Otsu segmentation is used to extract the welding head associated area to construct a welding head candidate area set, and the weighted centroid of the largest area in the welding head candidate area set is calculated as the two-dimensional image coordinates of the welding head in the current frame.

[0124] Step S143 , reconstructing the three-dimensional coordinates based on the two-dimensional image coordinates by combining the world coordinates of the welding head and the least squares estimation through the triangulation principle.

[0125] In a specific embodiment, the AI ​​analysis method for safety warning of confined space operations provided by the present invention includes step 4, binocular stereo reconstruction and three-dimensional fusion of key points.

[0126] The specific steps include:

[0127] Step 4.1, calculate the regional heat weight.

[0128] Specifically, for visible light images: construct a binary edge mask , the expression is:

[0129] ;

[0130] ;

[0131] Where, It is a preset global fixed threshold. Depending on the actual scenario, you can choose 0.2. Indicates other situations.

[0132] Afterwards, calculate the regional heat weight , the expression is:

[0133] ;

[0134] Where, Represents the normalized grayscale component, which can be set to 0.7; It is the grayscale image or sonar image corresponding to the visible light image.

[0135] Step 4.2: Extract the entire domain and screen the candidate region of the welding head to obtain the two-dimensional image coordinates of the welding head in the current frame.

[0136] Specifically, first, determine the set of candidate welding head regions: according to the regional heat weight, use Otsu segmentation (for example, cv2.threshold( , cv2.THRESH_OTSU) function), extract the area related to the welding head, each It's a whole connected area.

[0137] Then, choose the largest , calculate its weighted centroid As the two-dimensional image coordinates of the welding head in the current frame, the expression is:

[0138] ;

[0139] ;

[0140] Step 4.3: 3D space reconstruction and welding head coordinate calculation.

[0141] Specifically, the three-dimensional coordinate reconstruction is performed using the triangulation principle, that is, the least squares estimate of the following formula is solved:

[0142] ;

[0143] Where, It is pixels in the image In homogeneous coordinates, is the internal parameter matrix, are the rotation matrix and translation vector respectively, Indicates the splicing symbol, is the transpose of the matrix, Indicates the world coordinates of the welding head.

[0144] In step S150 , the world coordinates of the welding head are compared with the preset trajectory based on the three-dimensional space. If the deviation exceeds the set threshold, a safety warning is triggered.

[0145] In a specific embodiment, the AI ​​analysis confined space operation safety warning method provided by the present invention, step 5, safe operation warning.

[0146] Specifically, the world coordinates of the welding head are compared with the preset trajectory. If the deviation value is greater than the preset threshold, a safety warning is triggered.

[0147] The following describes the AI ​​analysis confined space operation safety warning device provided by the present invention. The AI ​​analysis confined space operation safety warning device described below and the AI ​​analysis confined space operation safety warning method described above can be referenced to each other.

[0148] In one embodiment, an AI analysis confined space operation safety warning device includes a visual frame trigger module, a distortion correction module, an image switching module, a three-dimensional space reconstruction module, and a safety warning trigger module.

[0149] The visual frame trigger module is used to obtain the size of the inner wall surface of the reactor and cut the flexible visual patch based on the size of the inner wall surface of the reactor to trigger the visual frame sequence.

[0150] The distortion correction module is used to detect local distortion areas in the visible light image in the visual frame sequence and perform pixel correction on the detected local distortion areas according to the calibration mapping table.

[0151] The image switching module is used to call the pre-trained U-Net semantic segmentation model to process the visible light image, output the corresponding binary mask map, and determine whether to switch the visible light image to the sonar image based on the binary mask map.

[0152] The three-dimensional space reconstruction module is used to calculate the regional heat weight based on the binary edge mask of the visible light image, and extract the welding head candidate area through Otsu segmentation to reconstruct the three-dimensional space.

[0153] The safety warning trigger module is used to compare the world coordinates of the welding head with the preset trajectory based on three-dimensional space. If the deviation exceeds the set threshold, a safety warning is triggered.

[0154] In this embodiment, the AI ​​analysis confined space operation safety warning device provided by the present invention, the visual frame trigger module is specifically used to:

[0155] The flexible patch is cut based on the size of the inner wall surface of the reactor to obtain a plurality of patch units, and a flexible visual patch unit set is constructed based on the plurality of patch units.

[0156] PI optical fiber film is selected as the patch unit, and a micro CMOS image sensor unit is embedded at equal intervals in the center of one side of each patch unit, and an ultrasonic transmitting and receiving unit for obtaining sonar images is set in the center of the other side.

[0157] Each micro CMOS image sensor unit is led to a data convergence box outside the reaction vessel through a catheter, so as to integrate multiple image sensor units in the data convergence box.

[0158] In this embodiment, the AI ​​analysis confined space operation safety warning device provided by the present invention, the visual frame trigger module is further used to:

[0159] The reactor is divided into multiple monitoring areas according to the weld position and the operation path, and the image sensor units are assigned initial monitoring priorities based on the hole spacing between each image sensor unit and the welding path.

[0160] The horizontal coordinates of the image sensor units numbered sequentially in the circumferential direction on the reactor at the same time point and the vertical coordinates of the binary detection results of the current orientation or tilt angle of the welding gun by each image sensor unit are obtained, and a binary response curve is determined based on the horizontal and vertical coordinates.

[0161] The image sensor unit at the welding gun position is determined based on the binarized response curve to preferentially obtain the visible light image captured by the image sensor unit at the welding gun position, and the visible light images captured by other image sensor units are combined to construct a visual frame sequence.

[0162] In this embodiment, the AI ​​analysis confined space operation safety warning device provided by the present invention, the distortion correction module is specifically used to:

[0163] A corresponding grayscale value image is extracted from the visible light image, and a corresponding baseline gradient map is generated according to the grayscale value image.

[0164] The baseline gradient image is compared with the real-time gradient image, and when the comparison deviation value exceeds a preset distortion threshold, the current pixel is determined to be a distorted pixel to determine the local distortion area.

[0165] In this embodiment, the AI ​​analysis confined space operation safety warning device provided by the present invention, the distortion correction module is specifically used to:

[0166] Get a pre-built calibration mapping table. The calibration mapping table consists of keys and values, and the value is equal to the difference between the pixel point of the distortion image and the pixel point of the corresponding calibration reference image.

[0167] Based on the keys and values ​​in the calibration mapping table, the distortion degree of each distorted pixel is calculated, and the index value is determined according to the distortion degree and the real-time gradient map corresponding to the distortion map and the calibration reference map, so as to correct the distorted pixel according to the index value.

[0168] In this embodiment, the AI ​​analysis confined space operation safety warning device provided by the present invention, the image switching module is specifically used to:

[0169] The image label pairs of smoke occlusion in the welding scene are obtained, and the image label pairs are used as training data to jointly train the U-Net semantic segmentation model using the cross entropy loss function and the Dice coefficient to obtain a pre-trained U-Net semantic segmentation model.

[0170] The visible light image is input into a pre-trained U-Net semantic segmentation model to output a binary mask of the same size as the visible light image. The first value in the binary mask represents the smoke pixel and the second value represents the clear area.

[0171] The area with the largest grayscale value is extracted from the visible light image as the welding head bright spot, and when the welding head bright spot output is a first value, it is determined that the welding head is blocked and the corresponding sonar image is switched at the same time.

[0172] In this embodiment, the AI ​​analysis confined space operation safety warning device provided by the present invention, the three-dimensional space reconstruction module is specifically used to:

[0173] Construct a binary edge mask of the visible light image and calculate the regional heat weight of the visible light image or the corresponding sonar image based on the binary edge mask.

[0174] Based on the regional heat weight, Otsu segmentation is used to extract the welding head associated area to construct a welding head candidate area set, and the weighted centroid of the largest area in the welding head candidate area set is calculated as the two-dimensional image coordinates of the welding head in the current frame.

[0175] The three-dimensional coordinates are reconstructed based on the two-dimensional image coordinates through the triangulation principle, combined with the world coordinates of the welding head and the least squares estimation.

[0176] The present disclosure may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0177] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punched card or raised structure in a groove on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse passing through a fiber optic cable), or an electrical signal transmitted through an electrical wire.

[0178] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0179] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, the state information of the computer-readable program instructions is used to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), so that the electronic circuit can execute the computer-readable program instructions, thereby implementing various aspects of the present disclosure.

[0180] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0181] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0182] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0183] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and the part of the module, program segment or instruction contains one or more executable instructions for realizing the prescribed logical function. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the prescribed function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0184] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. An AI analysis method for safety warning of confined space operations, characterized in that: The method comprises: Obtaining the size of the inner wall curved surface of the reactor, and cutting the flexible visual patch based on the size of the inner wall curved surface of the reactor to trigger a visual frame sequence; Detecting local distortion areas of the visible light image in the visual frame sequence, and performing pixel correction on the detected local distortion areas according to a calibration mapping table; Calling a pre-trained U-Net semantic segmentation model to process the visible light image, outputting a corresponding binary mask map, and determining whether to switch the visible light image to a sonar image based on the binary mask map; Calculating a regional heat weight based on a binary edge mask of the visible light image, and extracting a welding head candidate region through Otsu segmentation to reconstruct a three-dimensional space; Based on the three-dimensional space, the world coordinates of the welding head are compared with the preset trajectory. If the deviation exceeds a set threshold, a safety warning is triggered; The obtaining of the size of the inner wall curved surface of the reactor and cutting the flexible visual patch based on the size of the inner wall curved surface of the reactor to trigger the visual frame sequence includes: Dividing the reactor into multiple monitoring areas according to the weld locations and the operation path, and assigning initial monitoring priorities to the image sensor units according to the hole spacing between each image sensor unit and the operation path; Obtaining the horizontal coordinates of the image sensor units numbered sequentially in the circumferential direction on the reactor at the same time point, and the vertical coordinates of the binary detection results of the current orientation or tilt angle of the welding gun by each image sensor unit, and determining a binary response curve based on the horizontal and vertical coordinates; Determining the image sensor unit at the welding gun position based on the binarized response curve, so as to preferentially acquire the visible light image captured by the image sensor unit at the welding gun position, and constructing the visual frame sequence in combination with the visible light images captured by other image sensor units; The detecting of local distortion areas of the visible light image in the visual frame sequence and performing pixel correction on the detected local distortion areas according to a calibration mapping table includes: Obtain a pre-built calibration mapping table, wherein the calibration mapping table consists of a key and a value, and the value is equal to the difference between a pixel point of the distortion image and a pixel point of the corresponding calibration reference image; Calculating the distortion degree of each distorted pixel based on the keys and values ​​in the calibration mapping table, and determining an index value based on the distortion degree and a real-time gradient map corresponding to the distortion map and the calibration reference map, so as to correct the distorted pixel according to the index value; The method of calculating the regional heat weight based on the binary edge mask of the visible light image and extracting the welding head candidate area by Otsu segmentation to reconstruct the three-dimensional space includes: Constructing a binary edge mask of the visible light image, and calculating a regional heat weight of the visible light image or the corresponding sonar image based on the binary edge mask; Based on the regional heat weight, Otsu segmentation is used to extract the welding head associated area to construct a welding head candidate area set, and the weighted centroid of the largest area in the welding head candidate area set is calculated as the two-dimensional image coordinates of the welding head in the current frame; The three-dimensional coordinates are reconstructed based on the two-dimensional image coordinates by using the triangulation principle, combining the world coordinates of the welding head and the least squares estimation.

2. The AI ​​analysis confined space operation safety warning method according to claim 1 is characterized in that: The obtaining of the size of the inner wall curved surface of the reactor and cutting the flexible visual patch based on the size of the inner wall curved surface of the reactor to trigger the visual frame sequence includes: Cutting the flexible visual patch based on the size of the inner wall curved surface of the reactor to obtain a plurality of patch units, and constructing a flexible visual patch unit set based on the plurality of patch units; PI optical fiber film is selected as the patch unit, and a micro CMOS image sensor unit is embedded at equal intervals in the center of one side of each patch unit, and an ultrasonic transmitting and receiving unit for obtaining sonar images is set in the center of the other side; Each micro CMOS image sensor unit is led to a data convergence box outside the reaction vessel through a conduit, so as to integrate multiple image sensor units in the data convergence box.

3. The AI ​​analysis confined space operation safety warning method according to claim 1 is characterized in that: The detecting of local distortion areas of the visible light image in the visual frame sequence and performing pixel correction on the detected local distortion areas according to a calibration mapping table includes: Extracting a corresponding grayscale value image from the visible light image, and generating a corresponding baseline gradient map based on the grayscale value image; The baseline gradient image is compared with the real-time gradient image, and when the comparison deviation value exceeds a preset distortion threshold, the current pixel is determined to be a distorted pixel to determine the local distortion area.

4. The AI ​​analysis confined space operation safety warning method according to claim 1 is characterized in that: The calling of a pre-trained U-Net semantic segmentation model to process the visible light image, outputting a corresponding binary mask image, and determining whether to switch the visible light image to a sonar image according to the binary mask image includes: Obtaining image label pairs of smoke occlusion in a welding scene, and using the image label pairs as training data to jointly train the U-Net semantic segmentation model using a cross entropy loss function and a Dice coefficient to obtain the pre-trained U-Net semantic segmentation model; Inputting the visible light image into the pre-trained U-Net semantic segmentation model to output a binary mask image of the same size as the visible light image, wherein the first value in the binary mask image represents smoke pixels and the second value represents clear areas; The area with the largest grayscale value is extracted from the visible light image as the welding head bright spot, and when the welding head bright spot output is the first value, it is determined that the welding head is blocked and the corresponding sonar image is switched at the same time.

5. An AI analysis confined space operation safety warning device, characterized in that: The device is used to implement the AI ​​analysis confined space operation safety warning method according to any one of claims 1 to 4, comprising: A visual frame triggering module is used to obtain the size of the inner wall curved surface of the reactor and cut the flexible visual patch based on the size of the inner wall curved surface of the reactor to trigger the visual frame sequence; a distortion correction module, configured to detect local distortion areas of the visible light image in the visual frame sequence, and perform pixel correction on the detected local distortion areas according to a calibration mapping table; An image switching module is used to call a pre-trained U-Net semantic segmentation model to process the visible light image, output a corresponding binary mask image, and determine whether to switch the visible light image to a sonar image based on the binary mask image; A three-dimensional space reconstruction module is used to calculate the regional heat weight based on the binary edge mask of the visible light image and extract the welding head candidate area through Otsu segmentation to reconstruct the three-dimensional space; The safety warning trigger module is used to compare the world coordinates of the welding head with a preset trajectory based on the three-dimensional space, and trigger a safety warning if the deviation exceeds a set threshold.

6. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images