Large tank field safety monitoring method and device based on machine vision and electronic equipment

Abstract

The invention provides a large tank field safety monitoring method and device based on machine vision and electronic equipment, and the method comprises the steps: obtaining a tank field device image of a tank field device in a tank field through employing a plurality of cameras which are installed in advance; establishing global reference association between the space coordinate system and the camera coordinate system through the homogeneous transformation matrix; acquiring safety monitoring parameters of the tank area device in the tank area according to the tank area device image on the basis of global reference association; and determining a tank field safety monitoring result according to the safety monitoring parameters of the tank field device. According to the method, global reference association between a space coordinate system and a camera coordinate system is established through a homogeneous transformation matrix, and a unified reference frame is established in the whole tank field range, so that the position and direction of a camera can be accurately acquired, accurate measurement of safety monitoring parameters is performed according to tank field device images, more comprehensive safety monitoring is realized, and the safety monitoring efficiency is improved. The system can adapt to the rotation or movement of the camera, enables the safety monitoring to be more flexible and reliable, does not need a large number of sensors in the process, and reduces the cost.

Description

Technical Field

[0001] The present invention relates to the technical field of tank area safety monitoring, and in particular to a large tank area safety monitoring method, device and electronic equipment based on machine vision. Background Art

[0002] With the development of industrial automation and intelligence, the safety monitoring technology of large tank farms has attracted more and more attention. In order to ensure the safety of tank farms, various means such as sensor networks are often used to monitor tank farms in real time. However, there are some problems in the actual application of existing technologies, which limits their further promotion and application in the field of safety monitoring of tank farms.

[0003] Specifically, some existing technologies use sensor networks for monitoring. By deploying smoke sensors, temperature sensors and other equipment in the tank area, various environmental parameters in the tank area are monitored in real time. This method can timely understand the safety status in the tank area, but it has the following limitations: (1) A large number of sensors are required: In order to improve the accuracy and coverage of monitoring, a large number of sensors are often required, which increases the complexity and cost of the system; (2) Frequent inspection and maintenance: The sensor network needs to be regularly inspected and maintained to ensure the accuracy of the monitoring data. This brings a certain burden to the operators and increases the operating cost of the system.

[0004] In addition, some existing technologies use machine vision for monitoring, obtain real-time images of the environment in the tank area through image acquisition equipment, and use image processing technology to analyze and identify the images. This method can achieve real-time monitoring of targets in the tank area, but there are the following problems: (1) Sensitive to ambient light: In outdoor tank area monitoring, changes in light will affect image quality, thereby reducing detection and recognition accuracy; (2) Complex background interference: Large tank areas have complex background interference, such as pipelines, equipment, personnel, etc. These interferences are difficult to distinguish from target objects, affecting the monitoring effect.

[0005] At the same time, with the release of the new national "ambient air quality standards", volatile organic compounds (VOCs) emission control has also received more and more attention. Leakage and volatile organic compound detection and monitoring in large tank areas have also become more important. The traditional method is to judge by collecting physical parameters of volatile organic compounds or medium leakage. Regular collection can obtain the safety status of the tank area, but it is impossible to achieve real-time monitoring. Machine vision technology has 24-hour uninterrupted monitoring. With infrared imaging cameras and improved image processing algorithms, it can achieve remote, non-contact leakage monitoring of tank areas, pipelines, equipment, etc.

[0006] In summary, it is necessary to solve the problem that the existing tank area safety monitoring methods require high costs and have poor monitoring effects. Summary of the invention

[0007] The present invention provides a large tank area safety monitoring method, device and electronic equipment based on machine vision, which are used to overcome the defects of the existing tank area safety monitoring method that requires high cost and poor monitoring effect, so as to improve the accuracy and stability of tank area monitoring.

[0008] On the one hand, the present invention provides a large tank area safety monitoring method based on machine vision, comprising: using multiple pre-installed cameras to obtain tank area device images of tank area devices in the tank area; establishing a global reference association between a spatial coordinate system and a camera coordinate system through a homogeneous transformation matrix; based on the global reference association, obtaining safety monitoring parameters of the tank area devices in the tank area according to the tank area device images; and determining tank area safety monitoring results according to the safety monitoring parameters of the tank area devices.

[0009] Furthermore, the method of acquiring the tank area device images of the tank area devices in the tank area by using the pre-installed multiple cameras includes: acquiring the tank area device images when the multiple cameras are dynamically operated and / or the tank area devices are spatially moved.

[0010] Furthermore, the global reference association between the spatial coordinate system and the camera coordinate system is established through the homogeneous transformation matrix, including: translating the origin of the spatial coordinate system to the position of the camera through the translation matrix; rotating the spatial coordinate system to align with the camera coordinate system through the rotation matrix; combining the translation matrix and the rotation matrix to obtain a homogeneous transformation matrix; and establishing a global reference association from the spatial coordinate system to the camera coordinate system based on the homogeneous transformation matrix.

[0011] Further, the camera is a thermal infrared camera, and the tank area device image is a thermal infrared image; based on the global reference association, the safety monitoring parameters of the tank area device in the tank area are obtained according to the tank area device image, including: obtaining the temperature values ​​of each part of the tank area device in the thermal infrared image, and the temperature difference between adjacent parts of the tank area device; wherein the safety monitoring parameters include the temperature value and the temperature difference; correspondingly, determining the tank area safety monitoring result according to the safety monitoring parameters of the tank area device, including: determining a first part whose temperature value is higher than a first set threshold; and / or determining a second part whose temperature value is lower than a second set threshold; and / or determining a third part whose temperature difference is higher than a preset difference; determining the first part, the second part, and the third part as parts where there are signs of gas or liquid leakage, and obtaining the tank area safety monitoring result; wherein the first set threshold is greater than the second set threshold.

[0012] Furthermore, based on the global reference association, the safety monitoring parameters of the tank area devices in the tank area are obtained according to the tank area device image, including: performing edge detection on the tank area device image to obtain a crack contour; extracting crack characteristic parameters from the crack contour, the crack characteristic parameters including the position, length, width, direction and area of ​​the crack; wherein the safety monitoring parameters include the crack characteristic parameters; correspondingly, determining the tank area safety monitoring result according to the safety monitoring parameters of the tank area device, including: determining the risk level of the tank area device according to the crack characteristic parameters; determining that the risk level reaches a set level; determining the position of the crack in the crack characteristic parameters as a location that needs maintenance, repair or replacement, to obtain the irrigation area safety monitoring result.

[0013] Furthermore, it also includes: using multiple pre-installed cameras to obtain a target image of the operator in the tank area; determining the operator's authorization information and wearable equipment based on the target image; based on the global reference association, determining the operator's actual position in the tank area based on the target image; if any one of the operator's authorization information, wearable equipment and actual position does not meet the preset conditions, an alarm is issued to the security personnel in the tank area, so that safety measures can be taken in time.

[0014] In a second aspect, the present invention also provides a large tank area safety monitoring device based on machine vision, including: a tank area device image acquisition module, used to use multiple pre-installed cameras to acquire tank area device images of tank area devices in the tank area; a global reference association establishment module, used to establish a global reference association between the spatial coordinate system and the camera coordinate system through a homogeneous transformation matrix; a safety monitoring parameter acquisition module, used to acquire the safety monitoring parameters of the tank area devices in the tank area based on the tank area device images based on the global reference association; a safety monitoring result determination module, used to determine the tank area safety monitoring results based on the safety monitoring parameters of the tank area devices.

[0015] In a third aspect, the present invention also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a large tank area safety monitoring method based on machine vision as described above is implemented.

[0016] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described large tank area safety monitoring methods based on machine vision.

[0017] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-described large tank area safety monitoring methods based on machine vision.

[0018] The large tank area safety monitoring method based on machine vision provided by the present invention obtains the tank area device image of the tank area device in the tank area by using multiple pre-installed cameras, and establishes a global reference association between the spatial coordinate system and the camera coordinate system through a homogeneous transformation matrix, and then obtains the safety monitoring parameters of the tank area device in the tank area based on the tank area device image, thereby determining the tank area safety monitoring result based on the safety monitoring parameters of the tank area device. The method establishes a global reference association between the spatial coordinate system and the camera coordinate system through a homogeneous transformation matrix, and establishes a unified reference frame within the entire tank area. It can not only accurately obtain the position and direction of the camera, but also accurately measure the safety monitoring parameters according to the tank area device image to achieve more comprehensive safety monitoring, and can also adapt to the rotation or movement of the camera, making the safety monitoring more flexible and reliable. At the same time, this process does not require the deployment of a large number of sensors, reducing the cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a flow chart of a large tank area safety monitoring method based on machine vision provided in an embodiment of the present invention.

[0021] Figure 2 It is a structural schematic diagram of a large tank area safety monitoring device based on machine vision provided in an embodiment of the present invention.

[0022] Figure 3 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] It should be noted that the existing tank area safety monitoring methods have the following limitations: (1) The monitoring system is highly complex and costly: The existing technology has high complexity in terms of monitoring system and sensor layout, which leads to high system costs; (2) Data processing and analysis are difficult: The existing technology requires professionals to interpret and analyze data, which is difficult for non-professional operators and increases the deployment and maintenance costs of the system; (3) It is sensitive to ambient light and the monitoring effect is unstable: In outdoor tank area monitoring, changes in light may cause image quality to deteriorate and affect the monitoring effect; (4) Complex background interference is serious and target identification is difficult: In large tank area monitoring, background interference may lead to inaccurate target identification and increase the risk of false alarms and missed alarms; (5) Data collection is difficult: Tank areas are usually located in difficult-to-access areas, which makes data collection a major problem.

[0025] Considering this, the present invention proposes a new large tank area safety monitoring method based on machine vision. Specifically, Figure 1 A schematic flow chart of a large tank area safety monitoring method based on machine vision provided in an embodiment of the present invention is shown.

[0026] like Figure 1 As shown, the method includes steps S110-S140, and steps S110-S140 and related steps will be described in detail below.

[0027] S110, using a plurality of pre-installed cameras, acquiring tank area device images of tank area devices in the tank area.

[0028] Understandably, in order to monitor the tank farm, it is first necessary to install multiple cameras inside or around the tank farm to achieve full monitoring coverage.

[0029] When installing a camera, it is necessary to clearly define the camera's monitoring target, which includes but is not limited to personnel activities in the tank area, the status of tank area equipment, and the status of other equipment outside the tank area equipment.

[0030] At the same time, it is also necessary to focus on high-risk areas within the tank area, such as storage areas for flammable and explosive materials, key equipment and high-traffic areas, especially around the tank body, pipeline intersections, entrances and exits, etc.

[0031] Regarding the cameras to be installed, you can choose cameras with high resolution, wide dynamic range, and low light sensitivity that are suitable for industrial environments, and if necessary, they can be equipped with thermal infrared night vision functions.

[0032] The number of cameras installed can be set according to the monitoring coverage requirements and is not specifically limited here.

[0033] After the required number of cameras are installed, they can capture images of the tank farm equipment (tank farm equipment images) in real time and have the ability to sense position and direction.

[0034] It should be noted that the image acquisition device (camera) installed in this step can be fixed or rotatable, and no specific limitation is made here.

[0035] The collected images of tank farm equipment include but are not limited to volatile organic compounds, small leaks in storage tanks, and leak monitoring of buried facilities.

[0036] It should also be noted that this step uses multiple pre-installed cameras to obtain images of the tank area devices in the tank area. This can be uninterrupted 24 hours a day. The existing video surveillance equipment (cameras) installed in the tank area can be used to achieve artificial intelligence monitoring of the tank area without increasing costs. It can effectively guide on-site maintenance operations, reasonably plan the frequency of maintenance and inspection, and reduce the dimension of the selected key sensitive indicators of pipeline operation and maintenance safety. It can improve the accuracy of maintenance, improve the operating efficiency of on-site personnel, and effectively reduce maintenance costs. The prediction method forms a technical document for easy use by on-site staff.

[0037] S120, establishing a global reference association between the space coordinate system and the camera coordinate system through a homogeneous transformation matrix.

[0038] It can be understood that, based on the use of multiple pre-installed cameras to obtain the tank area device images in the tank area in step S110, further, step S120 will establish a global reference association between the space coordinate system and the camera coordinate system through a homogeneous transformation matrix.

[0039] Specifically, this embodiment uses homogeneous coordinate technology to represent three-dimensional positions and transformation relationships. Homogeneous coordinates can uniformly represent transformation operations such as translation, rotation, and scaling, thereby simplifying the conversion and alignment process between the space coordinate system and the camera coordinate system.

[0040] During the conversion and alignment process, the origin of the spatial coordinate system is first moved to the origin of the camera coordinate system, and then the spatial coordinate system is rotated and translated to make the spatial coordinates completely consistent with the camera coordinate system. In this way, a global reference association is established to ensure the accurate position and orientation when obtaining the tank area device image from the 3D scene.

[0041] Among them, the homogeneous transformation matrix is ​​a mathematical tool used to describe the transformation of the coordinates of a point in three-dimensional space between different coordinate systems. In a homogeneous coordinate system, a three-dimensional vector is represented as a four-dimensional vector , so that a matrix can be used to represent the translation and rotation operations from the space coordinate system to the camera coordinate system.

[0042] The spatial coordinate system is a fixed reference system in the tank area safety monitoring scenario, which can be a geodetic coordinate system or a coordinate system selected in the tank area.

[0043] The camera coordinate system is the local coordinate system of the camera, whose origin is located at the optical center of the camera. The x-axis and y-axis define the image plane of the camera, and the z-axis points to the direction of the optical axis of the camera.

[0044] Establishing a global reference relationship between the spatial coordinate system and the camera coordinate system will involve the conversion between the spatial coordinate system and the camera coordinate system, which requires knowing the position and posture of the camera relative to the spatial coordinate system, which is the camera intrinsic calibration and camera extrinsic calibration.

[0045] The camera's intrinsic parameters include focal length, principal point position, and distortion parameters. These parameters determine the camera's imaging model, which is the projection process from three-dimensional space to two-dimensional images. Intrinsic parameter calibration is usually performed using a checkerboard or other calibration pattern of known geometric shape.

[0046] The camera extrinsic parameters describe the translation and rotation of the camera coordinate system relative to the spatial coordinate system. The extrinsic parameters can be calibrated by placing some feature points with known positions in the spatial coordinate system, finding the corresponding points in the camera image, and then solving them using the least squares method or other optimization algorithms.

[0047] Then, according to the translation and rotation of the camera coordinate system relative to the spatial coordinate system, a homogeneous transformation matrix is ​​constructed. Through the homogeneous transformation matrix, the points in the spatial coordinate system can be transformed into the camera coordinate system, which means that a global reference association between the spatial coordinate system and the camera coordinate system is established, so that the corresponding position can be found in the acquired tank area device image.

[0048] S130, based on the global reference association and according to the tank area device image, obtaining safety monitoring parameters of the tank area devices in the tank area.

[0049] It can be understood that, based on the establishment of a global reference association between the spatial coordinate system and the camera coordinate system through the homogeneous transformation matrix in step S120, further, step S130 will obtain the safety monitoring parameters of the tank area device in the tank area according to the tank area device image based on the global reference association.

[0050] Specifically, by calibrating and aligning the spatial coordinate system with the camera coordinate system, the image of the tank area device can be accurately measured. No matter how the camera rotates or moves, the position of the monitored object (such as the tank area device in the tank area) in space can be accurately determined, thereby achieving accurate and safe monitoring of the tank area.

[0051] Accurately measure the tank area device image. Specifically, use the edge detection algorithm to highlight the edges of the tank area device, pipelines, valves and other equipment in the tank area device image. Use contour analysis to identify and locate the shapes of the tank area device, valves, pipelines and other equipment. Analyze the texture of the surface of the tank area device to detect possible cracks, corrosion and other abnormal conditions. Color information can also be used to identify signs of emergency situations such as leakage and fire.

[0052] In addition to the status monitoring of the above-mentioned static equipment, the tank area device images can also be used to identify the movement of people or vehicles in the tank area, identify unsafe behaviors of people in the tank area (such as not wearing protective equipment, illegal operations, etc.), and monitor the environmental conditions of the tank area (such as abnormal conditions such as smoke and flames).

[0053] Based on this, a series of safety monitoring parameters of the tank area equipment in the tank area can be obtained, including but not limited to the liquid level parameters (liquid level height), valve status (valve opening and closing status), pipeline connection status (tightness of pipeline connection), structural integrity parameters (whether cracks or corrosion occur), moving object / personnel status and tank area environmental parameters.

[0054] It should be noted that when obtaining the safety monitoring parameters of the tank area device in the tank area, it is necessary to determine the accurate position and direction of the tank area device and related equipment in the three-dimensional scene in the tank area device image based on the global reference association, which is crucial and cannot be ignored.

[0055] It is worth mentioning that this step uses homogeneous coordinate transformation to calibrate the errors introduced by the image acquisition device during lateral movement or pan / tilt rotation. The global reference association algorithm captures the same reference points of the asynchronous images. The calibration image and matching algorithm can effectively improve the image acquisition device's accurate positioning of oil and gas leaks and volatile organic compounds during rotation.

[0056] S140, determining a tank area safety monitoring result according to the safety monitoring parameters of the tank area device.

[0057] It can be understood that, based on the global reference association, step S130 obtains the safety monitoring parameters of the tank area devices in the tank area according to the tank area device image, and further, step S140 determines the final tank area safety monitoring result according to the safety monitoring parameters of the tank area devices.

[0058] Specifically, after obtaining the safety monitoring parameters of the tank area equipment in the tank area, these safety monitoring parameters are analyzed and compared with the pre-set normal operating parameter range. If the safety monitoring parameters exceed the normal operating parameter range, it indicates that there may be unsafe risks in the tank area equipment and response measures need to be formulated in a timely manner.

[0059] Among them, the tank area safety monitoring result can be a binary classification result, that is, the tank area device in the current tank area is in an abnormal state, or the tank area device in the current tank area is in a safe state.

[0060] The tank area safety monitoring result can also be a multi-classification result, including the possible risks and corresponding probabilities of the tank area devices in the tank area. For example, there is a 90% probability that the tank area device at position A in the current tank area has a risk of gas or liquid leakage. In this embodiment, by using multiple pre-installed cameras, the tank area device image of the tank area device in the tank area is obtained, and a global reference association between the spatial coordinate system and the camera coordinate system is established through a homogeneous transformation matrix, and then based on the global reference association, the safety monitoring parameters of the tank area device in the tank area are obtained according to the tank area device image, so that the tank area safety monitoring result is determined according to the safety monitoring parameters of the tank area device. This method establishes a global reference association between the spatial coordinate system and the camera coordinate system through a homogeneous transformation matrix, and establishes a unified reference frame within the entire tank area. It can not only accurately obtain the position and direction of the camera, but also accurately measure the safety monitoring parameters according to the tank area device image to achieve more comprehensive safety monitoring, and can also adapt to the rotation or movement of the camera, making the safety monitoring more flexible and reliable. At the same time, this process does not require the deployment of a large number of sensors, reducing the cost.

[0061] On the basis of the above-mentioned embodiment, the process of acquiring the image of the tank area device will be described in detail below.

[0062] Using a plurality of pre-installed cameras, images of tank area devices in the tank area are acquired, including: acquiring images of the tank area devices when the plurality of cameras are dynamically operated and / or the tank area devices are spatially moved.

[0063] It can be understood that in this embodiment, the monitored objects (tank area devices and other equipment in the tank area) are allowed to move freely in space without being affected by the rotation or movement of the camera. Even if the camera is transformed, based on the global reference association between the established spatial coordinate system and the camera coordinate system, the tank area devices and other equipment in the tank area can still be accurately tracked and measured, thereby improving the flexibility of monitoring.

[0064] On the basis of the above-mentioned embodiment, the process of establishing a global reference association will be described in detail below.

[0065] A global reference association between the spatial coordinate system and the camera coordinate system is established through the homogeneous transformation matrix, including: translating the origin of the spatial coordinate system to the position of the camera through the translation matrix; rotating the spatial coordinate system to align with the camera coordinate system through the rotation matrix; combining the translation matrix and the rotation matrix to obtain a homogeneous transformation matrix; and establishing a global reference association from the spatial coordinate system to the camera coordinate system based on the homogeneous transformation matrix.

[0066] It can be understood that global reference association is the process of aligning different coordinate systems (such as the spatial coordinate system and the camera coordinate system) so that they can be compared and operated under a unified reference frame. The global reference association between the spatial coordinate system and the camera coordinate system is established through the homogeneous transformation matrix, which involves the conversion between the spatial coordinate system and the camera coordinate system, and the conversion between the coordinate systems involves translation and rotation to ensure that the origins of the spatial coordinate system and the camera coordinate system coincide and that their directions are consistent.

[0067] Specifically, first, the origin of the spatial coordinate system needs to be moved to the origin of the camera coordinate system. This is because the origin of the camera coordinate system is usually the optical center of the camera, that is, the point where all light rays converge. The origin movement can be achieved through the translation matrix, and the translation matrix can be specifically referred to in the following formula (1).

[0068] (1).

[0069] In formula (1), is the position of the camera in the spatial coordinate system.

[0070] Then, in order to make the spatial coordinate system consistent with the direction of the camera coordinate system, the spatial coordinate system needs to be rotated to align with the camera coordinate system, which involves rotation around three axes (X axis, Y axis, Z axis) to match the direction of the camera. The rotation alignment process can be achieved by rotating the coordinate system. The rotation matrix can be obtained by multiplying the three basic rotation matrices. For details, see the following equations (2)-(5).

[0071] (2).

[0072] (3).

[0073] (4).

[0074] (5).

[0075] In equations (2)-(5), , , Respectively represent the basic rotation matrices of the spatial coordinate system rotating around the X-axis, Y-axis, and Z-axis. Indicates the pitch angle of the spatial coordinate system rotating around the X axis. Indicates the yaw angle of the spatial coordinate system rotating around the Y axis, Indicates the roll angle of the spatial coordinate system around the Z axis.

[0076] Next, the translation matrix and the rotation matrix are combined to obtain the homogeneous transformation matrix, which contains both rotation and translation information. The homogeneous transformation matrix can be seen in the following equation (6).

[0077] (6).

[0078] Once you have the homogeneous transformation matrix, you can use it to transform any point in the spatial coordinate system To the camera coordinate system. For a given point in the space coordinate system, its position in the camera coordinate system It can be expressed by the following formula (7).

[0079] (7).

[0080] In this way, the conversion from the spatial coordinate system to the camera coordinate system is completed, and a global reference association from the spatial coordinate system to the camera coordinate system is established, which can accurately describe and calculate the position and posture of the monitored object in the camera field of view.

[0081] By establishing the above-mentioned global reference association, it can be ensured that the image obtained from the three-dimensional scene can accurately reflect the real position and direction of the object in the scene.

[0082] On the basis of the above-mentioned embodiment, the process of finally determining the tank area safety monitoring result by using the tank area device image will be described in detail below.

[0083] In a specific embodiment, the camera is a thermal infrared camera, and the tank area device image is a thermal infrared image.

[0084] Correspondingly, based on the global reference association, the safety monitoring parameters of the tank area equipment in the tank area are obtained according to the tank area equipment image, including: obtaining the temperature value of each part of the tank area equipment in the thermal infrared image, and the temperature difference of adjacent parts on the tank area equipment; wherein, the safety monitoring parameters include temperature value and temperature difference.

[0085] It is easy to understand that the use of thermal infrared cameras to monitor the safety of tank area equipment is mainly to detect temperature anomalies through the analysis of thermal infrared images, so as to determine whether there is a potential gas or liquid leak.

[0086] Among them, the thermal infrared camera can detect the infrared radiation emitted by the object, thereby calculating the surface temperature of the tank area device. By analyzing the thermal infrared image, the temperature value of each part of the tank area device can be obtained.

[0087] Compare the temperatures of adjacent locations on the tank farm installation and calculate the temperature differential. Abnormally high temperature differentials may indicate uneven heat distribution, suggesting a possible leak or other problem.

[0088] After the temperature values ​​of various parts of the tank area device and the temperature difference between adjacent parts are monitored, the tank area safety monitoring results are further determined based on the safety monitoring parameters of the tank area device.

[0089] Specifically, determine a first location where the temperature value is higher than a first set threshold; and / or determine a second location where the temperature value is lower than a second set threshold; and / or determine a third location where the temperature difference is higher than a preset difference; determine the first location, the second location, and the third location as locations where there are signs of gas or liquid leakage, and obtain a tank area safety monitoring result; wherein the first set threshold is greater than the second set threshold.

[0090] Specifically, a higher temperature threshold, namely the first set threshold, is set to detect an overheated area, which may indicate a potential fire risk or overheating failure in the tank farm.

[0091] A lower temperature threshold, the second set threshold, is set to detect an overcooling region, which may indicate a cooling system failure or an unexpected temperature drop.

[0092] A temperature difference threshold, i.e. a preset difference value, is set to detect areas with excessive temperature gradients, which may be a direct effect of a leak.

[0093] When the temperature value of the first part of the tank farm device is higher than the first set threshold value, the first part may be at risk of overheating, and is determined as an abnormal part.

[0094] When the temperature value of the second part of the tank area device is lower than the second set threshold, the second part may have a cooling problem or a temperature drop caused by liquid leakage, and it is determined as an abnormal part.

[0095] When the temperature difference between adjacent parts of the tank area device is higher than the preset difference, there may be gas or liquid leakage in the adjacent parts, resulting in sudden changes in local temperature.

[0096] Among them, the first set threshold is greater than the second set threshold. Under this premise, the first set threshold and the second set threshold can be set according to actual conditions and are not specifically limited here.

[0097] Likewise, the preset difference value may also be set according to actual conditions, and is not specifically limited here.

[0098] After monitoring, if any one of the above-mentioned first, second and third parts exists, it is considered that there may be signs of gas or liquid leakage in the tank area. At this time, the abnormal area (any one or more of the first, second and third parts) and its corresponding temperature value are the tank area safety monitoring results.

[0099] Of course, in addition to abnormal areas and temperature values, corresponding maintenance suggestions / treatment measures can also be proposed for this risk and incorporated into the tank area safety monitoring results.

[0100] In this embodiment, by monitoring the tank area with a thermal infrared camera, potential safety hazards of the tank area equipment can be effectively identified, helping staff to take timely measures to prevent accidents.

[0101] In another specific embodiment, based on the global reference association, the safety monitoring parameters of the tank area equipment in the tank area are obtained according to the tank area equipment image, including: performing edge detection on the tank area equipment image to obtain a crack contour; extracting crack characteristic parameters from the crack contour, the crack characteristic parameters including the position, length, width, direction and area of ​​the crack; wherein the safety monitoring parameters include the crack characteristic parameters.

[0102] It is understood that after obtaining the tank area device image, a filter can be first applied to remove random noise in the tank area device image to ensure the accuracy of subsequent edge detection. At the same time, the color tank area device image can be converted into a grayscale image to simplify calculation and improve the efficiency of subsequent edge detection.

[0103] Then, edge detection is performed on the tank area device image to obtain the crack contour. Specifically, the Canny edge detection algorithm can be used to find the boundaries with significant intensity changes in the tank area device image, which can effectively identify the crack contour.

[0104] After edge detection, the boundaries of the cracks are found to form a closed crack outline, which is then filled to separate it from the background for further analysis.

[0105] Next, the specific location coordinates of the crack in the tank area device are recorded, and the total length of the crack, the width of the widest part of the crack, the direction of the crack and the total area covered by the crack are accurately measured to obtain the crack characteristic parameters, that is, the safety monitoring parameters.

[0106] After obtaining the safety monitoring parameters, the tank area safety monitoring results are determined based on the safety monitoring parameters of the tank area equipment. Specifically, the risk level of the tank area equipment is determined based on the crack characteristic parameters; it is determined that the risk level reaches the set level; the location of the cracks in the crack characteristic parameters is determined as a location that needs maintenance, repair or replacement, and the irrigation area safety monitoring results are obtained.

[0107] In the following, according to the location, length, width, direction and area of ​​the cracks, combined with the material properties and tank farm environment, the impact of the cracks on the structural safety of the tank farm equipment is evaluated, and then the risk level is determined. The higher the risk level, the greater the unsafe risk, and the more attention it needs.

[0108] The assessed risk level is compared with the set level to determine whether it is higher than the set level. If the assessed risk level is greater than or equal to the set level, the crack location with a high risk level will be marked as an abnormal area that requires special attention, and it will be identified as an abnormal part that requires maintenance, repair or replacement to obtain the tank area safety monitoring results.

[0109] The setting level can be set according to actual needs and is not specifically limited here.

[0110] In this embodiment, by performing edge detection and feature extraction on the tank farm device image, the structural health status of the tank farm device can be effectively monitored, and potential safety hazards can be discovered in advance, thereby reducing the possibility of accidents.

[0111] On the basis of the above-mentioned embodiment, the monitoring process of operators in the tank farm will be described in detail below.

[0112] Utilize multiple pre-installed cameras to obtain target images of operators in the tank area; determine the operator's authorization information and wearable equipment based on the target image; determine the operator's actual location in the tank area based on the target image based on the global reference association; if any of the operator's authorization information, wearable equipment, and actual location does not meet the preset conditions, send an alarm to the security personnel in the tank area so that safety measures can be taken in a timely manner.

[0113] It is understandable that multiple high-definition cameras are installed at key locations in the tank farm to ensure full coverage of all angles so as to capture images of operators in the tank farm from different angles.

[0114] Then, the operator in the target image is compared with the personnel information in the pre-stored personnel database. When the similarity reaches the similarity threshold, it can be determined that the operator in the target image corresponds to the personnel information in the pre-stored personnel database, and then the identity information and authorization information of the operator in the target image are determined.

[0115] At the same time, the operator's wearable equipment can also be obtained from the target image to determine whether the operator has correctly worn the necessary personal protective equipment, such as a helmet, protective clothing, and respirator.

[0116] Based on the established global reference association, combined with the camera's position information and the operator's relative position in the target image, the operator's actual position can be accurately obtained.

[0117] The authorization information, wearable devices and actual location are compared with the corresponding preset conditions. If any one of them does not meet the preset conditions, an alarm will be promptly issued to the security personnel in the tank area, triggering the corresponding emergency response process and taking relevant safety measures.

[0118] Regarding non-compliance with preset conditions, such as unauthorized personnel entering the tank area, operators not wearing personal protective equipment correctly or completely, and operators appearing in prohibited areas, no specific limitations are made here.

[0119] After receiving the alarm, the security personnel in the tank area will take prompt action, such as sending specialized personnel to the scene to verify the situation, initiating emergency evacuation procedures or adjusting the operation plan.

[0120] Of course, in addition to monitoring operators, this embodiment can also monitor mobile vehicles in the tank area, which will not be elaborated here one by one.

[0121] In this embodiment, by comprehensively monitoring the operators in the tank farm, accidents caused by human factors can be effectively prevented, thereby improving the overall safety management level of the tank farm.

[0122] In addition, in some embodiments, based on the tank area and pipeline service perception data, a comprehensive diagnosis and analysis can be performed to determine the source, location and main corrosion causes of the corrosion of the service pipeline, and maintenance strategies and governance recommendations can be automatically output based on the diagnosis results to provide guidance for predictive maintenance decisions for in-service pipelines and ensure the safe operation of the tank area.

[0123] Corresponding to the large tank area safety monitoring method based on machine vision described in the above embodiments, the present invention also provides a tank area safety monitoring device based on machine vision. Specifically, Figure 2 A schematic structural diagram of a tank area safety monitoring device based on machine vision provided by an embodiment of the present invention is shown.

[0124] like Figure 2As shown, the device includes: a tank area device image acquisition module 210, which is used to use multiple pre-installed cameras to acquire the tank area device image of the tank area device in the tank area; a global reference association establishment module 220, which is used to establish a global reference association between the space coordinate system and the camera coordinate system through a homogeneous transformation matrix; a safety monitoring parameter acquisition module 230, which is used to acquire the safety monitoring parameters of the tank area device in the tank area based on the tank area device image based on the global reference association; a safety monitoring result determination module 240, which is used to determine the tank area safety monitoring result based on the safety monitoring parameters of the tank area device.

[0125] In this embodiment, the tank area device image acquisition module 210 uses a plurality of pre-installed cameras to acquire the tank area device image of the tank area device in the tank area, and the global reference association establishment module 220 establishes a global reference association between the spatial coordinate system and the camera coordinate system through a homogeneous transformation matrix, and then the safety monitoring parameter acquisition module 230 acquires the safety monitoring parameters of the tank area device in the tank area based on the global reference association according to the tank area device image, so that the safety monitoring result determination module 240 determines the tank area safety monitoring result according to the safety monitoring parameters of the tank area device. The device establishes a global reference association between the spatial coordinate system and the camera coordinate system through a homogeneous transformation matrix, and establishes a unified reference frame within the entire tank area, which can not only accurately acquire the position and direction of the camera, but also accurately measure the safety monitoring parameters according to the tank area device image to achieve more comprehensive safety monitoring, and can also adapt to the rotation or movement of the camera, making the safety monitoring more flexible and reliable. At the same time, this process does not require the deployment of a large number of sensors, reducing the cost.

[0126] It should be noted that the tank area safety monitoring device based on machine vision provided in this embodiment and the large tank area safety monitoring method based on machine vision described in the above embodiments can be referenced to each other, and will not be repeated here.

[0127] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute a large tank area safety monitoring method based on machine vision, the method comprising: using a plurality of pre-installed cameras to obtain a tank area device image of a tank area device in a tank area; establishing a global reference association between a spatial coordinate system and a camera coordinate system through a homogeneous transformation matrix; based on the global reference association, obtaining the safety monitoring parameters of the tank area device in the tank area according to the tank area device image; and determining the tank area safety monitoring result according to the safety monitoring parameters of the tank area device.

[0128] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0129] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the large tank area safety monitoring method based on machine vision provided by the above methods. The method includes: using multiple pre-installed cameras to obtain tank area device images of tank area devices in the tank area; establishing a global reference association between the spatial coordinate system and the camera coordinate system through a homogeneous transformation matrix; based on the global reference association, obtaining safety monitoring parameters of the tank area devices in the tank area according to the tank area device images; and determining the tank area safety monitoring results according to the safety monitoring parameters of the tank area devices.

[0130] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the large tank area safety monitoring method based on machine vision provided by the above-mentioned methods, the method comprising: using a plurality of pre-installed cameras to obtain a tank area device image of the tank area device in the tank area; establishing a global reference association between the spatial coordinate system and the camera coordinate system through a homogeneous transformation matrix; based on the global reference association, obtaining the safety monitoring parameters of the tank area device in the tank area according to the tank area device image; and determining the tank area safety monitoring results according to the safety monitoring parameters of the tank area device.

[0131] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0132] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A large tank area safety monitoring method based on machine vision, characterized in that: include: Using multiple pre-installed cameras, images of tank farm equipment in the tank farm are acquired; Through the homogeneous transformation matrix, a global reference association between the space coordinate system and the camera coordinate system is established; Based on the global reference association, the safety monitoring parameters of the tank area devices in the tank area are obtained according to the tank area device images; The tank area safety monitoring result is determined according to the safety monitoring parameters of the tank area device.

2. The large tank area safety monitoring method based on machine vision according to claim 1 is characterized in that: The method of using a plurality of pre-installed cameras to obtain images of tank area devices in the tank area includes: When the multiple cameras are in dynamic operation and / or the tank farm device is moved in space, images of the tank farm device are acquired.

3. The large tank area safety monitoring method based on machine vision according to claim 1 is characterized in that: The method of establishing a global reference association between the space coordinate system and the camera coordinate system through a homogeneous transformation matrix includes: The origin of the spatial coordinate system is translated to the position of the camera through the translation matrix; Rotate the space coordinate system to align with the camera coordinate system through the rotation matrix; Combine the translation matrix and the rotation matrix to obtain a homogeneous transformation matrix; According to the homogeneous transformation matrix, a global reference association from the space coordinate system to the camera coordinate system is established.

4. The large tank area safety monitoring method based on machine vision according to claim 1 is characterized in that: The camera is a thermal infrared camera, and the tank area device image is a thermal infrared image; The obtaining of safety monitoring parameters of the tank area devices in the tank area based on the global reference association and according to the tank area device image comprises: Acquire the temperature value of each part of the tank area device in the thermal infrared image, and the temperature difference of adjacent parts of the tank area device; Wherein, the safety monitoring parameters include the temperature value and the temperature difference; Correspondingly, the determining of the tank area safety monitoring result according to the safety monitoring parameters of the tank area device includes: Determine a first location where the temperature value is higher than a first set threshold; and / or, determining a second location where the temperature value is lower than a second set threshold; and / or, determining a third location where the temperature difference is higher than a preset difference; Determine the first part, the second part, and the third part as parts where there are signs of gas or liquid leakage, and obtain the tank area safety monitoring result; The first set threshold is greater than the second set threshold.

5. The large tank area safety monitoring method based on machine vision according to claim 1 is characterized in that: The obtaining of safety monitoring parameters of the tank area devices in the tank area based on the global reference association and according to the tank area device image comprises: Performing edge detection on the tank area device image to obtain a crack contour; Extracting crack characteristic parameters from the crack profile, wherein the crack characteristic parameters include the position, length, width, direction and area of ​​the crack; Wherein, the safety monitoring parameters include the crack characteristic parameters; Correspondingly, the determining of the tank area safety monitoring result according to the safety monitoring parameters of the tank area device includes: Determining the risk level of the tank farm device according to the crack characteristic parameters; Determining that the risk level reaches the set level; The position of the crack in the crack characteristic parameters is determined as a location that needs maintenance, repair or replacement, and the irrigation area safety monitoring result is obtained.

6. The large tank area safety monitoring method based on machine vision according to any one of claims 1 to 5, characterized in that: Also includes: Use multiple pre-installed cameras to obtain target images of operators in the tank farm; Determining the authorization information and wearable equipment of the operator according to the target image; Based on the global reference association, determining the actual position of the operator in the tank farm according to the target image; When any one of the operator's authorization information, wearable equipment and actual location does not meet the preset conditions, an alarm is sent to the security personnel in the tank area so that safety measures can be taken in time.

7. A large tank area safety monitoring device based on machine vision, characterized in that: include: A tank farm device image acquisition module, for acquiring tank farm device images of tank farm devices in a tank farm using a plurality of pre-installed cameras; A global reference association establishment module, used to establish a global reference association between a space coordinate system and a camera coordinate system through a homogeneous transformation matrix; A safety monitoring parameter acquisition module, used to acquire safety monitoring parameters of tank area devices in the tank area based on the global reference association and according to the tank area device image; The safety monitoring result determination module is used to determine the tank area safety monitoring result according to the safety monitoring parameters of the tank area device.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, a large tank area safety monitoring method based on machine vision as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, a large tank area safety monitoring method based on machine vision as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, a large tank area safety monitoring method based on machine vision as described in any one of claims 1 to 6 is implemented.

Images