Port equipment monitoring method and system based on image processing

Abstract

The present invention relates to the technical field of equipment monitoring. Disclosed are a port equipment monitoring method and system based on image processing. The method comprises: acquiring an original image of equipment in a monitoring region during operation; acquiring a fused motion contrast map to obtain an edge intensity map, calculating a spatial distribution value to obtain a weighted edge intensity map, and calculating the semantic importance of each pixel point in the weighted edge intensity map to obtain a saliency map; acquiring key features of the equipment from the saliency map, and inputting the key features into a pre-trained deep learning model to perform classification and identification of equipment status; and on the basis of the classification and identification result, generating a monitoring report and sending same to a monitoring center. Dynamic changes of port equipment, edge information, and subtle changes of the port equipment are effectively captured, and high-level features are identified. On the basis of the obtained saliency map, the most important and noteworthy region in the image can be automatically identified and focused on, thereby significantly improving the monitoring accuracy for the port equipment.

Description

A Port Equipment Monitoring Method and System Based on Image Processing Technical Field

[0001] This invention relates to the field of equipment monitoring technology, and in particular to a port equipment monitoring method and system based on image processing. Background Technology

[0002] With the acceleration of global economic integration and the continuous increase in international shipping and trade volume, ports, as important logistics nodes, are playing an increasingly vital role in national economic and social development. To ensure the efficient and safe operation of ports, effective monitoring and management of various types of machinery and equipment within the ports is particularly important. However, traditional equipment monitoring methods largely rely on manual inspections, which suffer from low efficiency, high costs, and difficulty in timely detection of potential faults.

[0003] In recent years, with the development of computer vision technology and artificial intelligence, image processing-based equipment monitoring methods have gradually become a research hotspot. This method utilizes high-definition cameras installed at key locations in ports to collect image information of equipment during operation, and then analyzes and processes this information through image processing technology to achieve remote, real-time monitoring of equipment status. This method not only significantly improves the accuracy and timeliness of monitoring but also effectively reduces labor costs and enhances the level of intelligent port management. However, despite the many advantages this technology brings, some technical shortcomings still exist in practical applications, specifically in the following aspects: the difference in light intensity between day and night is huge. Strong sunlight during the day may cause image overexposure, while insufficient light at night may produce dim and blurry images, severely affecting image quality and leading to a decrease in recognition accuracy, thus hindering effective monitoring. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a port equipment monitoring method based on image processing, comprising:

[0005] Step S1: Deploy cameras in the monitoring area, adjust the camera exposure time and gain value according to the current light conditions in the monitoring area, and collect raw images of the equipment running in the monitoring area.

[0006] Step S2, preprocessing the original image, the preprocessing includes:

[0007] Step S21: Based on the multi-scale Laplacian pyramid, perform resolution analysis on the original image to obtain the feature image at each scale;

[0008] Step S22: Based on the feature image at each scale, obtain the fused motion contrast map of the pixels, obtain the edge intensity map based on the fused motion contrast map, calculate the spatial distribution value of each pixel based on the edge intensity map, obtain the weighted edge intensity map based on the spatial distribution value and the edge intensity map, calculate the semantic importance of each pixel in the weighted edge intensity map, and obtain the saliency map.

[0009] Step S3: Obtain the key features of the device from the saliency map, and input the key features into the pre-trained deep learning model to classify and identify the device status.

[0010] Step S4: Based on the classification and identification results, generate a monitoring report and send it to the monitoring center.

[0011] Furthermore, the camera's exposure time and gain value are adjusted according to the current lighting conditions in the monitored area, specifically including:

[0012] Step S11: Define a light threshold based on the lighting conditions; the light threshold includes a first light threshold and a second light threshold.

[0013] Step S12: Obtain the light intensity of the monitoring area under the current lighting conditions;

[0014] Step S13: Determine the adjustment parameters under the current lighting conditions;

[0015] Step S14: Adjust the camera exposure time and gain value according to the adjustment parameters, light intensity, first light threshold and second light threshold.

[0016] Furthermore, the adjustment functions for the camera exposure time and gain value are as follows:

[0017]

[0018] In the formula, This represents the exposure time under the i-th lighting condition. This represents the gain value under the i-th illumination condition. Represents the light intensity under the i-th illumination condition. , , , , , , representing the adjustment parameters used when the light intensity is greater than the first light threshold under the i-th lighting condition. , , , , , , representing the adjustment parameters used when the light intensity is less than or equal to the first light threshold and greater than the second light threshold under the i-th lighting condition. , , , , , , , represent the adjustment parameters used when the light intensity is less than or equal to the second light threshold under the i-th lighting condition.

[0019] Furthermore, based on the feature images at each scale, the motion contrast of pixels is obtained, including: obtaining the current frame feature image and the previous frame feature image at each scale; calculating the motion contrast of each pixel in the feature image at each scale based on the current frame feature image and the previous frame feature image to obtain a motion contrast map; and fusing the motion contrast maps at different scales to obtain a fused motion contrast map.

[0020] Furthermore, the formula for calculating motion contrast is:

[0021] ;

[0022] In the formula, Represents the j-th scale feature image pixel. Motion contrast, Represents the feature image pixels of the j-th scale and the t-th frame. pixel values, Represents the feature image pixels of the j-th scale and the (t-1)-th frame. The pixel value.

[0023] Further, an edge intensity map is obtained based on the fused motion contrast map, including: performing edge detection on the fused motion contrast map to obtain the edge intensity map.

[0024] Furthermore, the spatial distribution value of each pixel is calculated based on the edge intensity map, including: determining the center coordinates, and calculating the spatial distribution value of the pixel based on each pixel in the edge intensity map and the determined center coordinates.

[0025] Further, a weighted edge intensity image is obtained based on the spatial distribution value of each pixel and the edge intensity map; the semantic importance of each pixel in the weighted edge intensity image is calculated to generate a semantic feature map; the contrast between each pixel in the weighted edge intensity image and other pixels is calculated to obtain a contrast map; and a saliency map is obtained based on the semantic feature map and the contrast map.

[0026] Furthermore, the semantic importance of each pixel is calculated using a convolutional neural network.

[0027] The present invention also provides a port equipment monitoring system based on image processing, comprising the following modules:

[0028] Image acquisition module: used to deploy cameras in the monitoring area, adjust the camera exposure time and gain value according to the current light conditions in the monitoring area, and acquire raw images of the equipment in operation within the monitoring area;

[0029] Preprocessing module: Connected to the image acquisition module, used to preprocess the raw image;

[0030] Recognition module: Connected to the preprocessing module, it is used to obtain key features of the device from the saliency map, input the key features into the pre-trained deep learning model, and perform device status classification and recognition.

[0031] Monitoring module: Connected to the identification module, it is used to generate monitoring reports based on the classification and identification results and send them to the monitoring center.

[0032] The embodiments of the present invention have the following technical effects:

[0033] On the one hand, this invention, based on existing methods of acquiring raw images via cameras, adjusts the camera's exposure time and gain value according to lighting conditions, resulting in improved raw image quality compared to existing methods. On the other hand, this invention generates a saliency map based on the acquired high-quality raw images, effectively capturing dynamic changes, edge information, and subtle variations in port equipment, and identifying high-level features. Based on the obtained saliency map, it can automatically identify and focus on the most important and noteworthy areas in the image, significantly improving the monitoring accuracy of port equipment.

[0034] This invention performs resolution analysis on the original image based on a multi-scale Laplacian pyramid, obtaining feature images at each scale and highlighting more detailed information at different scales. Simultaneously, a dynamic contrast image is generated based on the illumination intensity of the current frame image and the previous frame image at each scale. These motion contrast images from different scales are then fused to obtain a fused motion contrast image. This allows for the capture of subtle motion changes, even very small displacements, enabling real-time monitoring and timely detection of dynamic changes in port equipment. By combining motion information at different scales, it can more comprehensively capture the dynamic changes of port equipment, enhance detailed information at different scales, help identify minute changes in equipment, and improve the sensitivity of the monitoring system.

[0035] This invention performs edge detection on motion comparison images and calculates gradient intensity to obtain edge intensity maps, thereby enhancing the detailed information of port equipment, more accurately identifying and tracking port equipment, precisely capturing the edge contours of port equipment, promptly detecting abnormal movements of equipment, and improving monitoring sensitivity.

[0036] This invention calculates the spatial distribution values ​​of pixels in an edge intensity map and generates a weighted edge intensity map based on the edge intensity map and the spatial distribution values. The weighted edge intensity map more clearly defines the edge contours of equipment, helping to improve the accuracy of port equipment monitoring. It also incorporates spatial distribution values, which tend to assign higher weights to pixels near the image center, helping to focus on key areas. By generating and utilizing the weighted edge intensity map, not only can the accuracy of tracking dynamic changes in port equipment be improved, but it also plays a crucial role in monitoring abnormal behavior, providing more comprehensive and intelligent support for port management.

[0037] This invention calculates the semantic importance of each pixel in a weighted edge intensity image, generating a semantic feature map; it also calculates the contrast between each pixel and other pixels in the weighted edge intensity image, obtaining a contrast map; and finally, a saliency map is derived from the semantic feature map and the contrast map. Semantic importance provides high-level features, namely, the specific equipment type of the port equipment. The saliency map contains more crucial information, including: dynamic changes in the equipment, the edge contours of the equipment, subtle changes in the equipment, and equipment type identification. The key features extracted from the saliency features are more accurate and can effectively improve monitoring precision during equipment monitoring. Attached Figure Description

[0038] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0039] Figure 1 is a flowchart of a port equipment monitoring method based on image processing provided in an embodiment of the present invention;

[0040] Figure 2 is a structural diagram of a port equipment monitoring system based on image processing provided in an embodiment of the present invention; Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0042] Figure 1 is a flowchart of a port equipment monitoring method based on image processing provided by an embodiment of the present invention. Referring to Figure 1, the method specifically includes:

[0043] Step S1: Deploy cameras in the monitoring area, adjust the camera exposure time and gain value according to the current lighting conditions in the monitoring area, and collect raw images of the equipment running in the monitoring area.

[0044] This embodiment should deploy cameras at key locations within the monitoring area to ensure full coverage of all critical points, such as crane operating areas and cargo stacking areas. The cameras should have the ability to automatically adjust exposure time and gain to adapt to different lighting conditions. They can be industrial cameras, security cameras, etc.

[0045] Step S11: Define a light threshold based on the lighting conditions; the light threshold includes a first light threshold and a second light threshold.

[0046] Historical image samples under different lighting conditions captured by the camera are acquired, and the brightness levels of the image samples are analyzed to determine the optimal image quality under which lighting conditions. Preferably, a first lighting threshold is set as the highest lighting intensity to ensure the image is not overexposed, while a second lighting threshold is set as the lowest lighting intensity to prevent the image from being too dark. For example, if the image quality is optimal when the average brightness value is around 128 (grayscale image range 0-255), then the first lighting threshold is set to 160, and the second lighting threshold is set to 90.

[0047] Step S12: Obtain the light intensity of the monitoring area under the current lighting conditions.

[0048] Preferably, the light intensity is measured directly using a light sensor installed near the camera.

[0049] Step S13: Determine the adjustment parameters under the current lighting conditions.

[0050] The adjustment parameters are coefficients used to calculate exposure time and gain value, which determine how to adjust exposure time and gain value under different lighting conditions to achieve the best image quality.

[0051] The adjustment parameter used when the light intensity is greater than the first light threshold under the i-th illumination condition is: , , , , , The adjustment parameters used when the light intensity under the i-th illumination condition is less than or equal to the first illumination threshold and greater than the second illumination threshold are: , , , , , The adjustment parameters used when the light intensity is less than or equal to the second light threshold under the i-th illumination condition are: , , , , , .

[0052] Step S14: Adjust the camera exposure time and gain value according to the adjustment parameters, light intensity, first light threshold and second light threshold.

[0053] ;

[0054] In the formula, This represents the exposure time under the i-th lighting condition. This represents the gain value under the i-th illumination condition. Represents the light intensity under the i-th illumination condition. , , , , , , representing the adjustment parameters used when the light intensity is greater than the first light threshold under the i-th lighting condition. , , , , , , representing the adjustment parameters used when the light intensity is less than or equal to the first light threshold and greater than the second light threshold under the i-th lighting condition. , , , , , , , represent the adjustment parameters used when the light intensity is less than or equal to the second light threshold under the i-th lighting condition.

[0055] The above adjustment parameters are obtained by acquiring recommended exposure times and gain values ​​under different lighting conditions from the camera user manual, and by performing statistical analysis on the acquired image samples to determine the adjustment coefficients.

[0056] Step S2: Preprocess the original image.

[0057] The preprocessing includes: removing abnormal pixels from the original image and filling in missing pixels using interpolation.

[0058] The preprocessing also includes:

[0059] Step S21: Based on the multi-scale Laplacian pyramid, perform resolution analysis on the original image to obtain the feature image at each scale.

[0060] Based on the mechanism of the multi-scale Laplacian pyramid, the original image is downsampled to generate several images of different resolutions, i.e., several multi-scale feature images.

[0061] Step S22: Based on the feature image at each scale, obtain the fused motion contrast map of the pixels, obtain the edge intensity map based on the fused motion contrast map, calculate the spatial distribution value of each pixel based on the edge intensity map, obtain the weighted edge intensity value based on the spatial distribution value and the edge intensity map, calculate the semantic importance of each pixel in the weighted edge intensity map, and obtain the saliency map.

[0062] In this embodiment, the motion contrast of pixels is obtained based on the feature image at each scale, including: obtaining the current frame feature image and the previous frame feature image at each scale; calculating the motion contrast of each pixel in the feature image at each scale based on the current frame feature image and the previous frame feature image to obtain a motion contrast map; and fusing the motion contrast maps at different scales to obtain a fused motion contrast map.

[0063] In this embodiment, the camera monitors the port equipment in the monitoring area in real time. Therefore, the camera captures one image per frame. In this embodiment, the motion contrast of each scale feature map pixel is calculated based on the feature image of the current frame and the feature image of the previous frame.

[0064] Motion contrast refers to the degree of change in pixel value at the same location between two consecutive frames. This change can be a change in grayscale value or a change in color value. This embodiment is not limited to this. A higher motion contrast indicates more significant motion, while a lower motion contrast indicates almost no motion or no motion. To eliminate noise or slight motion, an upper limit can be set. Motion contrast below this upper limit will be considered as no motion or insignificant motion and will be set to 0 or other background values. Motion contrast greater than or equal to the upper limit will be considered as significant motion. For example, the distribution of motion contrast at different scales is analyzed by histogram analysis, and the peak or valley value in the histogram is selected as the upper limit. Pixels with insignificant motion or no motion are eliminated using the upper limit, and the vacant pixel positions are filled by the difference in motion contrast of nearby valid pixels.

[0065] ;

[0066] In the formula, Represents the j-th scale feature image pixel. Motion contrast, Represents the feature image pixels of the j-th scale and the t-th frame. pixel values, Represents the feature image pixels of the j-th scale and the (t-1)-th frame. The pixel value.

[0067] The process of fusion of motion comparison images at different scales is as follows:

[0068] ;

[0069] Pixels in the motion fusion comparison image Motion contrast, The weights represent the feature images at the j-th scale.

[0070] In this embodiment, obtaining an edge intensity map based on the fused motion contrast map includes: performing edge detection on the fused motion contrast map to obtain the edge intensity map.

[0071] Edge detection of port equipment can be achieved using edge detection algorithms, and the gradient intensity of each pixel can be calculated using the Sobel operator. Edge detection is a mature existing technology, and the principles and implementation process of edge detection will not be elaborated in detail here.

[0072] Edge detection can help identify the boundaries of port equipment. For example, by detecting the edges of equipment such as containers and cranes, the location of these devices can be accurately identified and tracked. The contours of port equipment can also be extracted, aiding in shape analysis and classification. For example, by detecting the edges of containers, it can be determined whether their size and shape conform to standards. The Sobel operator highlights edges in an image by calculating the gradient intensity of each pixel. Gradient intensity reflects the rate of change of pixel values ​​in space, i.e., the salience of the edge. The higher the gradient intensity, the more pronounced the edge at that location. By applying the Sobel operator at different scales, edges of different scales can be detected. This is very useful for identifying port equipment of different sizes. For example, large-scale edges can be used to detect large equipment (such as cranes), while small-scale edges can be used to detect small equipment (such as sensors).

[0073] In summary, edge detection and gradient strength calculation enable real-time monitoring of the status and location of port equipment. For example, detecting the edges of containers allows for real-time tracking of their movement. Detecting the edges of equipment also allows for the timely detection of abnormalities. For instance, if the edge of a container suddenly disappears or deforms, it may indicate damage or displacement of the container.

[0074] In this embodiment, calculating the spatial distribution value of each pixel based on the edge intensity map includes: determining the center coordinates, and calculating the spatial distribution value of the pixel based on each pixel in the edge intensity map and the determined center coordinates.

[0075] The center coordinates are obtained based on the image size. For example, the image size is W × H, x... c =W / 2, y c =H / 2.

[0076] ;

[0077] In the formula, Representing pixels Spatial distribution values, Represents the center coordinates, Representative weight.

[0078] Based on the spatial distribution value of each pixel and the edge intensity map, a weighted edge intensity image is obtained; the semantic importance of each pixel in the weighted edge intensity image is calculated to generate a semantic feature map; the contrast between each pixel in the weighted edge intensity image and other pixels is calculated to obtain a contrast map; and a saliency map is obtained based on the semantic feature map and the contrast map.

[0079] The spatial distribution value of each pixel is multiplied by the gradient intensity of the pixel in the edge intensity map to obtain the weighted edge gradient intensity, thus obtaining the weighted edge intensity image.

[0080] The semantic importance of each pixel is calculated and a semantic feature map is generated by using a convolutional neural network. This part is existing technology and will not be described in detail in this embodiment.

[0081] The contrast calculation process involves constructing a neighborhood window, which can be 5×5 or 3×3. For each pixel (x, y), the local contrast between it and other pixels within the neighborhood window is calculated. :

[0082] ;

[0083] It refers to other pixels within a neighborhood window centered at pixel (x, y). It refers to other pixels within the neighboring window. It refers to the pixel value of a pixel (x, y). Other pixels The pixel value.

[0084] To highlight the contrast of the edge areas The edge intensity G can be used as a weight and multiplied by the local contrast C:

[0085] ;

[0086] Weighted contrast of each pixel Combined into a two-dimensional matrix, this forms the contrast image. .

[0087] In another implementation, based on the saliency image, the importance score of each pixel is calculated according to the saliency map; a filtering threshold is constructed for the importance score; the importance score of each pixel is compared with the filtering threshold, and if the importance score of the current pixel is greater than the filtering threshold, the current pixel is retained; if the importance score of the current pixel is less than or equal to the filtering threshold, the current pixel is removed; the positions of the removed pixels are filled using interpolation to obtain the final saliency map.

[0088] Step S3: Obtain the key features of the device from the saliency map, and input the key features into the pre-trained deep learning model to classify and identify the device status.

[0089] Key features are extracted from the saliency map. Traditional feature extraction methods (such as HOG and SIFT) or deep learning methods (such as CNN feature extraction layers) can be used to convert the extracted features into feature vectors. These feature vectors are then input into each element of a pre-trained deep learning model, which outputs the device state category.

[0090] In another implementation, key features are extracted from the final saliency map and state recognition is performed.

[0091] Step S4: Based on the classification and identification results, generate a monitoring report and send it to the monitoring center.

[0092] First, create a report template, including device name, status, timestamp, and other information. Iterate through all test samples, generating a report entry for each sample, and then summarize all entries into a single report list. You can choose from various methods to send the report to the monitoring center, such as email, HTTP request, or MQTT message.

[0093] As shown in Figure 2, a port equipment monitoring system based on image processing includes the following modules:

[0094] Image acquisition module: used to deploy cameras in the monitoring area, adjust the camera exposure time and gain value according to the current light conditions in the monitoring area, and acquire raw images of the equipment in operation within the monitoring area;

[0095] Preprocessing module: Connected to the image acquisition module, used to preprocess the raw image;

[0096] Recognition module: Connected to the preprocessing module, it is used to obtain key features of the device from the saliency map, input the key features into the pre-trained deep learning model, and perform device status classification and recognition.

[0097] Monitoring module: Connected to the identification module, it is used to generate monitoring reports based on the classification and identification results and send them to the monitoring center.

[0098] It should be noted that the terminology used in this invention is for describing specific embodiments only and is not intended to limit the scope of this application. As shown in this specification, unless the context clearly indicates otherwise, words such as "a," "an," "an," and / or "the" do not specifically refer to the singular and may include the plural. The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, or apparatus that includes said element.

[0099] It should also be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Unless otherwise expressly specified and limited, the terms "installed," "connected," "linked," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components. For those skilled in the art, the specific meaning of the above terms in the present invention can be understood according to the specific circumstances.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A port equipment monitoring method based on image processing, characterized in that, include: Step S1: Deploy cameras in the monitoring area, adjust the camera exposure time and gain value according to the current light conditions in the monitoring area, and collect raw images of the equipment running in the monitoring area. Step S2, preprocessing the original image, the preprocessing includes: Step S21: Based on the multi-scale Laplacian pyramid, perform resolution analysis on the original image to obtain the feature image at each scale; Step S22: Based on the feature image at each scale, obtain the fused motion contrast map of the pixels, obtain the edge intensity map based on the fused motion contrast map, calculate the spatial distribution value of each pixel based on the edge intensity map, obtain the weighted edge intensity map based on the spatial distribution value and the edge intensity map, calculate the semantic importance of each pixel in the weighted edge intensity map, and obtain the saliency map. The calculation of the spatial distribution value of each pixel based on the edge intensity map includes: determining the center coordinates, and calculating the spatial distribution value of the pixel based on each pixel in the edge intensity map and the determined center coordinates. In the formula, Representing pixels Spatial distribution values, Represents the center coordinates, Representative weights; Step S3: Obtain the key features of the device from the saliency map, and input the key features into the pre-trained deep learning model to classify and identify the device status. Step S4: Based on the classification and identification results, generate a monitoring report and send it to the monitoring center.

2. The port equipment monitoring method based on image processing according to claim 1, characterized in that, Adjust the camera's exposure time and gain value according to the current lighting conditions in the monitored area, specifically including: Step S11: Define a light threshold based on the lighting conditions; the light threshold includes a first light threshold and a second light threshold. Step S12: Obtain the light intensity of the monitoring area under the current lighting conditions; Step S13: Determine the adjustment parameters under the current lighting conditions; Step S14: Adjust the camera exposure time and gain value according to the adjustment parameters, light intensity, first light threshold and second light threshold.

3. The port equipment monitoring method based on image processing according to claim 2, characterized in that, The function for adjusting the camera exposure time and gain is: ； In the formula, This represents the exposure time under the i-th lighting condition. This represents the gain value under the i-th illumination condition. Represents the light intensity under the i-th illumination condition. 、 、 、 、 、 , representing the adjustment parameters used when the light intensity is greater than the first light threshold under the i-th lighting condition. 、 、 、 、 、 , representing the adjustment parameters used when the light intensity is less than or equal to the first light threshold and greater than the second light threshold under the i-th lighting condition. 、 、 、 、 、 , , represent the adjustment parameters used when the light intensity is less than or equal to the second light threshold under the i-th lighting condition.

4. The port equipment monitoring method based on image processing according to claim 1, characterized in that, Based on the feature images at each scale, the motion contrast of pixels is obtained, including: obtaining the current frame feature image and the previous frame feature image at each scale; calculating the motion contrast of each pixel in the feature image at each scale based on the current frame feature image and the previous frame feature image to obtain a motion contrast map; and fusing the motion contrast maps at different scales to obtain a fused motion contrast map.

5. The port equipment monitoring method based on image processing according to claim 4, characterized in that, The formula for calculating motion contrast is: ； In the formula, Represents the j-th scale feature image pixel. Motion contrast, Represents the feature image pixels of the j-th scale and the t-th frame. pixel values, Represents the feature image pixels of the j-th scale and the (t-1)-th frame. The pixel value.

6. The port equipment monitoring method based on image processing according to claim 4, characterized in that, The edge intensity map is obtained from the fused motion contrast map, including: edge detection is performed on the fused motion contrast map to obtain the edge intensity map.

7. The port equipment monitoring method based on image processing according to claim 1, characterized in that, Based on the spatial distribution value of each pixel and the edge intensity map, a weighted edge intensity image is obtained; the semantic importance of each pixel in the weighted edge intensity image is calculated to generate a semantic feature map; the contrast between each pixel in the weighted edge intensity image and other pixels is calculated to obtain a contrast map; and a saliency map is obtained based on the semantic feature map and the contrast map.

8. The port equipment monitoring method based on image processing according to claim 7, characterized in that, The semantic importance of each pixel is calculated using a convolutional neural network.

9. A port equipment monitoring system based on image processing, used to execute the port equipment monitoring method based on image processing according to any one of claims 1-8, characterized in that, Includes the following modules: Image acquisition module: used to deploy cameras in the monitoring area, adjust the camera exposure time and gain value according to the current light conditions in the monitoring area, and acquire raw images of the equipment in operation within the monitoring area; Preprocessing module: Connected to the image acquisition module, used to preprocess the raw image; Recognition module: Connected to the preprocessing module, it is used to obtain key features of the device from the saliency map, input the key features into the pre-trained deep learning model, and perform device status classification and recognition. Monitoring module: Connected to the identification module, it is used to generate monitoring reports based on the classification and identification results and send them to the monitoring center.

Images