A port equipment monitoring method and system based on image processing

Abstract

The present invention relates to the technical field of equipment monitoring, and discloses a port equipment monitoring method and system based on image processing, the method comprising: collecting original images of equipment in operation in the monitoring area, obtaining a fused motion contrast map, obtaining an edge strength map, calculating a spatial distribution value, obtaining a weighted edge strength map, calculating the semantic importance of each pixel in the weighted edge strength map, obtaining a saliency map, obtaining key features of the equipment from the saliency map, inputting the key features into a pre-trained deep learning model, and classifying and identifying the equipment status; based on the classification and identification results, generating a monitoring report and sending it to a monitoring center. The dynamic changes, edge information and subtle changes of port equipment are effectively captured, and high-level features are identified. Based on the obtained saliency map, the most important and most noteworthy areas in the image can be automatically identified and focused, which can significantly improve the monitoring accuracy of port equipment.

Description

Technical Field

[0001] The present invention relates to the technical field of equipment monitoring, and in particular to a port equipment monitoring method and system based on image processing. Background Art

[0002] With the acceleration of global economic integration, the volume of international shipping trade continues to increase. As an important logistics node, ports play an increasingly important role in national economic and social development. In order to ensure the efficient and safe operation of ports, it is particularly important to effectively monitor and manage various types of mechanical equipment in ports. However, traditional equipment monitoring methods mostly rely on manual inspections, which have problems such as low efficiency, high cost, and difficulty in timely detection of potential faults.

[0003] In recent years, with the development of computer vision technology and artificial intelligence, equipment monitoring methods based on image processing have gradually become a research hotspot. This method uses high-definition cameras installed at key locations in the port to collect image information during equipment operation, and analyzes and processes this information through image processing technology, thereby realizing remote and real-time monitoring of equipment status. This method can not only significantly improve the accuracy and timeliness of monitoring, but also effectively reduce labor costs and improve the level of intelligent port management. However, despite the many advantages brought by this technology, there are still some technical defects in practical applications, which are specifically manifested in the following aspects: the light intensity during the day and at night is very different. The strong sunlight during the day may cause the image to be overexposed, while the dark and blurred image may be produced at night due to insufficient light. The image quality may be seriously affected, resulting in a decrease in recognition accuracy and inability to achieve effective monitoring. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a port equipment monitoring method based on image processing, comprising:

[0005] Step S1, deploying a camera in the monitoring area, adjusting the camera exposure time and gain value according to the current light conditions in the monitoring area, and collecting the original image of the equipment in the monitoring area when it is running;

[0006] Step S2, preprocessing the original image, the preprocessing includes:

[0007] Step S21, performing resolution analysis on the original image according to the multi-scale Laplacian pyramid to obtain a feature image of each scale;

[0008] Step S22, obtaining a fused motion contrast map of pixels according to the feature images of each scale, obtaining an edge strength map according to the fused motion contrast map, calculating a spatial distribution value of each pixel according to the edge strength map, obtaining a weighted edge strength map according to the spatial distribution value and the edge strength map, calculating the semantic importance of each pixel in the weighted edge strength map, and obtaining a saliency map;

[0009] Step S3, obtaining key features of the device from the saliency map, inputting the key features into the pre-trained deep learning model, and performing classification and recognition of the device status;

[0010] Step S4: Generate a monitoring report based on the classification and identification results and send it to the monitoring center.

[0011] Furthermore, the camera exposure time and gain value are adjusted according to the current light conditions in the monitoring area, including:

[0012] Step S11, defining a light threshold according to a light condition; the light threshold includes a first light threshold and a second light threshold;

[0013] Step S12, obtaining the light intensity under the current light conditions in the monitoring area;

[0014] Step S13, determining adjustment parameters under current light conditions;

[0015] Step S14, adjusting the camera exposure time and gain value according to the adjustment parameter, the light intensity, the first light threshold and the second light threshold.

[0016] Furthermore, the adjustment function of the camera exposure time and gain value is:

[0017] ;

[0018] Where, T i represents the exposure time under the i-th lighting condition, G i Represents the gain value under the i-th lighting condition, I i represents the light intensity under the i-th lighting condition, , , , , , , respectively represent the adjustment parameters used when the illumination intensity under the i-th illumination condition is greater than the first illumination threshold, , , , , , , respectively represent the adjustment parameters used when the illumination intensity under the i-th illumination condition is less than or equal to the first illumination threshold and greater than the second illumination threshold, , , , , , , respectively represent the adjustment parameters used when the illumination intensity under the i-th illumination condition is less than or equal to the second illumination threshold.

[0019] Furthermore, according to the feature image of each scale, a fused motion contrast map of pixels is obtained, including: obtaining the feature image of the current frame and the feature image of the previous frame of each scale, and calculating the motion contrast of each pixel in the feature image of each scale according to the feature image of the current frame and the feature image of the previous frame to obtain a motion contrast map; and fusing the motion contrast maps of different scales to obtain a fused motion contrast map.

[0020] Furthermore, the calculation formula of motion contrast is:

[0021] ;

[0022] In the formula, represents the motion contrast of the j-th scale feature image pixel (x, y), Represents the pixel value of the feature image pixel (x, y) at the jth scale and the tth frame, Represents the pixel value of the feature image pixel (x, y) at the jth scale and the t-1th frame.

[0023] Furthermore, obtaining an edge intensity map according to the fused motion contrast map includes: performing edge detection on the fused motion contrast map to obtain an edge intensity map.

[0024] Furthermore, calculating the spatial distribution value of each pixel point according to the edge intensity map includes: determining the center coordinates, and calculating the spatial distribution value of the pixel point according to each pixel point in the edge intensity map and the determined center coordinates.

[0025] Furthermore, a weighted edge intensity image is obtained according to the spatial distribution value of each pixel and the edge intensity map; the semantic importance of each pixel in the weighted edge intensity image is calculated to generate a semantic feature map; the contrast between each pixel in the weighted edge intensity image and other pixels is calculated to obtain a contrast map, and a saliency map is obtained according to the semantic feature map and the contrast map.

[0026] Furthermore, the semantic importance of each pixel is calculated through a convolutional neural network.

[0027] The present invention also provides a port equipment monitoring system based on image processing, comprising the following modules:

[0028] Image acquisition module: used to deploy cameras in the monitoring area, adjust the camera exposure time and gain value according to the current light conditions in the monitoring area, and collect the original images of the equipment in the monitoring area when it is running;

[0029] Preprocessing module: connected with the image acquisition module, used to preprocess the original image;

[0030] Identification module: connected to the preprocessing module, used to obtain the key features of the device from the saliency map, input the key features into the pre-trained deep learning model, and perform classification and identification of the device status;

[0031] Monitoring module: connected with the identification module, used to generate monitoring reports based on classification and identification results and send them to the monitoring center.

[0032] The embodiments of the present invention have the following technical effects:

[0033] On the one hand, the present invention adjusts the exposure time and gain value of the camera according to the light conditions based on the existing original image collected by the camera. Compared with the existing original image, the quality of the original image collected by this application is improved; on the other hand, the present invention generates a saliency map based on the collected high-quality original image, effectively captures the dynamic changes, edge information and small changes of port equipment, and identifies high-level features. Based on the obtained saliency map, it can automatically identify and focus on the most important and most noteworthy areas in the image, which can significantly improve the monitoring accuracy of port equipment.

[0034] The present invention performs resolution analysis on the original image based on a multi-scale Laplacian pyramid to obtain a feature image of each scale, highlighting more detailed information at different scales. At the same time, a dynamic contrast image is obtained based on the illumination intensity of the current frame image and the previous frame image at each scale, and the motion contrast images of different scales are fused to obtain a fused motion contrast image. It is possible to capture subtle motion changes, even very small displacements can be monitored, to achieve real-time monitoring, and to promptly discover dynamic changes in port equipment. In combination with motion information at different scales, it is possible to more comprehensively capture dynamic changes in port equipment, enhance detailed information at different scales, help identify subtle changes in equipment, and improve the sensitivity of the monitoring system.

[0035] The present invention performs edge detection on the motion contrast image and calculates the gradient strength to obtain an edge strength image, thereby enhancing the detailed information of the port equipment, more accurately identifying and tracking the port equipment, precisely capturing the edge contour of the port equipment, timely discovering the abnormal movement of the equipment, and improving the monitoring sensitivity.

[0036] The present invention calculates the spatial distribution value of the pixel points of the edge intensity map, and generates a weighted edge intensity map based on the edge intensity map and the spatial distribution value. The weighted edge intensity map can more clearly define the edge contour of the equipment, which helps to improve the accuracy of port equipment monitoring. Combined with the spatial distribution value. The spatial distribution value tends to give higher weights to pixels near the center of the image, which helps to focus on key areas. By generating and utilizing a weighted edge intensity map, not only can the accuracy of tracking dynamic changes in port equipment be improved, but it can also play an important role in the abnormal behavior monitoring process, providing more comprehensive and intelligent support for port management.

[0037] The present invention calculates the semantic importance of each pixel in the weighted edge strength image to generate a semantic feature map; calculates the contrast between each pixel and other pixels in the weighted edge strength image to obtain a contrast map, and obtains a saliency map based on the semantic feature map and the contrast map. Semantic importance provides high-level features, namely, the specific equipment type of the port equipment. The saliency map contains more key information, namely: dynamic changes of equipment, edge contours of equipment, small changes of equipment, and equipment type identification. The key features extracted from the saliency features can be more accurate and can be used in the equipment monitoring process to effectively improve the monitoring accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 is a flow chart of a port equipment monitoring method based on image processing provided by an embodiment of the present invention;

[0040] Figure 2 It is a structural diagram of a port equipment monitoring system based on image processing provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0042] Figure 1is a flow chart of a port equipment monitoring method based on image processing provided by an embodiment of the present invention. Figure 1 , specifically including:

[0043] Step S1, deploying a camera in the monitoring area, adjusting the camera exposure time and gain value according to the current light conditions in the monitoring area, and collecting original images of the equipment in the monitoring area when it is running.

[0044] In this embodiment, cameras should be deployed at key locations in the monitoring area to ensure full coverage of all key points, such as crane working areas, cargo storage areas, etc. The camera should have the ability to automatically adjust the exposure time and gain value to adapt to different lighting conditions. It can be an industrial camera, a security camera, etc.

[0045] Step S11, defining a light threshold according to a light condition; the light threshold includes a first light threshold and a second light threshold.

[0046] Obtain historical image samples collected by the camera under different lighting conditions, analyze the brightness levels of the image samples, and determine under which lighting conditions the image quality is best. Preferably, the first lighting threshold is set to the highest lighting intensity to ensure that the image is not overexposed, and the second lighting threshold is set to the lowest lighting intensity to prevent the image from being too dark. For example, the image quality is best when the average brightness value of the image is around 128 (grayscale image range 0-255), then the first lighting threshold is set to 160 and the second lighting threshold is set to 90.

[0047] Step S12, obtaining the light intensity under the current light conditions in the monitoring area.

[0048] The light intensity is preferably measured directly by a light sensor mounted near the camera.

[0049] Step S13, determining adjustment parameters under current light conditions.

[0050] The adjustment parameters are coefficients used to calculate the exposure time and gain value, which determine how to adjust the exposure time and gain value under different lighting conditions to achieve the best image quality.

[0051] The adjustment parameter used when the light intensity under the i-th light condition is greater than the first light threshold is: , , , , , ; Adjustment parameters used when the light intensity under the i-th light condition is less than or equal to the first light threshold and greater than the second light threshold: , , , , , ; The adjustment parameters used when the light intensity under the i-th light condition is less than or equal to the second light threshold: , , , , , .

[0052] Step S14, adjusting the camera exposure time and gain value according to the adjustment parameter, the light intensity, the first light threshold and the second light threshold.

[0053] ;

[0054] Where, T i represents the exposure time under the i-th lighting condition, G i Represents the gain value under the i-th lighting condition, I i represents the light intensity under the i-th lighting condition, , , , , , , respectively represent the adjustment parameters used when the illumination intensity under the i-th illumination condition is greater than the first illumination threshold, , , , , , , respectively represent the adjustment parameters used when the illumination intensity under the i-th illumination condition is less than or equal to the first illumination threshold and greater than the second illumination threshold, , , , , , , respectively represent the adjustment parameters used when the illumination intensity under the i-th illumination condition is less than or equal to the second illumination threshold.

[0055] The above adjustment parameters are obtained by obtaining the recommended exposure time and gain value under different lighting conditions from the camera user manual, and performing statistical analysis on the collected image samples to determine the adjustment coefficient.

[0056] Step S2, preprocessing the original image.

[0057] The preprocessing includes: removing abnormal pixels in the original image and filling the missing pixels by interpolation.

[0058] The preprocessing further comprises:

[0059] Step S21, performing resolution analysis on the original image according to the multi-scale Laplacian pyramid to obtain a feature image at each scale.

[0060] According to the mechanism of multi-scale Laplacian pyramid, the original image is downsampled to generate several images with different resolutions, that is, several multi-scale feature images.

[0061] Step S22, according to the feature image of each scale, obtain a fused motion contrast map of the pixel points, obtain an edge strength map according to the fused motion contrast map, calculate the spatial distribution value of each pixel point according to the edge strength map, obtain a weighted edge strength value according to the spatial distribution value and the edge strength map, calculate the semantic importance of each pixel point in the weighted edge strength map, and obtain a saliency map.

[0062] In this embodiment, a fused motion contrast map of pixels is obtained according to the feature image of each scale, including: obtaining the feature image of the current frame and the feature image of the previous frame of each scale, calculating the motion contrast of each pixel in the feature image of each scale according to the feature image of the current frame and the feature image of the previous frame to obtain a motion contrast map; and fusing the motion contrast maps of different scales to obtain a fused motion contrast map.

[0063] In this embodiment, the camera monitors the port equipment in the monitoring area in real time. Therefore, the camera collects an image for each frame. This embodiment calculates the motion contrast of each pixel of the scale feature image based on the feature image of the current frame and the feature image of the previous frame.

[0064] Motion contrast refers to the degree of change in the pixel value at the same position between two consecutive frames. This change can be a change in grayscale value or a change in color value. This is not limited in this embodiment. A larger motion contrast indicates that the motion is more significant, and a smaller motion contrast indicates that there is almost no motion or no motion. In order to eliminate noise or slight motion, an upper limit value can be set. Motion contrast below this upper limit value will be regarded as no motion or insignificant motion, and thus will be set to 0 or other background values, and motion contrast greater than or equal to the upper limit value will be regarded as significant motion. Exemplarily, the distribution of motion contrast of different scales is analyzed by histogram, and the peak or valley value in the histogram is selected as the upper limit value. Pixels with insignificant or no motion are eliminated by the upper limit value, and the vacant pixel position is filled by difference through the motion contrast of nearby valid pixels.

[0065] ;

[0066] In the formula, represents the motion contrast of the j-th scale feature image pixel (x, y), Represents the pixel value of the feature image pixel (x, y) at the jth scale and the tth frame, Represents the pixel value of the feature image pixel (x, y) at the jth scale and the t-1th frame.

[0067] The fusion process of motion contrast images of different scales is as follows:

[0068] ;

[0069] Represents the motion contrast of the pixel (x, y) in the fused motion contrast map, Represents the weight of the j-th scale feature image.

[0070] In this embodiment, obtaining an edge intensity map according to the fused motion contrast map includes: performing edge detection on the fused motion contrast map to obtain an edge intensity map.

[0071] The edge detection of port equipment can be achieved through the edge detection algorithm, and the gradient intensity of each pixel can be calculated using the Sobel operator.

[0072] Edge detection can help identify the boundaries of port equipment. For example, by detecting the edges of equipment such as containers and cranes, the locations of these equipment can be accurately identified and tracked. The outlines of port equipment can also be extracted, which helps in shape analysis and classification. For example, by detecting the edges of containers, it can be determined whether their size and shape meet the standards. The Sobel operator can highlight the edges in the image by calculating the gradient intensity of each pixel. The gradient intensity reflects the rate of change of the pixel value in space, that is, the significance of the edge. The higher the gradient intensity, the more obvious the edge at that location. By applying the Sobel operator at different scales, edges of different scales can be detected. This is very useful for identifying port equipment of different sizes. For example, large-scale edges can be used to detect large equipment (such as cranes), while small-scale edges can be used to detect small equipment (such as sensors).

[0073] In summary, edge detection and gradient strength calculation can be used to monitor the status and position of port equipment in real time. For example, by detecting the edge of a container, the movement of the container can be tracked in real time. By detecting the edge of the equipment, abnormal conditions of the equipment can be discovered in time. For example, if the edge of a container suddenly disappears or deforms, it may mean that the container is damaged or shifted.

[0074] In this embodiment, calculating the spatial distribution value of each pixel point according to the edge intensity map includes: determining the center coordinates, and calculating the spatial distribution value of the pixel point according to each pixel point in the edge intensity map and the determined center coordinates.

[0075] The center coordinates are obtained according to the image size. For example, the image size is W×H, xc =W / 2,y c =H / 2.

[0076] ;

[0077] In the formula, Represents the spatial distribution value of the pixel point (x, y), (x c ,y c ) represents the center coordinates, and ρ represents the weight.

[0078] According to the spatial distribution value of each pixel and the edge strength map, a weighted edge strength image is obtained; the semantic importance of each pixel in the weighted edge strength image is calculated to generate a semantic feature map; the contrast between each pixel in the weighted edge strength image and other pixels is calculated to obtain a contrast map, and a saliency map is obtained according to the semantic feature map and the contrast map.

[0079] The spatial distribution value of each pixel is multiplied by the pixel gradient intensity in the edge intensity map to obtain the weighted edge gradient intensity, thereby obtaining a weighted edge intensity image.

[0080] The semantic importance of each pixel is calculated through a convolutional neural network and a semantic feature map is generated. This part belongs to the prior art and will not be described in detail in this embodiment.

[0081] Contrast calculation process: construct a neighborhood window, which can be a 5×5 or 3×3 window, and for each pixel point (x, y), calculate its local contrast with other pixels in the neighborhood window :

[0082] ;

[0083] Q(x, y) refers to other pixels in the neighborhood window centered on the pixel (x, y), (p, q) refers to other pixels in the neighborhood window, I(x, y) refers to the pixel value of the pixel (x, y), and I(p, q) refers to the pixel values ​​of other pixels (p, q).

[0084] To highlight the contrast of edge areas , the edge strength G can be used as a weight and multiplied by the local contrast C:

[0085] ;

[0086] The weighted contrast of each pixel Combined into a two-dimensional matrix, which is the contrast map .

[0087] In another embodiment, based on the saliency image, the importance score of each pixel is calculated according to the saliency map; a screening threshold is constructed for the importance score; the importance score of each pixel is compared with the screening threshold, and when the importance score of the current pixel is greater than the screening threshold, the current pixel is retained; if the importance score of the current pixel is less than or equal to the screening threshold, the current pixel is eliminated; and the positions of the eliminated pixels are filled using interpolation to obtain a final saliency map.

[0088] Step S3, obtaining key features of the device from the saliency map, inputting the key features into the pre-trained deep learning model, and performing classification and recognition of the device status.

[0089] Extract key features from the saliency map. You can use traditional feature extraction methods (such as HOG, SIFT) or deep learning methods (such as CNN feature extraction layer) to convert the extracted features into feature vectors. Input the feature vectors into the pre-trained deep learning model and output the device status category.

[0090] In another embodiment, key features are extracted from the final saliency map and state recognition is performed.

[0091] Step S4: Generate a monitoring report based on the classification and identification results and send it to the monitoring center.

[0092] First, create a report template, including information such as device name, status, timestamp, etc. Iterate through all test samples, generate report entries for each sample, and summarize all entries into a report list. You can choose a variety of ways to send reports to the monitoring center, such as via email, HTTP request, MQTT message, etc.

[0093] like Figure 2 As shown, a port equipment monitoring system based on image processing includes the following modules:

[0094] Image acquisition module: used to deploy cameras in the monitoring area, adjust the camera exposure time and gain value according to the current light conditions in the monitoring area, and collect the original images of the equipment in the monitoring area when it is running;

[0095] Preprocessing module: connected with the image acquisition module, used to preprocess the original image;

[0096] Identification module: connected to the preprocessing module, used to obtain the key features of the device from the saliency map, input the key features into the pre-trained deep learning model, and perform classification and identification of the device status;

[0097] Monitoring module: connected with the identification module, used to generate monitoring reports based on classification and identification results and send them to the monitoring center.

[0098] It should be noted that the terms used in the present invention are only for describing specific embodiments, rather than limiting the scope of the present application. As shown in the present specification, unless the context clearly indicates an exception, the words "one", "a", "a kind of" and / or "the" do not specifically refer to the singular, but may also include the plural. The terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method or device. In the absence of more restrictions, the elements defined by the sentence "include one..." do not exclude the presence of other identical elements in the process, method or device including the elements.

[0099] It should also be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. Unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", etc. should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A port equipment monitoring method based on image processing, characterized in that: include: Step S1, deploying a camera in the monitoring area, adjusting the camera exposure time and gain value according to the current light conditions in the monitoring area, and collecting the original image of the equipment in the monitoring area when it is running; Step S2, preprocessing the original image, the preprocessing includes: Step S21, performing resolution analysis on the original image according to the multi-scale Laplacian pyramid to obtain a feature image of each scale; Step S22, obtaining a fused motion contrast map of pixels according to the feature images of each scale, obtaining an edge strength map according to the fused motion contrast map, calculating a spatial distribution value of each pixel according to the edge strength map, obtaining a weighted edge strength map according to the spatial distribution value and the edge strength map, calculating the semantic importance of each pixel in the weighted edge strength map, and obtaining a saliency map; Calculating the spatial distribution value of each pixel point according to the edge intensity map includes: determining the center coordinates, and calculating the spatial distribution value of the pixel point according to each pixel point in the edge intensity map and the determined center coordinates; ； In the formula, Represents pixel The spatial distribution value of represents the center coordinates, ρ represents the weight; Step S3, obtaining key features of the device from the saliency map, inputting the key features into the pre-trained deep learning model, and performing classification and recognition of the device status; Step S4: Generate a monitoring report based on the classification and identification results and send it to the monitoring center.

2. A port equipment monitoring method based on image processing according to claim 1, characterized in that: Adjust the camera exposure time and gain value according to the current light conditions in the monitoring area, including: Step S11, defining a light threshold according to a light condition; the light threshold includes a first light threshold and a second light threshold; Step S12, obtaining the light intensity under the current light conditions in the monitoring area; Step S13, determining adjustment parameters under current light conditions; Step S14, adjusting the camera exposure time and gain value according to the adjustment parameter, the light intensity, the first light threshold and the second light threshold.

3. A port equipment monitoring method based on image processing according to claim 2, characterized in that: The adjustment function of camera exposure time and gain value is: ； Where, T i represents the exposure time under the i-th lighting condition, G i Represents the gain value under the i-th lighting condition, I i represents the light intensity under the i-th lighting condition, , , , , , , respectively represent the adjustment parameters used when the illumination intensity under the i-th illumination condition is greater than the first illumination threshold, , , , , , , respectively represent the adjustment parameters used when the illumination intensity under the i-th illumination condition is less than or equal to the first illumination threshold and greater than the second illumination threshold, , , , , , , respectively represent the adjustment parameters used when the illumination intensity under the i-th illumination condition is less than or equal to the second illumination threshold.

4. The port equipment monitoring method based on image processing according to claim 1 is characterized in that: According to the feature image of each scale, a fused motion contrast map of the pixel points is obtained, including: obtaining the feature image of the current frame and the feature image of the previous frame of each scale, calculating the motion contrast of each pixel point in the feature image of each scale according to the feature image of the current frame and the feature image of the previous frame to obtain the motion contrast map; and fusing the motion contrast maps of different scales to obtain the fused motion contrast map.

5. A port equipment monitoring method based on image processing according to claim 4, characterized in that: The calculation formula for motion contrast is: ； In the formula, represents the motion contrast of the j-th scale feature image pixel (x, y), Represents the pixel value of the feature image pixel (x, y) at the jth scale and the tth frame, Represents the pixel value of the feature image pixel (x, y) at the jth scale and the t-1th frame.

6. A port equipment monitoring method based on image processing according to claim 4, characterized in that: Obtaining an edge intensity map according to the fused motion contrast map includes: performing edge detection on the fused motion contrast map to obtain an edge intensity map.

7. The port equipment monitoring method based on image processing according to claim 1 is characterized in that: According to the spatial distribution value of each pixel and the edge strength map, a weighted edge strength image is obtained; the semantic importance of each pixel in the weighted edge strength image is calculated to generate a semantic feature map; the contrast between each pixel in the weighted edge strength image and other pixels is calculated to obtain a contrast map, and a saliency map is obtained according to the semantic feature map and the contrast map.

8. The port equipment monitoring method based on image processing according to claim 7 is characterized in that: The semantic importance of each pixel is calculated through a convolutional neural network.

9. A port equipment monitoring system based on image processing, used to execute the port equipment monitoring method based on image processing according to any one of claims 1 to 8, characterized in that: Includes the following modules: Image acquisition module: used to deploy cameras in the monitoring area, adjust the camera exposure time and gain value according to the current light conditions in the monitoring area, and collect the original images of the equipment in the monitoring area when it is running; Preprocessing module: connected with the image acquisition module, used to preprocess the original image; Identification module: connected to the preprocessing module, used to obtain the key features of the device from the saliency map, input the key features into the pre-trained deep learning model, and perform classification and identification of the device status; Monitoring module: connected with the identification module, used to generate monitoring reports based on classification and identification results and send them to the monitoring center.

Images