Signal indication system, data processing method and readable storage medium

Abstract

The invention discloses a signal indication system, a data processing method and a readable storage medium. The system comprises a camera unit arranged on a quay crane; the control unit is configured to receive image data of the camera shooting unit and comprehensively analyze the image data to obtain berthing and unberthing state information of ships on the two sides of the quay crane; and the display unit is configured to receive the ship berthing and unberthing state information so as to display the berthing and unberthing state, and the display unit is arranged in a wharf U-turn area. According to the embodiment of the invention, the problem that the docking and undocking states of the automatic wharf liner are still confirmed manually is solved.

Description

Technical Field

[0001] The present application relates to a signal indication system, a data processing method and a readable storage medium, belonging to the field of intelligent terminals. Background Art

[0002] Currently, the berthing and unberthing status of liners at automated terminals still depends on manual confirmation by the operation department. When a liner berths or unberths, the operation department needs to enclose the port ring road and carry out corresponding berthing and unberthing operations. During this process, the port ring road will be impassable, and vehicle U-turns are prohibited along the front shoreline of the port ring road. However, the berthing and unberthing information of the liner and the enclosure operation situation are only known to the operation department, and other relevant departments and personnel cannot obtain these key information in real time. This information asymmetry may lead to low coordination efficiency within the terminal and even affect the smoothness of the overall operation process. Therefore, there is an urgent need for a system that can monitor and share the berthing and unberthing status of liners in real time to improve the transparency and efficiency of terminal operations and reduce potential problems caused by information lag or lack. Summary of the Invention

[0003] In view of this, the present application provides a signal indication system, a data processing method and a readable storage medium. The embodiments of the present application solve the problem that the berthing and unberthing status of liners at automated terminals is still manually confirmed.

[0004] The first aspect of the embodiments of the present application discloses a signal indication system, which includes:

[0005] An imaging unit disposed on a quay crane;

[0006] A control unit configured to receive the image data of the imaging unit and comprehensively analyze it to obtain the berthing and unberthing status information of the ships on both sides of the quay crane; and

[0007] A display unit configured to receive the berthing and unberthing status information of the ship to display the berthing and unberthing status, and the display unit is disposed in the terminal U-turn area.

[0008] In one embodiment, the quay crane extends to the water surface compared with the shoreline, and the quay crane forms a preset angle with the shoreline; and / or

[0009] A plurality of the quay cranes are arranged along the port ring road, and the terminal U-turn areas are respectively disposed on the port ring road on the side of the first quay crane away from the adjacent quay crane and on the port ring road on the side of the last quay crane away from the adjacent quay crane.

[0010] In one embodiment, the imaging units are respectively disposed on both sides of the quay crane for photographing the water surface; and / or

[0011] The imaging unit is a wide-angle imaging device.

[0012] The second aspect of the embodiments of the present application discloses a data processing method, the method comprising:

[0013] S1 Receive image data;

[0014] S2 Obtain the berthing and unberthing state information of the ships on both sides of the quay crane based on the moving target detection algorithm of the frame difference method and the image data.

[0015] In one embodiment, the positive y - coordinate direction in the image coordinate system of the image data is the sea side;

[0016] The obtaining of the berthing and unberthing state information of the ships on both sides of the quay crane based on the moving target detection algorithm of the frame difference method and the image data includes:

[0017] S21 Obtain the border coordinates of the moving liner tracked and locked by the algorithm, and calculate its central pixel point as the center point of the moving target;

[0018] S22 Perform a difference operation on the y - coordinate value of the center point of the moving target in each frame of the image and the y - coordinate value of the center point in the previous frame to obtain the y - coordinate change value;

[0019] S23 Repeat step S22 within a preset number of seconds to obtain multiple y - coordinate change values, and calculate their average change value T;

[0020] S24 Compare the average change value T with a preset threshold t:

[0021] If T > t, the camera unit status value X is 1;

[0022] If T < -t, the camera unit status value X is -1;

[0023] Otherwise, the camera unit status value X is 0;

[0024] S25 Obtain the status values X1 and X2 of two camera units, and perform an OR logical operation on the status values:

[0025] If X1 is 1 and X2 is not -1, or X2 is 1 and X1 is not -1, then the output status value is 1, and it is determined that the liner is berthing;

[0026] If X1 is -1 and X2 is not 1, or X2 is -1 and X1 is not 1, then the output status value is -1, and it is determined that the liner is unberthing;

[0027] If both X1 and X2 are 0, then the output status value is 0, and it is determined that there is no berthing at this time;

[0028] If X1 is -1 and X2 is 1, or X1 is 1 and X2 is -1, then perform a lag detection, and re - execute steps S21 to S25 after a preset number of seconds.

[0029] In one embodiment, the method further includes:

[0030] S3 Obtain the berthing and unberthing information of the current terminal according to the berthing and unberthing status information of each quay crane, including which quay cranes are judged to be berthing, unberthing, or without berthing;

[0031] S4 Generate a corresponding display instruction according to the berthing and unberthing information, and send the display instruction to the display unit.

[0032] In one embodiment, the display unit displays the corresponding status according to the received display instruction, including:

[0033] S51 When the "berthing" instruction is received, the "berthing" entry on the display unit lights up;

[0034] S52 When the "unberthing" instruction is received, the "unberthing" entry on the display unit lights up;

[0035] S53 When the "berthing" or "unberthing" instruction is received, the "without berthing" entry does not light up;

[0036] S54 When the "berthing" or "unberthing" instruction is not received, the "without berthing" entry on the display unit lights up;

[0037] Wherein, the display of the "berthing" and "unberthing" information is carried out independently, and the display unit can display the two states of "berthing" and "unberthing" simultaneously.

[0038] In one embodiment, the berthing and unberthing status information of the ships on both sides of the quay crane is obtained from the motion target detection algorithm based on frame difference method and the image data, including:

[0039] S21 Based on the image data, perform four-frame difference processing on four consecutive frames of images and fuse with the Canny edge detection operator to obtain a binary motion target image, and the Canny edge detection operator is used to compensate the fourth frame of image;

[0040] S22 Dilate the motion target image and add a border;

[0041] S23 When the area of the border is greater than the dynamic threshold, determine that the motion target of the border is a moving liner;

[0042] S24 Track the moving liner based on the SORT algorithm;

[0043] S25 Determine the motion direction of the moving liner based on the front and back frame images to determine the berthing and unberthing status information of the ship. In one embodiment, the filtering method of the Canny edge detection operator is a bilateral filtering method; and / or

[0044] The boundary image threshold of the Canny edge detection operator is a locally adaptive threshold, and the locally adaptive threshold is as follows:

[0045]

[0046] where p xy is the gray value of the pixel point and its neighboring pixel points.

[0047] The third aspect of the embodiment of the present application discloses a computer-readable storage medium, and the computer-readable storage medium includes a stored program, wherein when the program runs, it controls the processor of the device where it is located to execute the data processing method of the above embodiment.

[0048] Compared with the prior art, the embodiment of the present application has the following beneficial effects:

[0049] The embodiment of the present application provides a signal indication system, a data processing method and a readable storage medium. The signal indication system includes: a camera unit arranged on the quay crane; a control unit configured to receive the image data of the camera unit and comprehensively analyze it to obtain the ship berthing and unberthing state information on both sides of the quay crane; and a display unit configured to receive the ship berthing and unberthing state information to display the berthing and unberthing state, and the display unit is arranged in the turning area of the terminal.

[0050] The present invention captures the dynamic image data of ship berthing and unberthing in real time by arranging a camera unit on the quay crane, and the control unit conducts comprehensive analysis to generate accurate ship berthing and unberthing state information, and finally displays it in real time through the display unit arranged in the turning area of the terminal. This system realizes the automatic monitoring and information sharing of the berthing and unberthing state, significantly improves the terminal operation efficiency, reduces the workload and potential errors of manual confirmation. At the same time, by intuitively displaying the berthing and unberthing state, it can standardize the enclosure operation of the circum-port road, avoid traffic chaos or safety hazards caused by information asymmetry, and further improve the safety and order of terminal operations. In addition, the intelligent design of the system reduces the labor cost, promotes the development of the terminal towards automation and intelligence, and provides reliable technical support for the efficient operation of modern ports. Description of the Drawings

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on the structures shown in these drawings.

[0052] Figure 1aSchematic diagram of an application scenario of a signal indication system provided by an embodiment of the present application.

[0053] Figure 1b Schematic diagram of an application scenario of a signal indication system provided by an embodiment of the present application.

[0054] Figure 2 Schematic diagram of communication among a camera, a quay crane industrial control computer, a server, and a display provided by an embodiment of the present application.

[0055] Figure 3 Schematic diagram of the positional relationship between a camera and a quay crane provided by an embodiment of the present application.

[0056] Figure 4 Schematic diagram of the positional relationship among a display, a quay crane, a ring port road, and a ship provided by an embodiment of the present application.

[0057] Figure 5 Schematic flowchart of a data processing method provided by an embodiment of the present application.

[0058] Figure 6 Schematic flowchart of another data processing method provided by an embodiment of the present application.

[0059] Figure 7 Schematic diagram of the structure of a data processing device provided by an embodiment of the present application. Detailed implementation manners

[0060] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0061] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention described here can be implemented in an order different from those illustrated or described here. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0062] Embodiment 1:

[0063] Figure 1a This is a schematic diagram of the application scenario of a signal indication system provided by an embodiment of the present application. As Figure 1a shown, the system may include: a camera unit 101 disposed on the quay crane; a control unit 102 configured to receive the image data of the camera unit and perform comprehensive analysis to obtain the ship berthing and unberthing status information on both sides of the quay crane; and a display unit 103 configured to receive the ship berthing and unberthing status information to display the berthing and unberthing status, and the display unit is disposed in the quay turning area.

[0064] Figure 1b This is a schematic diagram of the application scenario of a signal indication system provided by an embodiment of the present application. As Figure 1b shown, the system may include: a camera unit 101 disposed on the quay crane; a control unit 102 configured to receive the image data of the camera unit and perform comprehensive analysis to obtain the ship berthing and unberthing status information on both sides of the quay crane; and a display unit 103 configured to receive the ship berthing and unberthing status information to display the berthing and unberthing status, and the display unit is disposed in the quay turning area. Among them, the server 104 receives the ship berthing and unberthing status information and distributes it to the display unit, playing a role in data processing and transfer.

[0065] In this embodiment, the quay container crane is simply referred to as the quay crane. The camera unit 101 may be a high-definition camera or an infrared camera, the control unit 102 may be an industrial control computer or an image processing module, the display unit may be an LED display screen or a projection display system, and the server 104 may be a cloud server or an enterprise internal server, which is not limited herein.

[0066] It can be understood that the control unit may also be a processing module of the server.

[0067] Taking Figure 1b the signal indication system as an example, as Figure 2 shown, the cameras 1 and 2 on Bridge 1 and Bridge 14 respectively collect images and transmit the data to the corresponding quay crane industrial control computer; after the industrial control computer processes the images, it extracts the ship berthing and unberthing status information values and sends them to the server; after the server further integrates the data, it transmits the status information of berthing, no berthing, and unberthing to the display for the operator to monitor in real time.

[0068] In this embodiment, as Figure 3 shown, camera units for shooting towards the water surface are respectively disposed on both sides of the quay crane.

[0069] Optionally, the camera unit is a wide-angle imaging device.

[0070] In one embodiment, as Figure 3 shown, the quay crane 100 extends towards the water surface relative to the shoreline 300, and the quay crane forms a preset angle with the shoreline. The preset angle can be 75° or 90°, and the specific angle depends on the quay crane design. There is a liner 200 moving on the sea side outside the shoreline.

[0071] In one embodiment, as Figure 4 shown, multiple quay cranes are arranged along the Ring Port Road 400, and the terminal turning areas 105 are respectively set on the Ring Port Road on the side of the first quay crane away from the adjacent quay crane and on the Ring Port Road on the side of the last quay crane away from the adjacent quay crane.

[0072] Figure 5 It is a schematic flow chart of a data processing method provided by an embodiment of the present application. As Figure 5 shown, the method may include:

[0073] S1 Receive image data.

[0074] In this step, images with a real-time resolution of 1024*768 are collected. Among them, in the camera image coordinate system, the positive y coordinate direction is the sea side. There is one camera on each side of a quay crane, and a total of 24 frames of images are collected within 1 s.

[0075] S2 Obtain the ship berthing and unberthing state information on both sides of the quay crane based on the moving target detection algorithm using the frame difference method and the image data.

[0076] In this step, the improved frame difference method is used to analyze the image, mark the moving entities in the picture, and filter out the smaller entities. When the liner enters the camera's field of view, its movement trajectory will be marked. By analyzing whether the liner is moving towards the shoreline direction, it is judged whether it is berthing. The judgment result is obtained by comprehensively considering the state information of the two cameras.

[0077] On the processing logic layer, as Figure 6 shown, step S2 may include:

[0078] S21 Obtain the border coordinates of the moving liner tracked and locked by the algorithm, and calculate its central pixel point as the center point of the moving target.

[0079] S22 Perform a difference operation on the y coordinate value of the center point of the moving target in each frame of image and the y coordinate value of the center point in the previous frame to obtain the y coordinate change value.

[0080] S23 Repeat step S22 within a preset number of seconds to obtain multiple y coordinate change values, and calculate their average change value T.

[0081] S24 Compare the average change value T with a preset threshold t:

[0082] If T>t, the camera unit state value X is 1;

[0083] If T < -t, the status value X of the camera unit is -1;

[0084] Otherwise, the status value X of the camera unit is 0.

[0085] S25 Obtain the status values X1 and X2 of two camera units, and perform an OR logical operation on the status values:

[0086] If X1 is 1 and X2 is not -1, or X2 is 1 and X1 is not -1, the output status value is 1, and it is determined that the liner is berthing;

[0087] If X1 is -1 and X2 is not 1, or X2 is -1 and X1 is not 1, the output status value is -1, and it is determined that the liner is unberthing;

[0088] If both X1 and X2 are 0, the output status value is 0, and it is determined that there is no berthing at this time;

[0089] If X1 is -1 and X2 is 1, or X1 is 1 and X2 is -1, perform a lag detection, and re - execute steps S21 to S25 after a preset number of seconds.

[0090] In this embodiment, the preset number of seconds can be 1s or 2s, and the detection time is set according to actual requirements.

[0091] As a preferred implementation, if the judgment result is berthing, the industrial control computer sends the information to the front - edge display, and it is displayed as "berthing", and at the same time, the Huangan Road is prohibited from passing. Only when it is detected that there is no obvious movement within XX minutes (this time is slightly longer than the enclosure operation time) after the liner berths, the system changes the status to "no berthing", and sends it to the display, and at the same time, the Huangan Road resumes passing.

[0092] At the processing algorithm layer, step S2 may include:

[0093] S21 Based on the image data, perform four - frame difference processing on four consecutive frames of images, and fuse with the Canny edge detection operator to obtain a binary moving target image, and the Canny edge detection operator is used to compensate the fourth - frame image.

[0094] Before this step, it also includes:

[0095] 1) Pre - process the image data.

[0096] 1-1) The 1024*768 image captured by the quay crane camera is an RGB color image. Due to its large size and excessive details, it is not conducive to subsequent processing. Therefore, the YUV algorithm is used to convert the RGB image into a grayscale image that is easy to process. For each pixel of the original image, the grayscale image is calculated through the following formula (where Y is the brightness, and R, G, and B are the pixel values of the red, green, and blue channels respectively):

[0097] Y = 0.299*R + 0.587*G + 0.114*B

[0098] 1-2) There are usually picture noise points in the grayscale image, and the noise has a great impact on the image quality. Therefore, it is necessary to perform noise reduction processing on the grayscale image. In this embodiment, bilateral filtering is used for noise reduction. Bilateral filtering uses the method of weighted average, and uses the weighted average of the surrounding pixel brightness to replace the intensity of the current pixel, thereby weakening the influence of noise on the current pixel. Its principle formula is as follows:

[0099]

[0100] Among them, is the pixel brightness value after the final filtering process; is the image pixel value domain kernel; is the spatial domain kernel; w p is the normalization factor, which is the product of the image pixel value domain kernel and the spatial domain kernel; s is the neighborhood range of the center point; p is the position coordinate of the center point; q is the position coordinate of the neighborhood point; I p is the pixel brightness value of the middle point; I q is the pixel brightness value of the neighborhood pixel point.

[0101] In this step, the principle of the four-frame difference algorithm is as follows: First, the third frame and the first frame, and the fourth frame and the second frame in the continuous four frames are respectively subjected to difference calculations to obtain two difference images; then, these two difference images are subjected to AND operation, and the result is subjected to pixel dilation processing to obtain the contour image of the moving object. However, the contours detected by the traditional four-frame difference algorithm are often discontinuous and there are false edge phenomena. To solve this problem, the result of edge detection of the fourth frame using the Canny operator is introduced as compensation. First, through grayscale conversion and filtering processing, four consecutive frames of images P1(x,y), P2(x,y), P3(x,y), P4(x,y) are obtained, where x and y represent pixel positions. Then, the pixel grayscale values of the first frame and the third frame images, and the pixel grayscale values of the second frame and the fourth frame images are respectively subtracted. The operation formula is as follows:

[0102] D1(x,y) = |P1(x,y) - P3(x,y)|

[0103] D2(x,y) = |P2(x,y) - P4(x,y)|

[0104] To facilitate the selection of the moving target area, the difference images D1(x,y) and D2(x,y) are binarized, and the formula is as follows:

[0105]

[0106] where E i (x,y) is the result after binarization; T is the set detection threshold; the difference result is compared with the threshold. If the gray value of the pixel point is less than the threshold, the gray value of the pixel point is set to zero, otherwise it is set to 1, that is, white.

[0107] In this step, the principle of the Canny edge detection operator is as follows:

[0108] (1) Perform Gaussian filtering on the image.

[0109] (2) Calculate the gradient magnitude and direction according to the Sobel operator, and the formula is as follows:

[0110]

[0111] where I is the original image, d x is the gradient in the x - coordinate direction, d x is the gradient in the y - coordinate direction, S is the gradient magnitude, and θ is the gradient direction.

[0112] (3) Perform non - maximum suppression on each pixel point in the image: First, approximate the gradient direction of the pixel point to 8 direction angles (0°, 45°, 90°, 135°, 180°, 225°, 270°, 315°), and then compare the pixel intensity of this pixel point with the pixel intensities of the adjacent pixel points in the positive and negative directions of its gradient. If the intensity of this pixel point is the maximum, it is retained; otherwise, it is suppressed (set to zero).

[0113] (4) Use double thresholds to define the image boundary: Usually, a threshold upper bound and a threshold lower bound are set manually. When the gradient intensity of a pixel point is higher than the threshold upper bound, this pixel point must be a boundary and is defined as a strong boundary; when the gradient intensity is lower than the threshold lower bound, this pixel point must not be a boundary; when the gradient intensity is between the two thresholds, it is defined as a weak boundary and needs to be processed subsequently.

[0114] (5) Use hysteresis technology to define the weak boundary: If a weak - boundary pixel point is connected to a strong boundary, it is defined as a boundary; otherwise, it is deleted (defined as a non - boundary).

[0115] In one embodiment, the filtering method of the Canny edge detection operator is a bilateral filtering method; and / or

[0116] The boundary threshold of the Canny edge detection operator for defining the image boundary is a locally adaptive threshold, and the locally adaptive threshold is as follows:

[0117]

[0118] where p xy is the gray value of the pixel point and its neighboring pixel points.

[0119] Specifically, the improved Canny operator mainly changes the first-step filtering method and the fourth-step double-threshold method: in the first-step filtering, bilateral filtering is used instead of traditional Gaussian filtering to make the image clearer; in the fourth-step double-threshold method, the traditional method uses a globally defined fixed threshold, which is an empirical parameter, while the improved Canny operator uses a locally adaptive threshold, which can better adapt to the local features of the image.

[0120] The specific processing method is as follows: for an M×N digital image, a 3×3 matrix that moves smoothly in the digital image is generated, and the average gray value of the pixel points in the smoothly moving 3×3 matrix in the image is calculated as the adaptive threshold D. D is used as the upper threshold bound, and 0.5*D is used as the lower threshold bound, and then the image is subjected to double-threshold processing and hysteresis processing.

[0121] S22 Dilate the moving target image and add a border.

[0122] This step includes:

[0123] 2-1) Perform a sliding window process on the binary image: taking the target pixel point as the center, obtain a 3×3 window. Within this window, as long as the gray value of one pixel point is 1, all 9 pixel points in the window are set to 1. The specific formula is as follows:

[0124] p = p1|p2|p3|p4|p5|p6|p7|p8|p9

[0125] where p is the target pixel point, p1, p2, p3, p4, p5, p6, p7, p8, p9 are the pixels around the target pixel, and | is the OR operation.

[0126] 2-2) Process the extracted moving target area to add a border: First, traverse the input video in the x and y directions to calculate the coordinates of each pixel point; then, judge the area where the pixel value is 1 after binarization, that is, the area where the moving target is located. Among them, the maximum and minimum values of the pixel points in the x direction are used as the right and left boundaries of the rectangular border respectively, and the maximum and minimum values of the pixel points in the y direction are used as the upper and lower boundaries of the rectangular border respectively.

[0127] After this step, it also includes:

[0128] 3-1) Extract the bounding box coordinates of each moving object in the previous frame and the subsequent frame;

[0129] 3-2) Calculate the central pixel point of each moving object according to the bounding box coordinates;

[0130] 3-3) Calculate the Euclidean distance between the central pixel points of each moving object in the previous frame and the subsequent frame;

[0131] 3-4) Compare the Euclidean distance with a preset threshold;

[0132] 3-5) If the Euclidean distance is less than or equal to the threshold, determine that the two moving objects in the previous frame and the subsequent frame are the same object;

[0133] 3-6) If the Euclidean distance is greater than the threshold, determine that the two moving objects in the previous frame and the subsequent frame are different objects.

[0134] S23 When the area of the bounding box is greater than the dynamic threshold, determine that the moving object within the bounding box is a moving liner.

[0135] In this step, during the berthing and unberthing operations of the liner, only the movement of the liner needs to be detected. After extracting the moving object and adding the bounding box, it is necessary to filter the moving object. Calculate the area S of the bounding box of the moving object in the image, and compare it with the threshold P M Compare. If it is greater than or equal to the threshold, it is determined as a liner and retained; if it is less, it is deleted (set to zero), P M The calculation formula is as follows:

[0136] P M = max(P, 0.5 * S)

[0137] where P is an empirical parameter.

[0138] S24 Track the moving liner based on the SORT algorithm.

[0139] This step includes:

[0140] Before performing direction detection on the moving object, use the SORT algorithm to track the object. The SORT algorithm uses the Hungarian algorithm to match the estimated value b predicted by the Kalman filter p and the measurement value b obtained by the object detection algorithm m , and then uses b p and b m to update the current state to obtain b o , as the tracking result, where b pThe moving object bounding box bbox (bounding box) of frame t is predicted based on the information from 1 to t-1, b m is the bbox of frame t measured by the moving object detection algorithm, b o is the optimal estimated value.

[0141] The Kalman filter recurrence principle is as follows: First, calculate the state prediction value and the error covariance matrix between the state prediction value and the true state value. The formulas are as follows:

[0142]

[0143] Among them, is the predicted value of the state vector at time k-1 for time k, is the estimated value at time k-1, u k-1 is the acceleration at time k-1 and is also an input variable. A is the state transition matrix, and B is the control matrix for converting the input to the state.

[0144] Calculate the Kalman gain K based on the above two values, and then obtain the estimated value. The formulas are as follows:

[0145]

[0146] Among them, K k is the Kalman gain at time k, z k is the state measurement value at time k, v k is the measurement noise, H is the conversion matrix from the state variable to the measurement, is the estimated value at time k.

[0147] Finally, calculate the error covariance matrix between the estimated value and the true value to prepare for the next recurrence:

[0148]

[0149] The task of the Hungarian algorithm is to match the bbox of frame t with the bbox of frame t-1 pairwise to complete the tracking. The matching criterion is "minimum loss", and the loss is represented in the form of a loss matrix. In the SORT algorithm, the loss matrix is defined by the IOU (intersection over union) of b p and b m to define the loss matrix.

[0150] When an object enters and leaves the image, a unique identity needs to be created or destroyed accordingly. To create a tracker, SORT considers any detection with an overlap less than IOUmin to indicate the presence of an untracked object. The tracker is initialized using the geometry of the bounding box with a speed set to zero.

[0151] S25 Determine the moving direction of the moving liner based on the front and rear frame images, so as to determine the ship berthing and unberthing status information.

[0152] This step refers to the operation of the processing logic layer and will not be elaborated here.

[0153] In one embodiment, the method further includes:

[0154] S3 Obtain the berthing and unberthing information of the current terminal according to the berthing and unberthing status information of each quay crane, including which quay cranes are judged to be berthing, unberthing or without berthing.

[0155] S4 Generate a corresponding display instruction according to the berthing and unberthing information, and send the display instruction to the display unit. In one embodiment, the display unit displays the corresponding status according to the received display instruction, including:

[0156] S51 When receiving the "berthing" instruction, the "berthing" entry on the display unit lights up.

[0157] S52 When receiving the "unberthing" instruction, the "unberthing" entry on the display unit lights up.

[0158] S53 When receiving the "berthing" or "unberthing" instruction, the "without berthing" entry does not light up.

[0159] S54 When not receiving the "berthing" or "unberthing" instruction, the "without berthing" entry on the display unit lights up.

[0160] Among them, the display of the "berthing" and "unberthing" information is independent, and the display unit can display the two states of "berthing" and "unberthing" at the same time.

[0161] In some embodiments, obtaining the ship berthing and unberthing status information based on the output status value includes: if the output status value is 1, it is determined that the liner is berthing, and the berthing and unberthing status information is recorded as the first eigenvalue; if the output status value is -1, it is determined that the liner is unberthing, and the berthing and unberthing status information is recorded as the second eigenvalue; if the output status value is 0, it is determined that there is no berthing at this time, and the berthing and unberthing status information is recorded as the third eigenvalue.

[0162] In this embodiment, the first eigenvalue is 1, the second eigenvalue is 2, and the third eigenvalue is 3.

[0163] In some embodiments, the control unit is applied to the shore crane industrial control computer, and the display unit is a display; the signal indication system further includes a server, which is configured to: obtain the berthing and unberthing information of the current terminal according to the berthing and unberthing status information sent by each shore crane industrial control computer, including which shore cranes are judged to be berthing, unberthing or without berthing; generate corresponding display instructions according to the berthing and unberthing information, and send the display instructions to the display; wherein, the display displays the corresponding status according to the received display instructions, including: when receiving the "berthing" instruction, the "berthing" entry on the display lights up; when receiving the "unberthing" instruction, the "unberthing" entry on the display lights up; when receiving the "berthing" or "unberthing" instruction, the "without berthing" entry does not light up; when not receiving the "berthing" or "unberthing" instruction, the "without berthing" entry on the display lights up; wherein, the display of the "berthing" and "unberthing" information is independent, and the display can simultaneously display the two states of "berthing" and "unberthing".

[0164] In some embodiments, the control unit is applied to the shore crane industrial control computer, and the display unit is a display; the signal indication system further includes a server, which is configured to: obtain the berthing and unberthing information of the current terminal according to the berthing and unberthing status information sent by each shore crane industrial control computer, including which shore cranes are judged to be berthing, unberthing or without berthing; generate corresponding display instructions according to the berthing and unberthing information, and send the display instructions to the display; wherein, the display distinguishes the corresponding status by color according to the received display instructions, including: when receiving the "berthing" instruction, the "berthing" entry on the display is displayed in red; when receiving the "unberthing" instruction, the "unberthing" entry on the display is displayed in yellow; when receiving the "berthing" or "unberthing" instruction, the "without berthing" entry is displayed in gray; when not receiving the "berthing" or "unberthing" instruction, the "without berthing" entry on the display is displayed in green; wherein, the display of the "berthing" and "unberthing" information is independent, and the display can simultaneously display the two states of "berthing" and "unberthing" in red and yellow respectively.

[0165] In some embodiments, when the liner unberths, the operation department needs to initiate an unberthing operation application, and the system reads this information and displays it as "unberthing", and at the same time, the ring road around the port is prohibited from passing. The system continuously detects the liner. If its moving direction is unberthing, the "unberthing" state is maintained. When no large vertical shoreline moving entity is detected in the picture and the liner is not in the berthing state, the system judges it as "without berthing" and sends this information to the display.

[0166] Embodiment 2:

[0167] Figure 7 The structural schematic diagram of a data processing device provided by an embodiment of the present application is asFigure 7 As shown in the figure, the device may include the following modules:

[0168] A receiving module 701, configured to receive image data.

[0169] A detection and judgment module 702, configured to obtain the ship berthing and unberthing state information on both sides of the quay crane based on the moving target detection algorithm using the frame difference method and the image data.

[0170] Embodiment 3:

[0171] An embodiment of the present application further provides an electronic device, including: a memory storing an executable program; a processor configured to run the program, wherein when the program runs, it executes the methods in various embodiments of the present invention.

[0172] The above-mentioned memory may refer to a device inside a computer for storing data and programs, which may include a memory, a hard disk, etc. Among them, the memory can be used to temporarily store the running programs and data, and the hard disk can be used to store programs and data for a long time. The memory can be used to enable the computer to read and write data and execute programs; the above-mentioned processor can be responsible for executing the instructions in the computer program and performing data processing, and can be responsible for controlling and executing various operations, including arithmetic operations, logical operations, data transmission, etc.

[0173] Embodiment 4:

[0174] An embodiment of the present application further provides a computer-readable storage medium, the computer-readable storage medium including a stored executable program, wherein when the executable program runs, it controls the device where the computer-readable storage medium is located to execute the methods in various embodiments of the present invention.

[0175] The above-mentioned computer storage medium may refer to a medium in a computer memory for storing certain discontinuous physical quantities. The computer storage medium mainly includes semiconductors, magnetic cores, magnetic drums, magnetic tapes, laser discs, etc.; the stored program included in the computer-readable storage medium can be a set of instructions that can be recognized and executed by a computer, running on an electronic computer, and is an information tool to meet certain needs of people.

[0176] Embodiment 5:

[0177] An embodiment of the present application further provides a computer program product, including a computer program, and the computer program implements the methods in various embodiments of the present invention when executed by a processor.

[0178] The above-mentioned computer program product may refer to a software program that has been written, tested, and released, and can run on a computer or other devices. The computer program product may include application programs, operating systems, tool software, etc., and is used to implement specific functions or solve specific problems.

[0179] Example 6:

[0180] An embodiment of the present application also provides a computer program product, including a non-volatile computer-readable storage medium for storing a computer program, and when the computer program is executed by a processor, it implements the methods in various embodiments of the present invention.

[0181] The above-mentioned non-volatile computer-readable storage medium may refer to a medium for storing data. The non-volatile computer-readable storage medium can retain data without loss when power is off and can be used to store data for long-term preservation, such as operating systems, application programs, and user files. The non-volatile storage medium may include hard disk drives, solid-state drives, optical discs, and flash storage devices, etc.

[0182] Example 7:

[0183] An embodiment of the present application also provides a computer program, and when the computer program is executed by a processor, it implements the methods in various embodiments of the present invention described above.

[0184] The above-mentioned computer program may refer to a set of instructions for telling a computer to perform specific tasks or operations. The computer program can be written by a programmer using a specific programming language and may include contents such as algorithms, data structures, logic, and control flows. The computer program can be used for various purposes, including application software, operating systems, etc.

[0185] In the above embodiments of the present invention, the descriptions of various embodiments have their own focuses. For parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0186] In several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are merely illustrative. For example, the division of the units can be a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings or direct couplings or communication connections shown or discussed with each other can be through some interfaces. The indirect couplings or communication connections of units or modules can be electrical or other forms.

[0187] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0188] In addition, in each embodiment of the present invention, each functional unit can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0189] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present invention. The foregoing storage medium includes: various media such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks, or optical discs that can store program codes.

[0190] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A signal indication system, characterized in that: include: A camera unit installed on the quay bridge; A control unit is configured to receive the image data of the camera unit and conduct comprehensive analysis to obtain the berthing and unberthing status information of the ships on both sides of the quay bridge; as well as The display unit is configured to receive the ship berthing and unberthing status information to display the berthing and unberthing status, and the display unit is arranged in the turning area of ​​the dock.

2. The signal indication system according to claim 1, characterized in that: The quay bridge extends to the water surface relative to the shoreline, and the quay bridge forms a preset angle with the shoreline; and / or A plurality of the quay cranes are arranged along the harbor ring road, and the wharf turning areas are respectively arranged on the harbor ring road on the side of the first quay crane away from the adjacent quay crane and on the harbor ring road on the side of the last quay crane away from the adjacent quay crane.

3. The signal indication system according to claim 1, characterized in that: The camera units for shooting toward the water surface are respectively arranged on both sides of the quay bridge; and / or The camera unit is a wide-angle camera device.

4. A data processing method, characterized in that: include: S1 receives image data; S2 obtains the berthing and unberthing status information of the ships on both sides of the quay bridge based on the moving target detection algorithm of the frame difference method and the image data.

5. The data processing method according to claim 4, characterized in that: The positive y coordinate direction in the image coordinate system of the image data is the sea side; The moving target detection algorithm based on the frame difference method and the image data obtain the berthing and unberthing status information of the ships on both sides of the quay bridge, including: S21 obtains the frame coordinates of the moving liner tracked and locked by the algorithm, and calculates its central pixel point as the center point of the moving target; S22 performs a differential operation on the y coordinate value of the center point of the moving target in each frame of the image and the y coordinate value of the center point in the previous frame to obtain a y coordinate change value; S23 repeats step S22 within a preset second to obtain multiple y-coordinate change values, and calculates the average change value T thereof; S24 compares the average change value T with a preset threshold value t: If T>t, the camera unit state value X is 1; If T<-t, the camera unit state value X is -1; Otherwise, the camera unit state value X is 0; S25 obtains the state values ​​X1 and X2 of the two camera units, and performs an OR logic operation on the state values: If X1 is 1 and X2 is not -1, or X2 is 1 and X1 is not -1, the output status value is 1, judging that the liner is berthing; If X1 is -1 and X2 is not 1, or X2 is -1 and X1 is not 1, the output status value is -1, judging that the liner is leaving the berth; If X1 and X2 are both 0, the output state value is 0, and it is judged that there is no berthing at this time; If X1 is -1 and X2 is 1, or X1 is 1 and X2 is -1, a delayed detection is performed and steps S21 to S25 are executed again after a preset number of seconds.

6. The data processing method according to claim 4, characterized in that: The method further comprises: S3 obtains the berthing and unberthing information of the current terminal according to the berthing and unberthing status information of each quay crane, including which quay cranes are judged to be berthed, unberthed or not berthed; S4 generates a corresponding display instruction according to the berthing and unberthing information, and sends the display instruction to a display unit.

7. The data processing method according to claim 6, characterized in that: The display unit displays the corresponding state according to the received display instruction, including: S51 When the "Docking" command is received, the "Docking" item on the display unit lights up; S52 When the "unmooring" command is received, the "unmooring" item on the display unit lights up; S53 When receiving the "Docked" or "Undocked" command, the "No Docked" item does not light up; S54 When no "docking" or "undocking" command is received, the "no docking" item on the display unit lights up; The display of the "berthing" and "unberthing" information is performed independently, and the display unit can display the two states of "berthing" and "unberthing" at the same time.

8. The data processing method according to claim 4, characterized in that: The moving target detection algorithm based on the frame difference method and the image data obtain the berthing and unberthing status information of the ships on both sides of the quay bridge, including: S21 performs four-frame difference processing on four consecutive frames of images based on the image data, and fuses them with a Canny edge detection operator to obtain a binary moving target image, wherein the Canny edge detection operator is used to compensate the fourth frame of image; S22 dilates the moving target image and adds a border; S23, when the area of ​​the frame is greater than the dynamic threshold, determining that the moving target of the frame is a moving liner; S24 tracks the moving liner based on a SORT algorithm; S25 determines the moving direction of the moving liner based on the preceding and following frame images to determine the berthing and unberthing status information of the ship.

9. The data processing method according to claim 8, characterized in that: The filtering method of the Canny edge detection operator is a bilateral filtering method; and / or The image boundary threshold of the Canny edge detection operator is a local adaptive threshold, and the local adaptive threshold is as follows: Among them, p xy is the gray value of the pixel and its neighboring pixels.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the data processing method according to any one of claims 5 to 9 is executed in a processor of the device where the program is controlled.

Images