A method and device for detecting personnel against smashing on board a ship

Abstract

The present invention provides a method and device for detecting personnel on board against being smashed, wherein the method comprises: fusing and analyzing point cloud data collected by a 3D laser ranging radar and video data collected by a camera to obtain detection data; assessing the risk of personnel being smashed based on the detection data; and executing an emergency plan according to the risk assessment result. The method for detecting personnel on board against being smashed of the present invention detects the personnel directly under the spreader in real time through a 3D laser radar and a video camera. Once there is a risk of being smashed, the quay crane driver is notified to pay attention, and the spreader is controlled to slow down or brake to avoid the occurrence of safety accidents.

Description

Technical Field

[0001] The invention relates to the technical field of early warning for safe operation of quay cranes, and in particular to a method and a device for detecting personnel on board from being smashed. Background Art

[0002] At the shore of the container terminal, when the quay crane is loading and unloading ships, workers need to go on board to direct the operation and remove the container connection locks. When the quay crane driver is operating the spreader, there is a situation where the spreader is directly above the workers on board, which poses a major safety risk of people being hit.

[0003] The quay crane operation specification stipulates that workers are not allowed to be under the spreader during quay crane operation. However, the complex working environment on site and fatigue caused by high-intensity work will lead to the risk of falling on people. The main risks are as follows:

[0004] The crew on board lowered their heads while working and did not notice the quay crane appearing above their heads.

[0005] The quay crane is generally 60 meters high. When the driver observes below from the cab on top of the quay crane, the targets of the crew on board are too small, and due to blind spots in the field of vision, the crew under the spreader are not spotted.

[0006] There is a risk of people being hit due to inattention or misoperation by shipboard staff or quay crane drivers.

[0007] Currently, risk prevention mainly relies on the constraints of terminal rules and regulations and self-identification of danger by shipboard personnel and quay crane drivers, which cannot be completely avoided. Summary of the invention

[0008] One of the purposes of the present invention is to provide a method for detecting personnel on board against being smashed, which detects personnel directly under the spreader in real time through 3D laser radar and video camera. Once there is a risk of being smashed, the quay crane driver is notified and the spreader is controlled to slow down or brake to avoid the occurrence of safety accidents.

[0009] An embodiment of the present invention provides a method for detecting a person on board a ship from being smashed, comprising:

[0010] The point cloud data collected by the 3D laser ranging radar and the video data collected by the camera are fused and analyzed to obtain the detection data;

[0011] Based on the test data, assess the risk of personnel being hit;

[0012] Implement emergency response plans based on risk assessment results.

[0013] Preferably, the 3D laser ranging radar and the camera correspond one to one and are configured in two groups; one group is arranged at the bottom of the quay crane cab, and the other group is arranged at the front edge of the trolley platform.

[0014] Preferably, the fusion analysis step comprises:

[0015] Input the video data into a 2D object detector to obtain first detection output data;

[0016] Input the point cloud data into the 3D object detector to obtain second detection output data;

[0017] Fusing the first detection output data and the second detection output data to obtain fused data;

[0018] Sparse matrix calculation is performed on the fused data, and the output data after calculation is filled and compressed to obtain the output result of the model.

[0019] Preferably, the filling step comprises:

[0020] For non-empty elements, a 1*1 convolution is performed to obtain a k*n*1 t out Tensor.

[0021] Preferably, the detection data includes:

[0022] One or more combinations of the horizontal position of the sling, the vertical height of the sling, the direction of movement of the sling, the size of the sling, the vertical distance between the sling and the personnel, and whether the personnel are within the vertical projection range of the sling.

[0023] Preferably, the steps for assessing the risk of personnel being hit are as follows:

[0024] According to the movement trend of the spreader, determine whether there is a possibility of coordinate overlap between the spreader and the personnel;

[0025] If not, it is assessed as no risk;

[0026] If yes, determine the collision time;

[0027] When the collision time is less than or equal to the preset first time threshold, it is assessed as high risk;

[0028] When the collision time is greater than the first time threshold and less than or equal to the preset second time threshold, it is assessed as medium risk;

[0029] When the collision time is greater than the second time threshold and less than or equal to the preset third time threshold, it is assessed as low risk;

[0030] When the collision time is greater than the third time threshold, it is assessed as no risk.

[0031] Preferably, the emergency plan includes:

[0032] When the risk is low, the sound and light alarm set in the quay crane cab will sound an alarm;

[0033] When there is a medium risk, the spreader is controlled to slow down and the sound and light alarm installed in the quay crane cab is controlled to sound an alarm;

[0034] When the risk is high, the spreader is controlled to stop moving and the sound and light alarm installed in the quay crane cab is controlled to sound an alarm.

[0035] The present invention also provides a device for detecting personnel against being smashed on board a ship, comprising: a positioning module, an evaluation module and an emergency processing module; wherein the positioning module performs a fusion analysis on point cloud data collected by a 3D laser ranging radar and video data collected by a camera to obtain detection data; the evaluation module evaluates the risk of personnel being smashed based on the detection data; and the emergency processing module executes an emergency plan according to the risk evaluation result.

[0036] Preferably, the 3D laser ranging radar and the camera correspond one to one and are configured in two groups; one group is arranged at the bottom of the quay crane cab, and the other group is arranged at the front edge of the trolley platform.

[0037] Preferably, the fusion analysis step comprises:

[0038] Input the video data into a 2D object detector to obtain first detection output data;

[0039] Input the point cloud data into the 3D object detector to obtain second detection output data;

[0040] Fusing the first detection output data and the second detection output data to obtain fused data;

[0041] Sparse matrix calculation is performed on the fused data, and the output data after calculation is filled and compressed to obtain the output result of the model.

[0042] Preferably, the steps for assessing the risk of personnel being hit are as follows:

[0043] According to the movement trend of the spreader, determine whether there is a possibility of coordinate overlap between the spreader and the personnel;

[0044] If not, it is assessed as no risk;

[0045] If yes, determine the collision time;

[0046] When the collision time is less than or equal to the preset first time threshold, it is assessed as high risk;

[0047] When the collision time is greater than the first time threshold and less than or equal to the preset second time threshold, it is assessed as medium risk;

[0048] When the collision time is greater than the second time threshold and less than or equal to the preset third time threshold, it is assessed as low risk;

[0049] When the collision time is greater than the third time threshold, it is assessed as no risk.

[0050] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0051] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0053] Figure 1 A schematic diagram of a method for detecting a person on board a ship from being smashed in an embodiment of the present invention;

[0054] Figure 2 This is a schematic diagram of an analysis process in an embodiment of the present invention;

[0055] Figure 3 A network structure diagram of a target detection algorithm in an embodiment of the present invention;

[0056] Figure 4 A schematic diagram of a coordinate system for analyzing an application in an embodiment of the present invention;

[0057] Figure 5 It is a schematic diagram of a ship personnel anti-smashing detection device in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0059] The embodiment of the present invention provides a method for detecting a person on board a ship from being smashed. Figure 1 As shown, including:

[0060] Step 1: Fusion analysis is performed on the point cloud data collected by the 3D laser ranging radar and the video data collected by the camera to obtain detection data;

[0061] The equipment corresponding to the present invention includes: industrial computers, PLCs, 3D laser ranging radars, cameras, alarm terminals and other equipment; wherein the industrial computers and PLC controllers are in the quay crane room and communicate with the front-end acquisition equipment through the network. The front-end acquisition includes 3D laser radars and cameras, which are installed on the mobile platform of the quay crane trolley and move from the sea side to the land side with the spreader. The specific location includes a group of cameras and radars installed at the bottom of the driver's cab and another installed at the front of the trolley platform. This installation method collects point cloud and video data from both sides of the container to avoid blind spots in the field of vision caused by being blocked by the operating container. After startup, the laser radar and camera start to collect data. Then, through the 3D multi-model target detection algorithm, the spreader and personnel location information is obtained by fusing the laser radar 3D point cloud information and the camera visual information.

[0062] The shape, size and position information of the spreader are crucial to the accuracy of detection for personnel protection. In traditional quay crane tally intelligent identification projects, the trolley coordinate information in the quay crane PLC is often used as the main expression information of the spreader position characteristics. However, due to its delayed nature, PLC information cannot meet the requirements of real-time high frame rate detection. The PLC information docking method corresponding to different quay crane equipment is complicated. In addition, during the operation, the spreader position cannot be completely replaced by the trolley position information due to the inertia of movement. Therefore, this expression of spreader information is actually not accurate. Use multiple laser radars and cameras installed on the quay crane arm to obtain real-time point cloud and image information of the operation to complete the 3D target detection of the spreader, and then obtain the size of the spreader. The spreader movement speed and direction can be calculated through the numerical changes in the x and y directions.

[0063] Personnel detection is different from spreader detection. The size of personnel targets is small compared to the entire quay crane itself. For single vision detection, images are a regular and dense data type. By designing a feature extraction network in a deep learning algorithm, the accuracy and recall rate of small target detection can be effectively enhanced. However, this method can only obtain the pixel position of the personnel in the pixel coordinate system, but cannot obtain the real position of the personnel in the world coordinate system. At the same time, the optical imaging of the camera will inevitably produce other problems such as distortion. For the personnel anti-smashing function, the detection method of single image information cannot meet the warning distance of personnel anti-smashing and the high-precision positioning requirements of personnel. For single point cloud detection, 3D point cloud can provide location information well, but point cloud data is an unordered and sparse data type. The smaller the size of the detected target and the farther it is from the radar target, the fewer features are obtained by the laser radar. Similarly, for personnel anti-smashing, the detection method of single point cloud information cannot meet the requirements.

[0064] Multimodal learning is often better than unimodal learning in specific scenarios; 1. The quality of the latent representation space determines the effect of the multimodal model; 2. When the data volume reaches a certain scale, the more complete the modal types, the better the effect of the multimodal model. In this patent, the distribution and number of sensors are reasonable, the amount of data obtained is large, and the demand scenarios are relatively regular. Therefore, the use of multimodal detection algorithms can well make up for the shortcomings of single modality and meet project requirements.

[0065] The CLOCS (Camera-LiDAR Object Candidates) multimodal 3D target detection algorithm is used to complete the target detection of the hoist and the operator. The main network structure diagram is as follows Figure 3 As shown, it can be seen that there are three main stages: (1) 2D and 3D target detectors propose proposals respectively; (2) the proposals of the two modes are encoded into sparse tensors; (3) for non-empty elements, two-dimensional convolution is used to perform corresponding feature fusion. The specific algorithm is:

[0066] For 2D detectors generated The test result (i.e. the first test output data) is recorded as:

[0067]

[0068] In the formula, Represents an image Candidate detection objects. It represents some basic parameters and confidence of the detection box, specifically Indicates the coordinates of the upper left corner, Represents the coordinates of the lower right corner, For confidence.

[0069] For a 3D cloud scene, there are detection candidate objects (i.e., the second detection output data), which are recorded as:

[0070]

[0071]

[0072] In the formula, Represents the 3D cloud image Candidate detection objects. Represents some basic parameters and confidence of the detection box, specifically: Indicates the height of the detection box. Indicates the width of the detection box. Indicates the length of the detection box, are the coordinates of the center point, is the confidence level;

[0073] for 2D detection results, and 3D detection results, build a Tensor (Fused data). For each element of the tensor, it consists of four channels;

[0074]

[0075] in, The value calculated by the projection of the 2D detection box and the 3D detection box on the image; , It represents the confidence score of 2D and 3D detection boxes; It means the The 3D detection box is located in the point cloud , Normalized distance between planes. Tensor Lieutenant General Elements that are zero are removed because they are not geometrically consistent. Perform sparse matrix calculations. For p non-empty elements, a series of 1*1 convolutions are performed to obtain k*n*1 t out Tensor, padded and compressed to get the output of the model.

[0076] During the test, a coordinate system needs to be specified. The coordinate system consists of the origin, X-axis, Y-axis and Z-axis. The coordinate axes intersect each other perpendicularly. Figure 4 As shown, the origin is the installation position of the 3D laser radar and camera equipment, the x-axis is the moving direction of the trolley, the z-axis is the moving direction of the spreader, and the y-axis is the direction perpendicular to the moving direction of the trolley. The spreader size can be calculated through the x-direction and z-direction arrays of the point cloud, and the spreader moving speed and moving direction can be calculated through the changes in the Y-direction array.

[0077] The test data include:

[0078] One or more combinations of the horizontal position of the sling, the vertical height of the sling, the direction of movement of the sling, the size of the sling, the vertical distance between the sling and the personnel, and whether the personnel are within the vertical projection range of the sling.

[0079] Step 2: Based on the test data, assess the risk of personnel being hit;

[0080] According to the movement trend of the hoist, determine whether there is a possibility of coordinate overlap between the hoist and the personnel; if not, it is assessed as no risk; if so, determine the collision time; when the collision time is less than or equal to the preset first time threshold, it is assessed as high risk; when the collision time is greater than the first time threshold and less than or equal to the preset second time threshold, it is assessed as medium risk; when the collision time is greater than the second time threshold and less than or equal to the preset third time threshold, it is assessed as low risk; when the collision time is greater than the third time threshold, it is assessed as no risk;

[0081] The specific analysis steps are:

[0082] Set point is the location of the personnel, and its coordinates are ; Set point is the location of the spreader, click In the The coordinates of the second are ; The spreader position is sampled once per second, and the first time point The coordinates of , the second time point The coordinates of ; Calculate the velocity vector. The velocity vector can be calculated by the coordinates of the two sampling points. The calculation formula is as follows: ; Calculate the time required for collision, assuming Continue along the velocity vector The direction of linear motion can be predicted At any time in the future The position coordinates are: , ;in, , ; To detect collision, click At some point in the future The distance is zero; that is ; Expand and simplify to get:

[0083] , which means a The quadratic equation can be solved by the following steps to simplify it. , expand the equation and put it into standard quadratic form: , using the root-finding formula Solve ;in, ; , ; Determine the feasibility of collision, if , then there is no real solution, indicating that Will not with the point collision; if , then there is a real number solution, which means that a collision may occur, and the smaller positive root is taken as the collision time.

[0084] Step 3: Implement emergency plan according to risk assessment results.

[0085] When the risk is low, the sound and light alarm set in the cab of the quay crane will sound an alarm; when the risk is medium, the spreader will slow down and the sound and light alarm set in the cab of the quay crane will sound an alarm; when the risk is high, the spreader will stop and the sound and light alarm set in the cab of the quay crane will sound an alarm, and the driver needs to check the risk before operating. Figure 2 This is a specific analysis and early warning flow chart for this application.

[0086] In one embodiment, the method for detecting the personnel on board from smashing the vessel further includes:

[0087] Monitor the status of the lifting hook of the hoist to determine whether to carry out lifting; when lifting is carried out, collect the point cloud data of the 3D laser ranging radar and the video data of the camera.

[0088] This embodiment monitors the state of the hook to trigger the detection, thereby reducing the analysis frequency and unnecessary energy consumption. The state monitoring of the hook can be achieved by providing a contact sensor on the hook and / or obtaining the state monitoring data of the hook through communication with the quay crane control terminal in the quay crane machine room.

[0089] In one embodiment, the emergency plan further includes:

[0090] When the spreader is decelerated and / or braked, the broadcast module configured on the trolley platform sends reminder broadcast data; the reminder terminal worn by the corresponding target personnel issues an early warning after receiving the reminder broadcast data; the broadcast data includes: avoidance direction and countdown;

[0091] The initial value of the countdown is obtained by querying a pre-configured correspondence table of risk value and time according to the risk value; in the correspondence table, the risk value and the initial value of the countdown are associated one-to-one;

[0092] The avoidance direction is from the center position of the vertical projection of the sling to the position of the personnel.

[0093] This embodiment enables the ground personnel to take active avoidance actions by reminding them, thereby further improving the safety of the personnel. In addition, since reminding the personnel to avoid an emergency situation may cause the personnel to panic and run in the wrong direction, it is necessary to monitor the movement direction of the personnel after receiving the warning, and the movement direction of the personnel is from the position at the previous moment to the position at the current moment. The angle between the movement direction and the avoidance direction is calculated by the cosine angle calculation method. When the angle is less than the preset angle threshold (90), no secondary warning is issued. When the angle is greater than or equal to the angle threshold, a secondary warning is issued, and the information of the secondary warning includes steering prompt information. The steering prompt information includes: turn left by X degrees, turn right by X degrees. The specific prompt information can be queried through the angle prompt information database to obtain the prompt information corresponding to the angle.

[0094] Due to the broadcast method adopted, all ground workers can receive the broadcast, how to accurately deliver the early warning; broadcast antennas can be set up next to each group of 3D laser ranging radars and cameras, and the two antennas broadcast at the same time. There is a time difference between the two broadcast data received on the ground; in this way, the wearable terminal will receive two broadcast data; when the last broadcast data is received, the time difference between the two times is broadcasted, and then the time difference of the broadcasts received from other terminals and its own time difference are sorted from small to large, and the number of early warnings in the broadcast data (the number of people whose risk value exceeds the alarm threshold at the current moment) is used to determine whether the person corresponding to the mobile terminal is in the early warning object (when the sequence number is less than or equal to the number of early warnings, it is in the early warning object; otherwise, it is not), and when it is, an early warning is issued; in addition, the data corresponding to the early warning object in the broadcast data can be extracted according to the sequence number; that is, the above method can realize the early warning of multiple people in the dangerous range, and the broadcast data is the integration of the early warning data of all personnel. The early warning of the wearable terminal includes: one or more combinations of vibration, LED display and speaker broadcast.

[0095] The integration of broadcast data is to sort the distances of the corresponding broadcast positioning points in the coordinate system in advance according to the location distances of each warning object, and then mark them according to the serial numbers; for those with the same distances, they are sorted in the order of the data sent by the antenna under the cab first; at this time, the data that the wearable terminal needs to broadcast externally also includes the corresponding identification data of which antenna sent the data received first.

[0096] The risk assessment analysis in step 2 is performed on the plane where the person is located, and the default vertical projection range (points on the boundary are used as the analysis The point) is the position where the sling falls without any other influence; due to the complexity of the working conditions, the actual fall cannot be directly opposite the vertical projection range. This embodiment provides a more accurate determination of the positional relationship, specifically including: obtaining the operation data of the sling, the size of the sling, the vertical height of the sling and the current wind direction and wind force as analysis data; performing feature extraction on the analysis data, and constructing a state parameter set based on the extracted feature values; determining the influence area map from a pre-configured influence area determination library based on the state parameter set; overlapping the positioning area in the influence area map with the vertical projection range to achieve the determination of the influence area.

[0097] Among them, the state parameter set includes: parameters indicating the wind speed, parameters indicating the wind direction, parameters indicating the size of the sling, parameters indicating the vertical height of the sling, parameters indicating the movement direction of the sling, parameters indicating the stress conditions of each sling of the sling, etc.; the impact area determination library is constructed in advance based on a large amount of simulation test data through analysis by professionals, and the impact area map in the impact area determination library corresponds to the state parameter set one by one; and the impact area map includes: positioning area and impact area; the impact area is a collection of areas where the position may fall when actually falling in the construction data; the positioning area corresponds to the vertical projection range of the sling in the construction data.

[0098] The present invention also provides a shipboard personnel anti-smashing detection device, such as Figure 5 As shown, it includes: a positioning module 1, an evaluation module 2 and an emergency processing module 3; wherein the positioning module 1 performs a fusion analysis on the point cloud data collected by the 3D laser ranging radar and the video data collected by the camera to obtain the detection data; the evaluation module 2 evaluates the risk of personnel being hit based on the detection data; the emergency processing module 3 executes the emergency plan according to the risk assessment result.

[0099] Among them, the 3D laser ranging radar and the camera correspond one to one and are configured in two groups; one group is set at the bottom of the quay crane cab, and the other group is set at the front of the trolley platform.

[0100] The fusion analysis steps include:

[0101] Input the video data into a 2D object detector to obtain first detection output data;

[0102] Input the point cloud data into the 3D object detector to obtain second detection output data;

[0103] Fusing the first detection output data and the second detection output data to obtain fused data;

[0104] Sparse matrix calculation is performed on the fused data, and the output data after calculation is filled and compressed to obtain the output result of the model.

[0105] Among them, the assessment steps for the risk of people being hit are as follows:

[0106] According to the movement trend of the spreader, determine whether there is a possibility of coordinate overlap between the spreader and the personnel;

[0107] If not, it is assessed as no risk;

[0108] If yes, determine the collision time;

[0109] When the collision time is less than or equal to the preset first time threshold, it is assessed as high risk;

[0110] When the collision time is greater than the first time threshold and less than or equal to the preset second time threshold, it is assessed as medium risk;

[0111] When the collision time is greater than the second time threshold and less than or equal to the preset third time threshold, it is assessed as low risk;

[0112] When the collision time is greater than the third time threshold, it is assessed as no risk.

[0113] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for detecting personnel on board from being smashed, characterized in that: include: The point cloud data collected by the 3D laser ranging radar and the video data collected by the camera are fused and analyzed to obtain the detection data; Based on the test data, assess the risk of personnel being hit; Implement emergency response plans based on risk assessment results; Among them, the assessment steps for the risk of people being hit are as follows: According to the movement trend of the spreader, determine whether there is a possibility of coordinate overlap between the spreader and the personnel; If not, it is assessed as no risk; If yes, determine the collision time; When the collision time is less than or equal to the preset first time threshold, it is assessed as high risk; When the collision time is greater than the first time threshold and less than or equal to the preset second time threshold, it is assessed as medium risk; When the collision time is greater than the second time threshold and less than or equal to the preset third time threshold, it is assessed as low risk; When the collision time is greater than the third time threshold, it is assessed as no risk; The steps for collision analysis are: Set point is the location of the personnel, and its coordinates are ; Set point is the location of the spreader, point In the The coordinates of the second are ; The spreader position is sampled once per second, and the first time point The coordinates of , the second time point The coordinates of ; Calculate the velocity vector. The velocity vector is calculated through the coordinates of the two sampling points. The calculation formula is as follows: ; Calculate the time required for collision, assuming Continue along the velocity vector Direction of linear motion, prediction At any time in the future The position coordinates are: , ;in, , ; To detect collision, click At some point in the future The distance is zero; that is ; Expand and simplify to get: , which means a The quadratic equation can be solved by the following steps: , expand the equation and put it into standard quadratic form: , using the root-finding formula Solve ;in, ; , ; Determine the feasibility of collision, if , then there is no real solution, indicating that Will not with the point Collision; if , then there is a real number solution, which means that a collision may occur, and the smaller positive root is taken as the collision time; The position of the spreader involved in the collision analysis corresponds to the impact area of ​​the spreader falling; the steps to determine the impact area are as follows: Acquire the operation data of the spreader, the size of the spreader, the vertical height of the spreader and the current wind direction and wind force as analysis data; extract features from the analysis data and construct a state parameter set based on the extracted feature values; determine the influence area map from a pre-configured influence area determination library based on the state parameter set; overlap the positioning area in the influence area map with the vertical projection range to determine the influence area; Among them, the state parameter set includes: parameters representing the wind speed, parameters representing the wind direction, parameters representing the size of the sling, parameters representing the vertical height of the sling, parameters representing the movement direction of the sling, and parameters representing the stress conditions of each sling of the sling; the impact area determination library is constructed in advance based on a large amount of simulation test data through analysis, and the impact area map in the impact area determination library corresponds one to one with the state parameter set; and the impact area map includes: positioning area and impact area; the impact area is a collection of areas where the position may fall during actual falling in the construction data; the positioning area corresponds to the vertical projection range of the sling in the construction data.

2. The method for detecting the personnel on board from being smashed as claimed in claim 1, characterized in that: The 3D laser ranging radar and camera correspond one to one and are configured in two groups; one group is set at the bottom of the quay crane cab, and the other group is set at the front of the trolley platform.

3. The method for detecting the personnel on board from being smashed as claimed in claim 1, characterized in that: The fusion analysis steps include: Input the video data into a 2D object detector to obtain first detection output data; Input the point cloud data into the 3D object detector to obtain second detection output data; Fusing the first detection output data and the second detection output data to obtain fused data; Perform sparse matrix calculation on the fused data, fill and compress the output data after calculation to obtain the output result of the model; The output result is used as the detection data.

4. The method for detecting the personnel on board from being smashed as claimed in claim 3, characterized in that: The test data includes: One or more combinations of the horizontal position of the sling, the vertical height of the sling, the direction of movement of the sling, the size of the sling, the vertical distance between the sling and the personnel, and whether the personnel are within the vertical projection range of the sling.

5. The method for detecting the personnel on board from being smashed as claimed in claim 1, characterized in that: The emergency plan includes: When the risk is low, the sound and light alarm set in the quay crane cab will sound an alarm; When there is a medium risk, the spreader is controlled to slow down and the sound and light alarm installed in the quay crane cab is controlled to sound an alarm; When the risk is high, the spreader is controlled to stop moving and the sound and light alarm installed in the quay crane cab is controlled to sound an alarm.

6. A shipboard personnel anti-smashing detection device, characterized in that: include: Positioning module, evaluation module and emergency response module; the positioning module integrates and analyzes the point cloud data collected by the 3D laser ranging radar and the video data collected by the camera to obtain detection data; the evaluation module evaluates the risk of personnel being hit based on the detection data; the emergency response module implements the emergency plan according to the risk assessment results; The steps for assessing the risk of personnel being hit are as follows: According to the movement trend of the spreader, determine whether there is a possibility of coordinate overlap between the spreader and the personnel; If not, it is assessed as no risk; If yes, determine the collision time; When the collision time is less than or equal to the preset first time threshold, it is assessed as high risk; When the collision time is greater than the first time threshold and less than or equal to the preset second time threshold, it is assessed as medium risk; When the collision time is greater than the second time threshold and less than or equal to the preset third time threshold, it is assessed as low risk; When the collision time is greater than the third time threshold, it is assessed as no risk; The steps for collision analysis are: Set point is the location of the personnel, and its coordinates are ; Set point is the location of the spreader, point In the The coordinates of the second are ; The spreader position is sampled once per second, and the first time point The coordinates of , the second time point The coordinates of ; Calculate the velocity vector. The velocity vector is calculated through the coordinates of the two sampling points. The calculation formula is as follows: ; Calculate the time required for collision, assuming Continue along the velocity vector Direction of linear motion, prediction At any time in the future The position coordinates are: , ;in, , ; To detect collision, click At some point in the future The distance is zero; that is ; Expand and simplify to get: , which means a The quadratic equation can be solved by the following steps: , expand the equation and put it into standard quadratic form: , using the root-finding formula Solve ;in, ; , ; Determine the feasibility of collision, if , then there is no real solution, indicating that Will not with the point Collision; if , then there is a real number solution, which means that a collision may occur, and the smaller positive root is taken as the collision time; The position of the spreader involved in the collision analysis corresponds to the impact area of ​​the spreader falling; the steps to determine the impact area are as follows: Acquire the operation data of the spreader, the size of the spreader, the vertical height of the spreader and the current wind direction and wind force as analysis data; extract features from the analysis data and construct a state parameter set based on the extracted feature values; determine the influence area map from a pre-configured influence area determination library based on the state parameter set; overlap the positioning area in the influence area map with the vertical projection range to determine the influence area; Among them, the state parameter set includes: parameters representing the wind speed, parameters representing the wind direction, parameters representing the size of the sling, parameters representing the vertical height of the sling, parameters representing the movement direction of the sling, and parameters representing the stress conditions of each sling of the sling; the impact area determination library is constructed in advance based on a large amount of simulation test data through analysis, and the impact area map in the impact area determination library corresponds one to one with the state parameter set; and the impact area map includes: positioning area and impact area; the impact area is a collection of areas where the position may fall during actual falling in the construction data; the positioning area corresponds to the vertical projection range of the sling in the construction data.

7. The shipboard personnel anti-smashing detection device according to claim 6, characterized in that: The 3D laser ranging radar and camera correspond one to one and are configured in two groups; one group is set at the bottom of the quay crane cab, and the other group is set at the front of the trolley platform.

8. The shipboard personnel anti-smashing detection device according to claim 6, characterized in that: The fusion analysis steps include: Input the video data into a 2D object detector to obtain first detection output data; Input the point cloud data into the 3D object detector to obtain second detection output data; Fusing the first detection output data and the second detection output data to obtain fused data; Sparse matrix calculation is performed on the fused data, and the output data after calculation is filled and compressed to obtain the output result of the model.

Images