Method and device for autonomously selecting optimal grabbing point of grab bucket of bridge type ship unloader

Abstract

The invention discloses a bridge type ship unloader grab bucket optimal grabbing point autonomous selection method and device, and belongs to the technical field of ship unloaders, and the method comprises the steps: obtaining multi-frame first laser radar point cloud data and multi-frame second laser radar point cloud data, and obtaining a grab bucket optimal grabbing point of a bridge type ship unloader grab bucket on the basis of the multi-frame first laser radar point cloud data and the multi-frame second laser radar point cloud data; obtaining a cabin global point cloud map; based on the cabin global point cloud map, through hatch identification, material pile point cloud data is obtained; based on the material pile point cloud data, multiple evaluation dimension scores of each candidate grabbing point are determined; based on the multiple evaluation dimension scores of each candidate grabbing point, obtaining a comprehensive score through normalization; and based on the comprehensive score, searching in the grabbing point space through a genetic algorithm to obtain an optimal grabbing point, responding to the optimal grabbing point, generating a control instruction set, and generating the control instruction set for controlling the grab bucket to move to a target position to execute grabbing operation.

Description

TECHNICAL FIELD

[0001] The application discloses an autonomous selection method and device for an optimal grabbing point of a bridge ship unloader grab, and belongs to the technical field of ship unloaders. BACKGROUND

[0002] At present, the core link of the unloading operation of bulk cargo ships, that is, the selection of the grabbing point of the grab, still highly depends on the naked-eye observation of the ship unloader driver and the long-term accumulated operation experience. This is a highly manual operation mode, and the inherent limitations of the mode result in a plurality of technical problems to be solved urgently. The operation safety is poor. The internal environment of the ship cabin is complex, and problems such as visual dead angle, dust diffusion, uneven illumination or insufficient night lighting exist, which seriously interfere with the driver's vision. The relative position and distance of the grab, the bulkhead and the surrounding board of the hatch and the steep cliff of the material pile are unreliable through manual judgment, and serious accidents such as collision and scraping between the grab or the rope and the structure of the ship cabin are prone to occur, which not only causes equipment damage, brings high maintenance cost and downtime, but also poses a major threat to production safety. The comprehensive efficiency is low. The driver cannot accurately and quickly perceive the three-dimensional form distribution of the material pile in the whole ship cabin, and it is difficult to instantaneously locate the real optimal grabbing point. The amount of grabbing (full bucket rate) directly depends on the personal experience and current state of the driver, and the fluctuation is large, lacking stability and optimality. Frequent underloading grabbing (insufficient grabbing amount) will increase the operation cycle and reduce the efficiency. The ship has a risk of overturning. During the unloading process of the bulk cargo ship, maintaining the overall weight balance of the ship is a core safety requirement, and this balance needs to be achieved through a balanced grabbing sequence. If the unilateral load is excessively reduced, the ship will directly cause the roll torque to exceed the standard, and then trigger the risk of overturning. However, manual operation does not have the ability to plan and remember the overall grabbing path, and the driver selects the grabbing point according to subjective feeling, which is easy to cause the unloading amount of the left and right sides and the front and rear sections of the ship cabin to be unbalanced. As the unloading operation proceeds, the inclination angle of the ship will gradually increase, which not only may trigger the forced shutdown warning of the port safety management system, but also may directly cause a major safety accident of ship overturning, which directly and seriously threatens the life and property safety of the ship body, the cargo in the cabin and the on-site operating personnel. SUMMARY

[0003] In view of the defects of the prior art, the application provides an autonomous selection method and device for an optimal grabbing point of a bridge ship unloader grab, which solves the problems of grabbing amount fluctuation and path imbalance caused by the current manual experience dependence, and realizes the intelligentization, high efficiency and safety of the unloading operation.

[0004] The technical scheme of the application is as follows:

[0005] According to a first aspect of an embodiment of the present application, a method for autonomously selecting an optimal grabbing point of a grab bucket of a bridge-type ship unloader is provided, which is applied to a system for autonomously selecting an optimal grabbing point of a grab bucket of a bridge-type ship unloader. The system comprises a first laser radar and a second laser radar, which are respectively arranged on a ship unloader trolley. The method comprises:

[0006] Calibrating coordinate systems of the first laser radar and the second laser radar, performing horizontal correction on the calibrated first laser radar point cloud data and the calibrated second laser radar point cloud data to obtain corrected first laser radar point cloud data and corrected second laser radar point cloud data;

[0007] Obtaining multiple frames of first laser radar point cloud data and multiple frames of second laser radar point cloud data, and obtaining a global point cloud map of a ship cabin based on the multiple frames of first laser radar point cloud data and the multiple frames of second laser radar point cloud data;

[0008] Based on the global point cloud map of the ship cabin, a heap point cloud data is obtained through hatch recognition;

[0009] Based on the heap point cloud data, multiple evaluation dimension scores of each candidate grabbing point are determined, wherein the evaluation dimension scores at least include: a grabbing feasibility score, a safety index score, an efficiency evaluation index score, a full-bucket rate evaluation index score, and a balance evaluation index score;

[0010] Based on the multiple evaluation dimension scores of each candidate grabbing point, a comprehensive score is obtained through a normalized genetic algorithm, and the comprehensive score is obtained through a genetic algorithm based on the evaluation dimension scores;

[0011] Based on the comprehensive score, an optimal grabbing point is obtained through a genetic algorithm in a grabbing point space, a control instruction set is generated in response to the optimal grabbing point, and the control instruction set is used to control the grab bucket to move to a target position to perform a grabbing operation.

[0012] Further, the method further comprises:

[0013] In response to the completion of the grabbing by the grab bucket, after the grab bucket is controlled to be lifted to a safe height, the point cloud data of the corresponding grabbing area in the global three-dimensional point cloud map is deleted according to the target position information of the current grabbing point, the point cloud data of the corresponding grabbing area is scanned and reconstructed in real time, the reconstructed point cloud data is updated to the global three-dimensional point cloud map, and the optimal grabbing point is obtained based on the comprehensive score until a preset operation termination condition is met.

[0014] Further, after the corrected first laser radar point cloud data and the corrected second laser radar point cloud data are obtained, the method further comprises:

[0015] Based on the corrected first laser radar point cloud data, the corrected second laser radar point cloud data and the filtering parameters, the first laser radar point cloud data and the second laser radar point cloud data are obtained through straight-through filtering, and the filtering parameters at least include: the closing time of the grab bucket, the closing width of the grab bucket, the height difference between the laser radar center and the highest point of the grab bucket, and the height of the grab bucket.

[0016] Further, based on the multi-frame first laser radar point cloud data and the multi-frame second laser radar point cloud data, a global point cloud map of the cabin is obtained, including:

[0017] Based on the multi-frame first laser radar point cloud data and the multi-frame second laser radar point cloud data, point cloud splicing is performed through SLAM technology to obtain a global point cloud map of the cabin.

[0018] Further, based on the global point cloud map of the cabin, the material pile point cloud data is obtained through hatch recognition, including:

[0019] Based on the global point cloud map of the cabin, a point cloud region near the hatch is obtained;

[0020] Based on the point cloud region near the hatch, the hatch edge point cloud data is obtained through the AlphaShape algorithm;

[0021] Based on the hatch edge point cloud data, the out-hatch structure point cloud data is obtained through the K-Means clustering algorithm;

[0022] Based on the out-hatch structure point cloud data, the hatch range data is obtained;

[0023] Based on the hatch range data, the material pile point cloud data is obtained.

[0024] Further, based on the material pile point cloud data, multiple evaluation dimension scores of each candidate grabbing point are determined, including:

[0025] Based on the material pile point cloud data, multiple points to be evaluated are determined;

[0026] Based on the multiple points to be evaluated, multiple evaluation dimensions are evaluated respectively to obtain multiple evaluation dimension scores of each candidate grabbing point, and the evaluation dimensions include: grabbing feasibility, safety index, efficiency evaluation index, full-bucket rate evaluation index and balance evaluation index.

[0027] Further, the grabbing feasibility is used to judge whether the grab bucket will collide with the hatch when the grabbing operation is performed at the candidate grabbing point (x, y);

[0028] And / or;

[0029] The safety index is used to quantify the safety margin between the candidate grabbing point and the hatch edge;

[0030] And / or;

[0031] The efficiency evaluation index is used to evaluate the operation cost required for moving from the previous grabbing point to the current candidate point, and the efficiency is quantified by normalizing the weighted Manhattan distance;

[0032] And / or;

[0033] The full bucket rate evaluation index is used to analyze the height relationship between the candidate point and the eight surrounding neighborhood grids;

[0034] And / or;

[0035] The balance evaluation index is used to select a position symmetrical to the historical grabbing point, and the comprehensive score of each candidate point.

[0036] According to a second aspect of the embodiment of the present application, a bridge ship unloader grab optimal grabbing point autonomous selection device is provided, the device comprises:

[0037] The correction module is configured to calibrate the coordinate systems of the first laser radar and the second laser radar, and perform horizontal correction on the calibrated first laser radar point cloud data and the calibrated second laser radar point cloud data to obtain the corrected first laser radar point cloud data and the corrected second laser radar point cloud data.

[0038] The acquisition module is configured to acquire a plurality of frames of first laser radar point cloud data and a plurality of frames of second laser radar point cloud data, and obtain a global point cloud map of the ship cabin based on the plurality of frames of first laser radar point cloud data and the plurality of frames of second laser radar point cloud data.

[0039] The recognition module is configured to obtain the stockpile point cloud data through hatch recognition based on the global point cloud map of the ship cabin.

[0040] The evaluation module is configured to determine a plurality of evaluation dimension scores of each candidate grabbing point based on the stockpile point cloud data, wherein the evaluation dimension scores at least include a grabbing feasibility score, a safety index score, an efficiency evaluation index score, a full bucket rate evaluation index score, and a balance evaluation index score.

[0041] The scoring module is configured to obtain a comprehensive score based on the plurality of evaluation dimension scores of each candidate grabbing point through a normalized genetic algorithm, and obtain the comprehensive score based on the evaluation dimension scores through a genetic algorithm.

[0042] The generation module is configured to search for an optimal grabbing point in the grabbing point space based on the comprehensive score through a genetic algorithm, generate a control instruction set, and control the grab bucket to move to a target position to perform a grabbing operation.

[0043] According to a third aspect of the embodiment of the present application, a terminal is provided, comprising:

[0044] One or more processors;

[0045] a memory for storing the one or more processor-executable instructions;

[0046] wherein the one or more processors are configured to:

[0047] perform the method of the first aspect of the embodiments of the application.

[0048] According to a fourth aspect of the embodiments of the application, there is provided a non-transitory computer-readable storage medium, which, when instructions in the storage medium are executed by a processor of a terminal, enables the terminal to perform the method of the first aspect of the embodiments of the application.

[0049] According to a fifth aspect of the embodiments of the application, there is provided an application program product, which, when the application program product is running on a terminal, enables the terminal to perform the method of the first aspect of the embodiments of the application.

[0050] The application has the following beneficial effects:

[0051] The application provides a bridge ship unloader grab optimal grabbing point autonomous selection method and device, precise three-dimensional perception of a ship cabin and a material pile is realized through double laser radars, visual limitations caused by environmental interference are eliminated, collision risks of a grab and a ship cabin structure are greatly reduced, and operation safety is improved; combination of multi-dimensional evaluation and a genetic algorithm realizes scientific selection of an optimal grabbing point, improves stability of a full-bucket rate, reduces operation cycle times, and improves comprehensive efficiency; introduction of a balanced evaluation index ensures that a ship load is balanced during unloading, and overturning risks are effectively avoided; meanwhile, autonomous selection of a grabbing point and automatic control of a grab are realized, manual dependence is reduced, and stability and intelligent level of operation are improved.

[0052] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the application. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 is a flowchart of a bridge ship unloader grab optimal grabbing point autonomous selection method according to an exemplary embodiment.

[0054] Figure 2 is a schematic diagram of laser radar installation positions and layouts in a bridge ship unloader grab optimal grabbing point autonomous selection system according to an exemplary embodiment.

[0055] Figure 3 is a flowchart of a bridge ship unloader grab optimal grabbing point autonomous selection method according to an exemplary embodiment.

[0056] Figure 4It is a flow chart of applying a genetic algorithm to select an optimal grabbing point in a bridge ship unloader grab optimal grabbing point autonomous selection method according to an exemplary embodiment.

[0057] Figure 5 It is a scanning path schematic diagram of constructing a global three-dimensional point cloud map in a bridge ship unloader grab optimal grabbing point autonomous selection method according to an exemplary embodiment.

[0058] Figure 6 It is a structural schematic block diagram of a bridge ship unloader grab optimal grabbing point autonomous selection device according to an exemplary embodiment.

[0059] Figure 7 It is a terminal structural schematic block diagram according to an exemplary embodiment. DETAILED DESCRIPTION

[0060] The technical solutions of the present application will be described clearly and completely below with reference to the drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0061] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application.

[0062] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connecting" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0063] The embodiment of the present application provides a bridge ship unloader grab optimal grabbing point autonomous selection method, which is realized by a terminal. The terminal can be a desktop computer or a notebook computer, etc. The terminal at least includes a CPU, etc.

[0064] Embodiment one

[0065] Figure 1is a flow chart of a bridge ship unloader grab optimal grabbing point autonomous selection method according to an exemplary embodiment, the method is applied to a bridge ship unloader grab optimal grabbing point autonomous selection system, the bridge ship unloader grab optimal grabbing point autonomous selection system comprises a first laser radar and a second laser radar, the first laser radar and the second laser radar are arranged on a ship unloader trolley respectively, and the method comprises:

[0066] Step S10, calibrate the coordinate system of the first laser radar and the second laser radar, and perform horizontal correction on the calibrated first laser radar point cloud data and the calibrated second laser radar point cloud data to obtain corrected first laser radar point cloud data and corrected second laser radar point cloud data;

[0067] Step S20, obtain multiple frames of first laser radar point cloud data and multiple frames of second laser radar point cloud data, and obtain a ship cabin global point cloud map based on the multiple frames of first laser radar point cloud data and the multiple frames of second laser radar point cloud data;

[0068] Step S30, based on the ship cabin global point cloud map, the material pile point cloud data is obtained through hatch recognition;

[0069] Step S40, based on the material pile point cloud data, determine the multiple evaluation dimension scores of each candidate grabbing point, wherein the evaluation dimension scores at least include: grabbing feasibility score, safety index score, efficiency evaluation index score, full bucket rate evaluation index score and balance evaluation index score;

[0070] Step S50, based on the multiple evaluation dimension scores of each candidate grabbing point, the comprehensive score is obtained through the normalized genetic algorithm, and the comprehensive score is obtained through the genetic algorithm based on the evaluation dimension scores;

[0071] Step S60, based on the comprehensive score, the optimal grabbing point is obtained by searching in the grabbing point space through the genetic algorithm, a control instruction set is generated, and the control instruction set is used to control the grab bucket to move to the target position to perform the grabbing operation.

[0072] Further, the method further comprises:

[0073] In response to the completion of the grabbing of the grab bucket, after the grab bucket is lifted to a safe height, the point cloud data of the corresponding grabbing area in the global three-dimensional point cloud map is deleted according to the target position information of the present grabbing point, the point cloud data of the corresponding grabbing area is scanned and reconstructed in real time, the reconstructed point cloud data is updated to the global three-dimensional point cloud map, and the optimal grabbing point is obtained based on the comprehensive score until the preset operation termination condition is met.

[0074] Further, after obtaining the corrected first laser radar point cloud data and the corrected second laser radar point cloud data, the method further comprises:

[0075] Based on the corrected first laser radar point cloud data, the corrected second laser radar point cloud data and the filtering parameters, the first laser radar point cloud data and the second laser radar point cloud data are obtained through straight-through filtering, and the filtering parameters at least include: the closing time of the grab bucket, the closing width of the grab bucket, the height difference between the laser radar center and the highest point of the grab bucket, and the height of the grab bucket.

[0076] Further, based on the multi-frame first laser radar point cloud data and the multi-frame second laser radar point cloud data, a global point cloud map of the cabin is obtained, including:

[0077] Based on the multi-frame first laser radar point cloud data and the multi-frame second laser radar point cloud data, point cloud splicing is performed through SLAM technology to obtain a global point cloud map of the cabin.

[0078] Further, based on the global point cloud map of the cabin, the material pile point cloud data is obtained through hatch recognition, including:

[0079] Based on the global point cloud map of the cabin, a point cloud region near the hatch is obtained;

[0080] Based on the point cloud region near the hatch, the hatch edge point cloud data is obtained through the AlphaShape algorithm;

[0081] Based on the hatch edge point cloud data, the out-hatch structure point cloud data is obtained through the K-Means clustering algorithm;

[0082] Based on the out-hatch structure point cloud data, the hatch range data is obtained;

[0083] Based on the hatch range data, the material pile point cloud data is obtained.

[0084] Further, based on the material pile point cloud data, a plurality of evaluation dimension scores are calculated for each candidate grabbing point, including:

[0085] Based on the material pile point cloud data, a plurality of points to be evaluated are determined;

[0086] Based on the plurality of points to be evaluated, a plurality of evaluation dimensions are evaluated respectively to obtain a plurality of evaluation dimension scores of each candidate grabbing point, and the evaluation dimensions include: grabbing feasibility, safety index, efficiency evaluation index, full-bucket rate evaluation index and balance evaluation index.

[0087] Further, the grabbing feasibility is used to judge whether the grab bucket will collide with the hatch when the grabbing operation is performed at the candidate grabbing point (x, y);

[0088] And / or;

[0089] The safety index is used to quantify the safety margin between the candidate grabbing point and the hatch edge.

[0090] and / or;

[0091] Efficiency evaluation index is used to evaluate the operation cost required to move from the last grabbing point to the current candidate point, and the efficiency is quantified by normalizing the weighted Manhattan distance;

[0092] and / or;

[0093] Full bucket rate evaluation index is used to analyze the height relationship between the candidate point and the eight surrounding neighborhood grids;

[0094] and / or;

[0095] Balance evaluation index is used to select the position symmetrical to the historical grabbing point, and the comprehensive score of each candidate point is calculated.

[0096] Embodiment two

[0097] Figure 3 is a flow chart of a bridge ship unloader grab optimal grabbing point autonomous selection method according to an exemplary embodiment, the method comprising:

[0098] In step S201, two semi-solid laser radars are fixed on the ship unloader trolley, a point cloud registration algorithm based on feature matching is used to align the coordinate system of one radar to the coordinate system of the other radar; the unified point cloud data is horizontally corrected to ensure that the data coordinate system is consistent with the actual working scene; at the same time, the point cloud interference of the grab itself is filtered out (to avoid the influence of the grab structure on the material pile point cloud recognition), and after completion, it enters step S202.

[0099] In step S201, the specific process of laser radar calibration and point cloud preprocessing is as follows:

[0100] The specific installation position of the laser radar on the ship unloader is shown in the attached Figure 2 The selected semi-solid laser radar has the characteristics of fixed field of view angle. Compared with the traditional 360° panoramic laser radar, this arrangement scheme has obvious advantages: through the design of directional field of view angle, the scanning range of the laser radar can be strictly limited, ensuring that the collected point cloud data comes completely from the internal area of the ship cabin, avoiding the mixing of point cloud information from irrelevant areas such as the wharf ground, surrounding equipment, and sky. This design not only improves the purity and effectiveness of the point cloud data, but also directly reduces the data processing amount and reduces the system computing load.

[0101] The purpose of the calibration of the laser radar is to unify the point cloud data in different coordinate systems to the same coordinate system. Take the laser radar 2 as the main radar and the laser radar 1 as the auxiliary radar. The specific process is as follows: first, the point cloud data of the two laser radars is synchronously collected in the same static scene. In order to ensure the registration effect, the scene should contain rich, clear and easily identifiable features. 5-10 frames of point cloud are continuously collected for each radar respectively for accumulation, so as to enhance the density and stability of the point cloud.

[0102] Subsequently, a certain common fixed feature region is manually cropped from the two groups of point cloud data for subsequent registration operation. The registration process is divided into two stages of coarse registration and fine registration:

[0103] Coarse registration: the sampling consistency initial alignment algorithm (SAC-IA) is used to calculate the initial transformation matrix. This method is based on the FPFH (Fast Point Feature Histograms) feature descriptor, combined with the RANSAC strategy to find the optimal corresponding relationship, and solve the roughly aligned rigid transformation. Let the source point cloud be P and the target point cloud be Q, and the algorithm steps are as follows:

[0104] (1) Randomly sample s points in P, and ensure that the minimum distance between them is greater than the set threshold d min

[0105] (2) For each sampling point, determine a group of candidate corresponding points in Q through feature similarity search, and select one as the matching point;

[0106] (3) Calculate the transformation T based on the current matching point pair, and calculate the error score using the following robust loss function, and finally select the transformation T with the minimum error coarse As the result of coarse registration:

[0107]

[0108] Where e i represents the error of the ith matching point pair after the current transformation, t e is the error threshold. This function suppresses the influence of false matching by imposing a linear penalty on larger errors, and improves the robustness of registration.

[0109] Fine registration: fine registration is performed on the basis of the initial alignment provided by coarse registration, and the iterative closest point algorithm (ICP) is used to realize high-precision optimization. ICP finds the closest corresponding points iteratively and continuously minimizes the error between the corresponding points to solve the optimal transformation matrix. Its objective function is to minimize the mean square error of the current corresponding point pair:

[0110]

[0111] Where: p iLet p be a point in source point cloud P; x i Let p be a point in source point cloud P; x i The corresponding nearest point in target point cloud Q;

[0112] R is a rotation matrix, t is a translation vector; N p is the number of valid matching points;

[0113] The algorithm iteratively performs the "find the nearest point-compute the transformation-apply the transformation" process until convergence, and finally obtains a high-precision transformation matrix T fine =(R, t),

[0114] Let p be a point in source point cloud P; x T The calculation formula of the aligned point p' after two-stage transformation is:

[0115]

[0116] Thus, the point cloud of one laser radar is accurately registered to a certain laser radar coordinate system.

[0117] The laser radar will cause deviation between the point cloud data and the real world coordinate system due to the reasons of inclined installation and orientation. In order to perform horizontal correction, the following steps can be used to realize the horizontal alignment of point cloud:

[0118] (1) Data acquisition: vertically hoist the grab bucket and laser radar system which have completed preliminary spatial registration. After stabilization, continuously collect 5-10 frames of point cloud data for accumulation, and manually intercept the clear ground area point cloud;

[0119] (2) Ground fitting: use RANSAC algorithm to fit the ground point cloud, extract the mathematical model of the ground plane, and calculate the normal vector n=(n x , n y , n z ) of the plane;

[0120] (3) Point cloud rotation correction: according to the ground plane normal vector n and the unit vector z=(0, 0, 1) of the world coordinate system z axis, calculate the expected rotation angle θ and the rotation axis r:

[0121]

[0122] r=n×z (5)

[0123] Where the rotation axis r is perpendicular to the plane formed by n and z. The point cloud direction correction is to rotate the original ground normal vector n by θ angle around the axis r, so that it is aligned with the z axis. The corresponding rotation matrix R is given by the Rodrigues rotation formula:

[0124]

[0125] where [r] × is the skew matrix of vector r.

[0126] A unified coordinate system is established with the center of the laser radar 2 as the origin O, and the X axis is along the length direction of the grab bucket, and the Y axis is along the width direction of the grab bucket. In order to eliminate the interference of the point cloud of the grab bucket itself on the global map, according to the length L grab of the closed grab bucket, the width W grab of the closed grab bucket, the height difference ΔH between the center of the laser radar and the highest point of the grab bucket (when located at the vertex), and the height H grab of the grab bucket, the straight-through filtering is used to remove the point cloud that meets the following conditions at the same time:

[0127]

[0128] Where Δx is the compensation margin in the X axis direction (used to cover the size deviation caused by the non-coincidence of the center of the laser radar and the center of the grab bucket in the X axis direction, and to ensure that there is no omission filtering of the point cloud in the length direction of the grab bucket); and Δy is the compensation margin in the Y axis direction (the action is the same as Δx, which is used to cover the size deviation caused by the non-coincidence of the center of the laser radar and the center of the grab bucket in the Y axis direction, and to take into account the slight attitude deviation in the width direction of the grab bucket, and to ensure that there is no omission filtering of the point cloud in the direction).

[0129] This operation can accurately filter out the point cloud in the range of the grab bucket structure, retain the effective data of the cargo pile and the environment, and improve the quality of the map.

[0130] In step S202, the control unloader cart travels along the length direction of the ship for one berth, and the control trolley travels back and forth in the width direction of the berth multiple times. In this process, the two radars continuously scan, combine the real-time collected point cloud data with the built-in IMU of the laser radar, use the SLAM technology to splice the point cloud, construct a complete global three-dimensional point cloud map covering the entire ship hatch and internal cargo pile, and enter step S203 after completion.

[0131] In step S202, the specific process of constructing the global three-dimensional point cloud map is as follows:

[0132] In order to construct the global three-dimensional point cloud map, as shown in Figure 5 , an "arch" type strategy is used to scan the ship hatch, and the specific implementation steps are as follows:

[0133] (1) Initial state confirmation: the cart has been accurately positioned at the start end of the berth (coordinate X0); the trolley has been moved in advance and parked at the near-shore side of the berth (coordinate Y0). The near-shore side limit position is usually determined by the physical limit of the unloader or the actual width of the hatch, and it is necessary to ensure that the laser radar beam can completely cover the near-shore side hatch or the effective edge of the cargo pile through pre-calibration, so as to avoid the scanning blind area.

[0134] (2) Scan parameter calculation: According to the installation height, pitch angle of the laser radar and the maximum estimated slope of the material pile, the effective scanning width W of the radar on the surface of the material pile is calculated. According to the required overlap rate O (usually set to 20% ~ 30%) for point cloud splicing, the stepping distance between each two scans of the cart is calculated:

[0135] ΔX = W · (1 - O) (8)

[0136] At the same time, according to the radar scanning frequency, the uniform running speed v of the trolley is set y , to ensure that the density of the point cloud in the motion direction meets the modeling requirements.

[0137] (3) First scan: keep the cart at X0 unchanged, control the trolley to move at a speed v y from the near-shore side (Y0) to the far-shore side (Y max ). Two radars continuously collect point cloud data, record the IMU attitude data built-in the main radar, use the FAST-LIO2 algorithm of tight coupling iterative Kalman filter to build the point cloud map, and generate the first point cloud strip covering the full width of the hatch.

[0138] (4) Cart stepping and iterative scanning: after completing each longitudinal scan, control the cart to step ΔX to the next position (X1 = X0 + ΔX) along the length direction of the ship. Then the trolley runs in reverse (if the last scan is from top to bottom, then this scan is from bottom to top), and the next scan is performed. Repeat this process to form a "bow" shaped scanning path until the entire hatch length is covered.

[0139] (5) Edge and dead angle supplementary scanning:

[0140] After completing the main "bow" shaped scanning, for the four corners of the hatch, the connection between the hatch walls and other areas prone to be missed, according to the preliminary generated point cloud map, the blind area is automatically identified or the specific supplementary scanning path is manually planned, and local fine scanning is performed to ensure that the global three-dimensional point cloud map is fully covered and has no blind area, meeting the requirements of subsequent grabbing planning.

[0141] Using "bow" shaped scanning can greatly reduce the displacement of heavy carts, and more continuous scanning work is completed by more flexible trolleys, thereby significantly improving the overall scanning efficiency, reducing energy consumption and equipment wear and tear, while ensuring the continuity and splicing accuracy of the point cloud data, and finally obtaining a global three-dimensional point cloud P global .

[0142] Step S203, according to the Z-axis height information, the point cloud area near the hatch is intercepted, the Alpha-Shape algorithm is used to extract the hatch edge point cloud, and the K-Means clustering algorithm is used to separate the hatch structure. The hatch point cloud is fitted, the hatch range is calculated, and then the effective point cloud data of the material pile is intercepted according to the range, and step S204 is entered.

[0143] In step S203, the specific process of the stockpile point cloud interception is as follows:

[0144] When the grab bucket performs the stockpile grabbing operation, its working range is limited to the ship cabin hatch area and cannot reach the part outside the hatch. Therefore, only the stockpile point cloud in the hatch range belongs to the actual operable effective data, and the point cloud outside the hatch needs to be removed. Based on this, the extraction method of the stockpile region of interest (ROI) point cloud is as follows:

[0145] (1) Data interception: First, separate the point cloud subset containing the deck and hatch from the global point cloud. Operation: use the pass-through filter (PassThroughFilter) to filter along the Z axis.

[0146] Keep the point P i (x i , y i , z i ) that satisfies the following conditions:

[0147] z max -Δ≤z i ≤z max (9)

[0148] Where z max is the maximum value of the point cloud on the Z axis, representing the approximate height of the deck plane. Δ is a tolerance value used to ensure that the intercepted point cloud can completely contain the hatch upright wall (usually set according to the hatch coaming height).

[0149] Output point cloud set P deck ={P i ∈P global |z max -Δ≤z i ≤z max}.

[0150] (2) Boundary extraction: use the Alpha-Shape algorithm to extract the internal and external boundary points from P deck . This algorithm can be understood as rolling a "circle roll" with a radius of α around the point set, and all points that can be reached by this circle roll form the boundary.

[0151] For any two points P deck and P i in the point set P j , if there is a circle (sphere in 3D) with a radius of α that passes through P i and P j , and does not contain any other points in P deck , then the line segment is a boundary edge, and P i and Pj is the boundary point.

[0152] The selection of parameter a is crucial. When a tends to 0, every point becomes a boundary point; when a tends to infinity, the algorithm degenerates to the computation of the convex hull of the point set. Here, 3 times of the average distance of the point cloud is chosen.

[0153] Finally, the boundary point cloud P boundary is obtained.

[0154] (3) Boundary clustering: the extracted boundary P boundary contains both the outer boundary and the inner boundary of the hatch. K-Means clustering algorithm is used to separate them. The algorithm only uses the two-dimensional coordinates (x, y) of the points as features to ignore the fluctuation interference in the height direction. The clustering process first randomly initializes two cluster centers and Then the following two steps are iteratively performed until the change of the cluster centers converges to a set threshold: in the assignment step, each point is assigned to the nearest cluster center, forming two temporary point sets; in the update step, the mean of each point set is recalculated as the new cluster center. After the iteration, the two point sets P outer = S1 and P inner = S2 are output. Finally, by judging the number of point clouds in each cluster, the inner boundary point cloud P hatch of the hatch can be automatically identified.

[0155] (4) Interest region point cloud extraction: after obtaining the inner boundary point cloud P hatch of the hatch, first calculate the minimum and maximum values of the point cloud in the horizontal and vertical directions (X and Y axes), to determine an initial rectangular region that can completely enclose the inner boundary of the hatch. The boundary of the rectangular region can be represented as:

[0156] x min = min(x i ), x max = max(x i ), y min = min(y i ), y max = max(y i ) (10)

[0157] where (x i , y i ) are the plane coordinates of the points in P hatch .

[0158] Then, within the spatial range of this rectangular region, all point clouds in the original global point cloud are extracted, i.e. all point cloud data inside the hatch, which can be mathematically expressed as:

[0159] P roi = {P i ∈ P global | x min ≤ x i ≤ x max , y min ≤ y max} (11)

[0160] So far, P roi is the internal hatch pile point cloud of interest, which can be used for subsequent grabbing planning and analysis.

[0161] Step S204, a multi-objective evaluation system of grabbing points is constructed, including the following indexes: grabbing feasibility: this index is a binary discriminant item, which is used to judge whether the grabbing point is in the position that can be reached by the ship unloader mechanical structure and has no collision; safety index: calculate the Euclidean distance from the candidate grabbing point to the nearest hatch edge; efficiency evaluation index: calculate the walking distance of the large and small cars of the ship unloader from the last grabbing point to the current candidate point; full bucket rate evaluation index: calculate the height difference between the average height of the point cloud in the surrounding area and the average height of the adjacent area with the candidate point as the center; balance evaluation index: calculate the symmetry degree of the current candidate point with the set according to the historical grabbing point position set. The comprehensive score of each candidate point is calculated by weighted summation, and then step S205 is entered.

[0162] In step S204, the specific process of the multi-objective evaluation system of the optimal grabbing point is as follows:

[0163] As Figure 4 shown, the selection of the optimal grabbing point is based on a comprehensive multi-objective optimization evaluation system, which deeply integrates perception data, dynamic constraints and operation process requirements, aiming to overall plan the efficiency, safety and stability of the grabbing operation, so as to realize the overall performance optimization of the ship unloader automatic operation. The specific calculation and decision process is as follows:

[0164] Firstly, the system selects each evaluation point from the candidate grabbing point set extracted from the preprocessed pile point cloud, and calculates the following five core evaluation indexes in parallel for each point:

[0165] (1) grabbing feasibility S feas : this index is a binary discriminant item, which is used to judge whether the grabber will collide with the hatch when the grabbing operation is carried out at the candidate grabbing point (x, y). According to the geometric parameters of the grabber (closed length L grad , closed width W grab ), the coordinates of the four corner points of the grabber in the grabbing point coordinate system can be calculated when the grabber is opened.

[0166] Assuming that the center of the grabber is aligned with the grabbing point p c (x c, y c ), and the length direction of the grab bucket is parallel to the X-axis, and the width direction is parallel to the Y-axis, then the coordinates of the four corner points are:

[0167]

[0168] Check whether all four corner points of the grab bucket are located within the hatch boundary range. Only when all the corner points are located inside the hatch, is the grabbing considered feasible, without collision risk.

[0169] The mathematical expression is:

[0170]

[0171] Only when S feas = 1, the candidate point enters the subsequent comprehensive evaluation.

[0172] (2) Safety index: This index is used to quantify the safety margin between the candidate grabbing point and the hatch edge. It is evaluated by calculating the shortest Euclidean distance from the candidate grabbing point p c (x c , y c ) to the hatch boundary fitted by the point cloud. The farther the distance, the lower the risk of accidental collision of the grab bucket with the hatch wall during operation, and the higher the safety margin.

[0173] The mathematical expression is:

[0174] D safety = min(x c -x min , x max -x c , y c -y min , y max -y c ) (17)

[0175] The larger the D safety value, the higher the operation safety margin, and the lower the risk of accidental collision.

[0176] (3) Efficiency evaluation index: It evaluates the operation cost required to move from the last grabbing point to the current candidate point. The efficiency is quantified by normalized weighted Manhattan distance, aiming to significantly reduce the moving weight of the car, minimize the empty running time, and improve the operation efficiency. The specific calculation method is as follows:

[0177] Record the last grabbing point coordinates p p (x p , y p ), and the current candidate grabbing point coordinates p c (x c , y c), the distance the grab needs to move from the last grab point p p to the current candidate point p c The distance to move:

[0178] Δx = |x c -x p | (18)

[0179] Δy = |y c -y p | (19)

[0180] where Δx is the relative movement distance of the large car, and Δy is the relative movement distance of the small car.

[0181] Full stroke normalization is performed on the movement distance:

[0182] Normalized large car movement distance: Δx norm = Δx / X range ;

[0183] When Δx norm = 0, it means the large car does not move, and when Δx norm = 1, it means the entire cabin length is moved.

[0184] Normalized large car movement distance: Δy norm = Δy / Y range ;

[0185] When Δy norm = 0, it means the small car does not move, and when Δy norm = 1, it means the entire cabin width is moved.

[0186] where:

[0187] X range = X max -X min , the full stroke range of the large car (X-axis);

[0188] Y range = Y max -Y min , the full stroke range of the large car (Y-axis);

[0189] Calculate the weighted comprehensive efficiency cost:

[0190] C travel = w x · Δx norm + w y · Δy norm (20)

[0191] Generally, w x is greater than w y , aiming to reduce the movement of the large car.

[0192] Convert the cost to the efficiency score:

[0193] S efficiency = 1-C travel (21)

[0194] Finally, the calculation formula of the efficiency evaluation index S efficiency is:

[0195]

[0196] (4) Full bucket rate evaluation index: In order to more finely assess whether the candidate point is a local convex (material peak). By analyzing the height relationship of the candidate point and the eight surrounding neighborhood grids, the convexity degree is comprehensively judged, so as to estimate the full bucket rate. The specific implementation method is as follows:

[0197] Taking the current candidate grabbing point coordinate p c (x c ,y c ) as the center, a 3x3 grid is divided, and each grid is a square with a side length of L grab Each grid is numbered, and the central grid (1,1) is the core grid.

[0198] For each grid (i,j), extract all point clouds falling within its boundary. Calculate the average value of all point Z coordinates in the grid as the height H(i,j) of the grid:

[0199] H(i,j) = mean(z k for all points k in grid(i,j)) (22)

[0200] If there is no point cloud data in a certain grid, a very low height value is assigned, which is easy to identify in subsequent calculation.

[0201] Calculate the difference between the core grid H(1,1) and the grid with the lowest height among the surrounding eight neighborhood grids.

[0202] H fulfill = H(1,1) - min(H(i,j) for all(i,j)≠(1,1)) (23)

[0203] Ensure that the grabbing point is convex in any direction, which can effectively avoid grabbing points on the "slope".

[0204] (5) Balance evaluation index: Avoid continuous grabbing on one side, which may cause the ship body to tilt. The position symmetric to the historical grabbing point should be actively selected for operation. The specific calculation method is as follows:

[0205] Calculate the center of the historical grabbing point distribution:

[0206] Center hist = (x avg , y avg ) = (mean(x i for all p i in H), mean(y i for all p i in H)) (24)

[0208] where:

[0209] H is a set of historical grab points.

[0210] Get the physical hatch center:

[0211] Center hatch = (x center , y center ) = ((x min +x max ) / 2, (y min +y max ) / 2) (25)

[0212] Calculate the symmetric point of the distribution center about the physical hatch center:

[0213] Center target = (x target , y target ) = (2*x center -x avg , 2*y center -y avg ) (26)

[0214] Calculate the Euclidean distance of the current candidate point to the symmetric point of the distribution center about the physical hatch center:

[0215] D balance = sqrt((x c -r target ) 2 +(y c -y target ) 2 ) (27)

[0216] The smaller the distance D balance , the better, use the Gaussian function to map it to the larger the better:

[0217] S balance = exp(-(D balance 2 ) / (2*σ 2 )) (28)

[0218] Let the set of effective candidate grabbing points after feasibility filtering be: C = {c1, c2,..., c N}, for any candidate point c k , the original values of its four core evaluation indexes have been calculated: D safety , S efficiency , H fulfill and S balance .

[0219] To eliminate dimensional differences, the original values of each index need to be mapped to the dimensionless [0, 1] interval.

[0220] First, calculate the maximum and minimum values of all candidate points on each index as the normalization reference:

[0221]

[0222]

[0223] For each candidate point c k , linearly normalize it according to the relative position of its index value to the global extreme value:

[0224]

[0225] Assign a preset weight to each normalized index to calculate the comprehensive score of each candidate point. The weight coefficients need to satisfy:

[0226] w safety +w efficiency +w fulfill +w balance =1 (41)

[0227]

[0228] Step S205, apply genetic algorithm to search for the global optimal grabbing point with the highest comprehensive score in the effective point cloud of the material pile using the comprehensive score obtained in step 4) as the objective function; after determining the optimal grabbing point, output control instructions to drive the grab bucket to move to the position to perform the grabbing operation, and after the grabbing is completed, enter step S206.

[0229] In step S205, the specific process of using genetic algorithm to find the optimal grabbing point is as follows:

[0230] Genetic algorithm is a meta-heuristic algorithm that simulates the natural selection process, which is very suitable for finding global optimal or near-optimal solution in complex and high-dimensional search space (such as the entire stockpile point cloud). The core idea is "survival of the fittest", which simulates selection, crossover (hybridization), mutation and other operations to make the population evolve in a better direction. The steps of using genetic algorithm to find the optimal grabbing point are as follows (1) problem definition and coding:

[0231] Decision variable: horizontal coordinate of grabbing point

[0232] X = (x, y), x ∈ [x min , x max ], y ∈ [y min , y max ] (43)

[0233] Search space:

[0234] Ω = {(x, y) | x min ≤ x ≤ x max , y min ≤ y ≤ y max} (44)

[0235] Chromosome coding:

[0236]

[0237] (2) Initialize population:

[0238] Initial population (size N):

[0239] P (0 ) = {c1, c2,... c N} (46)

[0240] Individual generation method:

[0241] x i = x min + rand() · (x max - x min ) (47)

[0242] y i = y min + rand() · (y max - y min ) (48)

[0243] Where rand() ~ U(0, 1).

[0244] (3) Fitness function:

[0245]

[0246] Weighted composite scoring function:

[0247]

[0248] Weight constraints:

[0249] w safety +w efficiency +w fulfill +w balance =1 (51)

[0250] (4) Select operation:

[0251] Individual c i Probability of being selected:

[0252]

[0253] in:

[0254] F min =min j＝1,...，N F(c i (53)(5) Cross operation:

[0255] Parent generation: c p =[x p y p ] T c q =[x q y q ] T

[0256] Offspring generation:

[0257] First, calculate the crossover coefficient β:

[0258]

[0259] Among them u~U(0,1), η c This is the cross-distribution index.

[0260] Offspring coordinates:

[0261] x o =0.5[(1+β)x p +(1-β)x q (55)

[0262] x r =0.5[(1-β)x p +(1+β)x q (56)

[0263] y o =0.5[(1+β)yp + (1 - β)y q ] (57)

[0264] y r = 0.5[(1 - β)y p + (1 + β)y q ] (58)

[0265] (6) Mutation operation:

[0266] Mutation amount of gene:

[0267]

[0268] where r k ~ U(0, 1), η m is the crossover distribution index.

[0269] Mutation result:

[0270] x' = x + δ x · (x max - x min ), y' = y + δ y · (y max - y min ) (60)

[0271] Boundary constraint:

[0272] x' = min(x max , max(x min , x')), y' = min(y max , max(y min , y')) (61)

[0273] (7) Elite preservation strategy:

[0274] Preserve the N e highest fitness individuals in the current population, and directly enter the next generation:

[0275]

[0276] (8) New population formation:

[0277] The new population is composed of two parts:

[0278] P (t+1) = ε (t) ∪ Q (t) (63)

[0279] Where: ε (t) : elite individuals (N e ); Q (t): New individuals (N-N e ) are generated by selection-crossover-variation;

[0280] (9) Termination condition:

[0281] The maximum number of iterations T max is reached;

[0282] Step S206, after the grab bucket completes grabbing and is lifted to a safe height, the point cloud data of the grabbing area is deleted from the global three-dimensional point cloud map according to the position information of the grabbing point this time; in the process of lifting the grab bucket, the deleted area is scanned and the point cloud is reconstructed in real time by using the laser radar combined with the SLAM technology, and the reconstructed point cloud data is updated to the global map, so that the dynamic refreshing of the global point cloud map is realized, and after completion, step S207 is entered.

[0283] In step S206, the specific process of local point cloud updating is as follows:

[0284] After the grab bucket completes grabbing and is lifted to a predetermined height, the system first deletes the point cloud of the corresponding area from the global map according to the position information of the grabbing point this time. Then, when the grab bucket stops rising, the real-time height difference between the point cloud center and the highest point of the grab bucket is calculated to accurately remove the point cloud interference of the grab bucket itself. Finally, the system calls the SLAM module to scan and reconstruct the map of the above-mentioned deleted area in real time, thereby realizing efficient and dynamic updating of the global map.

[0285] Step S207, steps S205 and S206 are executed in a loop, and the historical grabbing point positions of each grabbing are recorded in real time; until the preset termination condition of the work (such as the completion of unloading the material pile in the cabin, reaching the preset work time, etc.) is met, the work process is stopped.

[0286] Embodiment Two

[0287] Figure 6 An optimal grabbing point autonomous selection device for a bridge ship unloader grab bucket is shown according to an example embodiment, and the device comprises:

[0288] A correction module 310 is configured to calibrate coordinate systems of the first laser radar and the second laser radar, perform horizontal correction on the calibrated first laser radar point cloud data and the calibrated second laser radar point cloud data, and obtain corrected first laser radar point cloud data and corrected second laser radar point cloud data;

[0289] An acquisition module 320 is configured to acquire a plurality of frames of first laser radar point cloud data and a plurality of frames of second laser radar point cloud data, and obtain a global point cloud map of a ship cabin based on the plurality of frames of first laser radar point cloud data and the plurality of frames of second laser radar point cloud data;

[0290] The recognition module 330 is configured to obtain the stockpile point cloud data by hatch recognition based on the global point cloud map of the ship cabin.

[0291] The evaluation module 340 is configured to determine evaluation dimension scores based on the stockpile point cloud data, wherein the evaluation dimension scores at least include: a feasibility score of grabbing, a safety index score, an efficiency evaluation index score, a full bucket rate evaluation index score, and a balance evaluation index score.

[0292] The scoring module 350 is configured to obtain a comprehensive score by a genetic algorithm based on the evaluation dimension scores.

[0293] The generation module 360 is configured to obtain an optimal grabbing point based on the comprehensive score, and generate a control instruction set in response to the optimal grabbing point, wherein the control instruction set is used to control the grab bucket to move to a target position to perform a grabbing operation.

[0294] Embodiment three

[0295] Figure 7 is a structural block diagram of a terminal provided by an embodiment of the present application. The terminal can be the terminal in the above embodiments. The terminal can be a portable mobile terminal, such as a smart phone, a tablet computer. The terminal can also be referred to as a user equipment, a portable terminal, and other names.

[0296] Generally, the terminal includes a processor and a memory.

[0297] The processor can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), a FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor can also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit). The coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor can be integrated with a GPU (Graphics Processing Unit) that is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor can also include an AI (Artificial Intelligence) processor for processing machine learning-related computing operations.

[0298] The memory can include one or more computer-readable storage media. The computer-readable storage media can be tangible and non-transitory; the computer-readable storage media can also include volatile and non-volatile, removable and non-removable tangible media, and include but is not limited to: RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, game cartridges, cartridges of other computer-readable media.

[0299] In some embodiments, the terminal can further include a peripheral device interface and at least one peripheral device. Specifically, the peripheral device includes at least one of radio frequency circuit, touch display screen, camera, audio circuit, positioning component and power supply.

[0300] The peripheral device interface can be used to connect the at least one peripheral device related to I / O (Input / Output) to the processor and the memory. In some embodiments, the processor, the memory and the peripheral device interface are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor, the memory and the peripheral device interface can be implemented on a separate chip or circuit board, and the present embodiment is not limited in this regard.

[0301] The radio frequency circuit is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chip set, a subscriber identity module card, etc. The radio frequency circuit can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit can also include NFC (Near Field Communication) related circuit, and the present application is not limited in this regard.

[0302] The touch display screen is used to display a UI (User Interface). The UI can include graphics, text, icons, video, and any combination thereof. The touch display screen also has the ability to capture touch signals on or above the surface of the touch display screen. The touch signals can be input to the processor as control signals for processing. The touch display screen is used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the touch display screen can be one, arranged on the front panel of the terminal; in other embodiments, the touch display screen can be at least two, arranged on different surfaces of the terminal or in a folding design; in still other embodiments, the touch display screen can be a flexible display screen, arranged on a curved surface or a folding surface of the terminal. Even, the touch display screen can also be arranged in an irregular shape, that is, a special-shaped screen. The touch display screen can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0303] The camera assembly is used to capture images or videos. Optionally, the camera assembly includes a front camera and a rear camera. Generally, the front camera is used to implement video calls or selfies, and the rear camera is used to implement photo or video shooting. In some embodiments, the rear camera is at least two, which are any one of a main camera, a depth-of-field camera, and a wide-angle camera, to implement the background blur function by fusing the main camera and the depth-of-field camera, and the panorama shooting and VR (Virtual Reality) shooting functions by fusing the main camera and the wide-angle camera. In some embodiments, the camera assembly can also include a flash. The flash can be a single-color-temperature flash or a dual-color-temperature flash. The dual-color-temperature flash refers to the combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0304] The audio circuit is used to provide an audio interface between the user and the terminal. The audio circuit can include a microphone and a speaker. The microphone is used to capture sound waves of the user and the environment, and convert the sound waves into an electrical signal input to the processor for processing or input to the radio frequency circuit to implement voice communication. For the purpose of stereo sound collection or noise reduction, the microphone can be multiple, arranged at different parts of the terminal. The microphone can also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert the electrical signal from the processor or the radio frequency circuit into sound waves. The speaker can be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for ranging purposes. In some embodiments, the audio circuit can also include a headphone jack.

[0305] The positioning component is used to locate the current geographical position of the terminal to realize navigation or LBS (Location Based Service). The positioning component can be a positioning component based on the GPS (Global Positioning System) of the United States, the Beidou system of China or the Galileo system of Russia.

[0306] The power supply is used to supply power to various components in the terminal. The power supply can be alternating current, direct current, disposable battery or rechargeable battery. When the power supply includes a rechargeable battery, the rechargeable battery can be a wired charging battery or a wireless charging battery. The wired charging battery is a battery charged through a wired line, and the wireless charging battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0307] Those skilled in the art can understand that the structure shown in the above embodiments is not a limitation on the terminal, and the terminal can include more or less components than the structure shown in the drawings, or combine certain components, or adopt a different component arrangement. Figure 7 The structure shown in the above embodiments is not a limitation on the terminal, and the terminal can include more or less components than the structure shown in the drawings, or combine certain components, or adopt a different component arrangement.

[0308] Embodiment Four

[0309] In the exemplary embodiments, a computer readable storage medium is also provided, which stores a computer program. The program is executed by a processor to implement a method for autonomously selecting an optimal grabbing point of a bridge ship unloader grab provided by all the inventive embodiments of the present application.

[0310] Any combination of one or more computer readable medium can be employed. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the above. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0311] Computer readable signal media can include a propagated data signal with computer readable program code embodied therein. For example, a propagated signal can be an electromagnetic signal, an optical signal, and / or any other suitable type of signal. Such a propagated signal can carry computer readable program code in the form of electrical signals, optical signals, and / or magnetic

[0312] Program code embodied on a computer readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0313] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In an embodiment, electronic program guide data can be provided to a user's computer from a remote computer.

[0314] Embodiment Five

[0315] In an exemplary embodiment, an application program product is also provided, comprising one or more instructions executable by the processor of the above-mentioned device to complete the above-mentioned method for autonomously selecting a grab point of a bridge ship unloader grab.

[0316] While the embodiments of the application have been disclosed as above, it is not limited to the applications listed in the specification and the embodiments. It can be fully applied to various fields suitable for the application. Additional modifications can be easily made by those skilled in the art. Therefore, the application is not limited to specific details and the figures shown and described herein without departing from the general concept defined by the claims and their equivalents.

Claims

1. An autonomous selection method for the optimal grabbing point of a bridge ship unloader grab, applied to an autonomous selection system for the optimal grabbing point of a bridge ship unloader grab, the autonomous selection system for the optimal grabbing point of a bridge ship unloader grab comprising a first laser radar and a second laser radar, the first laser radar and the second laser radar being respectively arranged on a ship unloader trolley, characterized in that, The method comprises: Calibrating coordinate systems of the first laser radar and the second laser radar, performing horizontal correction on the calibrated first laser radar point cloud data and the calibrated second laser radar point cloud data to obtain corrected first laser radar point cloud data and corrected second laser radar point cloud data; Obtaining multiple frames of first laser radar point cloud data and multiple frames of second laser radar point cloud data, and obtaining a global point cloud map of a ship cabin based on the multiple frames of first laser radar point cloud data and the multiple frames of second laser radar point cloud data; Based on the global point cloud map of the ship cabin, a hatch recognition is performed to obtain stockpile point cloud data; Based on the stockpile point cloud data, multiple evaluation dimension scores of each candidate grabbing point are determined, wherein the evaluation dimension scores at least include a grabbing feasibility score, a safety index score, an efficiency evaluation index score, a full bucket rate evaluation index score, and a balance evaluation index score; Based on the multiple evaluation dimension scores of each candidate grabbing point, a comprehensive score is obtained through a normalized genetic algorithm; Based on the comprehensive score, an optimal grabbing point is searched in a grabbing point space through a genetic algorithm, and a control instruction set is generated in response to the optimal grabbing point, wherein the control instruction set is used to control a grab bucket to move to a target position to perform a grabbing operation.

2. The method according to claim 1, characterized in that, The method further comprises: In response to the completion of the grabbing by the grab bucket, after the grab bucket is lifted to a safe height, the point cloud data of a corresponding grabbing area in the global three-dimensional point cloud map is deleted according to the target position information of the current grabbing point, the point cloud data of the corresponding grabbing area is scanned and reconstructed in real time, the reconstructed point cloud data is updated to the global three-dimensional point cloud map, and the optimal grabbing point is obtained based on the comprehensive score until a preset operation termination condition is met.

3. The method of claim 1, wherein, After obtaining the corrected first laser radar point cloud data and the corrected second laser radar point cloud data, the following steps are further included: Based on the corrected first laser radar point cloud data, the corrected second laser radar point cloud data, and filtering parameters, the first laser radar point cloud data and the second laser radar point cloud data are obtained through a pass-through filter, wherein the filtering parameters at least include a grab bucket closing time, a grab bucket closing width, a height difference between a laser radar center and a highest point of the grab bucket, and a height of the grab bucket.

4. The method of claim 1, wherein, Based on the multiple frames of first laser radar point cloud data and the multiple frames of second laser radar point cloud data, the global point cloud map of the ship cabin is obtained, which comprises: Based on the multiple frames of first laser radar point cloud data and the multiple frames of second laser radar point cloud data, point cloud splicing is performed through a SLAM technology to obtain the global point cloud map of the ship cabin.

5. The method of claim 1, wherein, Based on the global point cloud map of the ship cabin, the stockpile point cloud data is obtained through the hatch recognition, which comprises: Based on the global point cloud map of the ship cabin, a point cloud area near a hatch is obtained; Based on the point cloud area near the hatch, hatch edge point cloud data is obtained through an AlphaShape algorithm; Based on the hatch edge point cloud data, out-hatch structure point cloud data is obtained through a K-Means clustering algorithm; Based on the hatch opening structure point cloud data, hatch range data is obtained; Based on the hatch range data, the stockpile point cloud data is obtained.

6. The method of claim 1, wherein, Based on the stockpile point cloud data, the multiple evaluation dimension scores of each candidate grabbing point are determined, including: Based on the stockpile point cloud data, multiple points to be evaluated are determined; Based on the multiple points to be evaluated, multiple evaluation dimensions are evaluated respectively to obtain the multiple evaluation dimension scores of each candidate grabbing point, and the evaluation dimensions include: grabbing feasibility, safety index, efficiency evaluation index, full-bucket rate evaluation index, and balance evaluation index.

7. The method of claim 6, wherein, The grabbing feasibility is used to judge whether the grab bucket will collide with the hatch when the grabbing operation is performed at the candidate grabbing point (x, y); And / or; The safety index is used to quantify the safety margin between the candidate grabbing point and the hatch edge; And / or; The efficiency evaluation index is used to evaluate the operation cost required to move from the previous grabbing point to the current candidate point, and the efficiency is quantified by normalized weighted Manhattan distance; And / or; The full-bucket rate evaluation index is used to analyze the height relationship between the candidate point and the surrounding eight neighborhood grids; And / or; The balance evaluation index is used to select a position symmetrical to the historical grabbing point, and the comprehensive score of each candidate point.

8. An autonomous optimal grab point selection device for a bridge ship unloader grab, characterized in that, The device comprises: a correction module for calibrating the coordinate systems of the first laser radar and the second laser radar, performing horizontal correction on the calibrated first laser radar point cloud data and the calibrated second laser radar point cloud data, and obtaining corrected first laser radar point cloud data and corrected second laser radar point cloud data; an acquisition module for acquiring multiple frames of first laser radar point cloud data and multiple frames of second laser radar point cloud data, and obtaining a ship cabin global point cloud map based on the multiple frames of first laser radar point cloud data and the multiple frames of second laser radar point cloud data; an identification module for obtaining stockpile point cloud data through hatch identification based on the ship cabin global point cloud map; an evaluation module for determining multiple evaluation dimension scores of each candidate grabbing point based on the stockpile point cloud data, wherein the evaluation dimension scores at least include: grabbing feasibility score, safety index score, efficiency evaluation index score, full-bucket rate evaluation index score, and balance evaluation index score; a scoring module for obtaining a comprehensive score by a normalized genetic algorithm based on the multiple evaluation dimension scores of each candidate grabbing point evaluation dimension score; a generation module for obtaining an optimal grabbing point by a genetic algorithm in a grabbing point space based on the comprehensive score, and generating a control instruction set in response to the optimal grabbing point, wherein the control instruction set is used to control the grab bucket to move to a target position to perform a grabbing operation.

9. A terminal, characterized by comprising: It comprises: one or more processors; a memory for storing instructions executable by the one or more processors; wherein the one or more processors are configured to: execute a bridge ship unloader grab bucket optimal grabbing point autonomous selection method as claimed in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, comprising: When the instructions in the storage medium are executed by the processor of the terminal, the terminal can execute the method for autonomously selecting an optimal grabbing point of a bridge unloader grab according to any one of claims 1 to 7.

Images