A bridge crane intelligent sensing system and working method thereof
By setting up an image acquisition sensor and a displacement sensor on the bridge crane, combining the perceived positioning and target recognition modules, the intelligent perception system of the bridge crane is realized, solving the problems of positioning and environmental perception, avoiding accidents and prolonging the sensor Service life.
Patent Information
- Application Number
- CN202111630303.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-28
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-12-28
AI Technical Summary
The positioning method of existing bridge cranes has problems such as high installation difficulty, long implementation cycle and high equipment cost, and the surrounding environment cannot be sensed, resulting in easy collision accidents during unmanned operations.
Using an intelligent perception system including displacement sensors, image acquisition sensors and data processing systems, by collecting image point cloud data and position height data above and below the crane, the perception positioning module is used for real-time positioning, and the sensing knowledge identification module is used for target recognition, and through The perception fusion module calculates the position information of the target object to achieve automatic obstacle avoidance.
It realizes accurate positioning and environmental perception of the bridge crane, avoids collisions with goods or equipment, and extends the service life of the sensor.
Smart Images

Figure CN114314345B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of automatic detection, and in particular relates to an intelligent sensing system for a bridge crane and a working method thereof. Background Art
[0002] Bridge crane is a widely used industrial lifting equipment. As a special large-scale mechanical equipment, it is used in various industrial environments, but its operating efficiency and degree of automation are still low, and manual operation is required. With the development of technology, unmanned operation of cranes may become a trend in the future. When performing unmanned operation, cranes need to accurately position themselves and be able to intelligently perceive the surrounding environment.
[0003] In the prior art, the positioning of cranes is mainly achieved by using Graham lines and bar code scales, but the above methods have problems such as high installation difficulty, long implementation period and high equipment cost. In addition, there is also a method of installing sensors on the crane's running track to detect the operating position of the crane, but in certain working scenarios, such as when the track beam is unbalanced, the operation of the crane will cause the sensor to wear and fail to meet the positioning requirements, making the sensor basically in an ineffective state. In addition, existing bridge cranes are unable to perceive the surrounding environment and cannot achieve automatic obstacle avoidance. When the crane is in unmanned operation, it is easy to collide with the cargo or other equipment under the crane, causing accidents. Summary of the invention
[0004] The object of the present invention is to provide an intelligent sensing system for a bridge crane and a working method thereof, so as to solve at least one technical problem in the prior art.
[0005] In order to achieve the above object, the present invention adopts the following technical solutions:
[0006] In a first aspect, the present invention provides an intelligent sensing system for a bridge crane, comprising a sensor subsystem and a data processing subsystem;
[0007] The sensor subsystem includes a displacement sensor disposed on the lifting device of the crane, used to collect position height data of the lifting device; a first image acquisition sensor disposed above the crane trolley, used to collect first image point cloud data of the environment above the crane; and a second image acquisition sensor disposed below the crane trolley, used to collect second image point cloud data of the environment below the crane.
[0008] The data processing subsystem includes a perception positioning module, a perception identification module and a perception fusion module; the perception positioning module is used to calculate the position information of the crane based on the positioning algorithm according to the position height data and the first image point cloud data; the perception identification module is used to identify the target under the crane based on the target detection algorithm according to the second image point cloud data; the perception fusion module is used to perform feature fusion on the position information of the crane and the target identification result of the target object to calculate the position information of the target object.
[0009] In one possible design, the first image acquisition sensor and the second image acquisition sensor both include one or a combination of two-dimensional laser radar, three-dimensional laser radar, monocular camera, binocular camera and depth camera; the displacement sensor includes a displacement absolute encoder.
[0010] In a possible design, the sensor subsystem includes a first displacement absolute encoder disposed on the lifting device, a first three-dimensional laser radar disposed vertically on the trolley, and two second three-dimensional laser radars disposed on opposite sides of the bottom of the trolley;
[0011] Among them, the horizontal field of view angle of the first three-dimensional laser radar is 360 degrees, and its vertical field of view angle α1 is not less than 25 degrees. The vertical field of view angle α2 and horizontal field of view angle β1 of the second three-dimensional laser radar are both not less than 60 degrees.
[0012] In a possible design, the sensor subsystem includes a second displacement absolute encoder disposed on the lifting device, a third three-dimensional laser radar vertically disposed on the trolley, and two first binocular cameras disposed on first opposite sides of the bottom of the trolley;
[0013] Among them, the horizontal field of view angle of the third three-dimensional laser radar is 360 degrees, and its vertical field of view angle α3 is not less than 25 degrees. The vertical field of view angle α4 and the horizontal field of view angle β2 of the first binocular camera are both not less than 60 degrees.
[0014] In a possible design, the sensor subsystem also includes two fourth three-dimensional laser radars arranged on the second opposite sides of the bottom of the vehicle;
[0015] Among them, the vertical field of view angle α5 and the horizontal field of view angle β3 of the fourth three-dimensional laser radar are not less than 60 degrees.
[0016] In a second aspect, the present invention provides a working method of the intelligent sensing system of a bridge crane as described in any possible design of the first aspect, comprising:
[0017] Collecting first image point cloud data of the environment above the crane, second image point cloud data of the environment below the crane, and position height data of the lifting device;
[0018] Calculate the position information of the crane based on the positioning algorithm according to the first image point cloud data and the position height data;
[0019] According to the second image point cloud data, target recognition is performed on the target object under the crane based on a target detection algorithm;
[0020] The position information of the crane and the target result of the target object are feature fused to calculate the position information of the target object.
[0021] In a possible design, the position information of the crane is calculated based on the positioning algorithm according to the first image point cloud data and the position height data, including:
[0022] When the crane trolley and the trolley are moving, the trolley and the trolley are positioned in real time based on the Slam positioning algorithm according to the first image point cloud data collected, and the coordinate position information of the trolley and the trolley in the grid map is calculated;
[0023] When the lifting device of the crane is moving upward or downward, the position height information of the lifting device is directly acquired according to the collected position height data.
[0024] In a possible design, target recognition is performed on the target object under the crane based on the target detection algorithm according to the second image point cloud data, including:
[0025] Based on the 3D point cloud target detection algorithm, first feature data is extracted separately from the second image point cloud data collected by each second image acquisition sensor, and target recognition is performed on each first feature data to obtain respective initial target recognition results;
[0026] Multimodal recognition fusion is performed on all initial target recognition results, and the first target recognition result is obtained through recognition decision.
[0027] In a possible design, after performing target recognition on each first feature data to obtain respective initial target recognition results, the method further includes:
[0028] Performing data fusion on the second point cloud data acquired by all second image acquisition sensors, extracting second feature data from the fused point cloud data, performing target recognition on the second feature data, and obtaining a target fusion recognition result;
[0029] The initial target recognition result and the target fusion recognition result are subjected to hybrid multimodal recognition fusion, and a second target recognition result is obtained through recognition decision.
[0030] In a possible design, feature fusion is performed on the position information of the crane and the target result of the target object to calculate the position information of the target object, including:
[0031] The coordinate position information of the trolley and the cart and the position height information of the lifting device are feature fused with the first target recognition result or the second target recognition result to calculate the position information of the target object in the grid map.
[0032] Beneficial effects:
[0033] The present invention arranges an image acquisition sensor on the crane to respectively collect image point cloud data above and below the crane, and arranges a displacement sensor to collect position height data of the lifting sling, then uses a perception positioning module to perform real-time positioning of the crane, uses a perception recognition module to perform target recognition on the target object below the crane, and uses a perception fusion module to perform feature fusion on the crane position information and the target recognition result, and calculates the position information of the target object, so that automatic obstacle avoidance can be performed according to the position information of the target object to avoid accidents. In addition, the hardware setting of the present invention is simple, the sensor will not be worn during use, and the service life of the sensor is extended. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 is a structural block diagram of the intelligent sensing system for the bridge crane in this embodiment;
[0035] Figure 2 This is a schematic diagram of one installation of the sensor in this embodiment;
[0036] Figure 3 Another installation diagram of the sensor in this embodiment;
[0037] Figure 4 Another installation diagram of the sensor in this embodiment;
[0038] Figure 5 Flow chart of the intelligent sensing method for bridge crane in this embodiment.
[0039] Among them, 1-trolley; 2-truck; 3-lifting equipment. DETAILED DESCRIPTION
[0040] In order to make the purpose, technical solutions and advantages of the embodiments of this specification more clear, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0041] Example
[0042] In order to solve the technical problems in the prior art that the crane positioning method is difficult to install, has a long implementation period, has high equipment cost, and cannot perceive the surrounding environment of the crane, the present embodiment provides an intelligent perception system for a bridge crane and a working method thereof, by arranging an image acquisition sensor on the crane to respectively collect image point cloud data above and below the crane, and by arranging a displacement sensor to collect the position height data of the lifting sling 3, and then using the perception positioning module to perform real-time positioning of the crane, and using the perception recognition module to perform target recognition on the target object below the crane, and using the perception fusion module to perform feature fusion on the crane position information and the target recognition result, and calculate the position information of the target object, so that automatic obstacle avoidance can be performed according to the position information of the target object to avoid accidents, and the hardware setting of the present invention is simple, the sensor will not be worn during use, and the service life of the sensor is extended.
[0043] like Figure 1-4 As shown, in a first aspect, the present invention provides an intelligent sensing system for a bridge crane, comprising a sensor subsystem and a data processing subsystem;
[0044] The sensor subsystem includes a displacement sensor disposed on the crane lifting device 3, which is used to collect the position height data of the lifting device 3; a first image acquisition sensor disposed above the crane trolley 1, which is used to collect the first image point cloud data of the environment above the crane; and a second image acquisition sensor disposed below the crane trolley 1, which is used to collect the second image point cloud data of the environment below the crane.
[0045] The data processing subsystem includes a perception positioning module, a perception identification module and a perception fusion module; the perception positioning module is used to calculate the position information of the crane based on the positioning algorithm according to the position height data and the first image point cloud data; the perception identification module is used to identify the target under the crane based on the target detection algorithm according to the second image point cloud data; the perception fusion module is used to perform feature fusion on the position information of the crane and the target identification result of the target object to calculate the position information of the target object.
[0046] It should be noted that the number of the first image acquisition sensor, the second image acquisition sensor and the displacement sensor can be configured according to the actual industrial environment of the crane, and is not specifically limited.
[0047] Based on the above disclosed content, this embodiment arranges an image acquisition sensor on the crane to respectively collect image point cloud data above and below the crane, and arranges a displacement sensor to collect the position height data of the lifting sling 3, and then uses the perception positioning module to perform real-time positioning of the crane, and uses the perception recognition module to identify the target object under the crane, and uses the perception fusion module to perform feature fusion on the crane position information and the target recognition result, and calculates the position information of the target object, so that automatic obstacle avoidance can be performed according to the position information of the target object to avoid accidents. The hardware setting of the present invention is simple, the sensor will not wear out during use, and the service life of the sensor is extended.
[0048] In a specific embodiment, the first image acquisition sensor and the second image acquisition sensor both include one or a combination of two-dimensional laser radar, three-dimensional laser radar, monocular camera, binocular camera and depth camera; the displacement sensor includes a displacement absolute encoder. Of course, it can be understood that the image acquisition sensor in this embodiment is not limited to the above-mentioned sensors, and the displacement sensor in this embodiment is not limited to the displacement absolute encoder, which will not be explained one by one here.
[0049] like Figure 2 As shown, in a specific embodiment, the sensor subsystem includes a first displacement absolute encoder arranged on the lifting device 3, a first three-dimensional laser radar arranged vertically on the trolley 1, and two second three-dimensional laser radars arranged on opposite sides of the bottom of the trolley 1;
[0050] Among them, the horizontal field of view angle of the first three-dimensional laser radar is 360 degrees, and its vertical field of view angle α1 is not less than 25 degrees. The vertical field of view angle α2 and horizontal field of view angle β1 of the second three-dimensional laser radar are both not less than 60 degrees.
[0051] It should be noted that the horizontal field of view of the first three-dimensional laser radar is 360 degrees, and its vertical field of view angle α1 is not less than 25 degrees. It can scan a very wide area of the surrounding environment above the crane. It is vertically arranged on the trolley 1. As the trolley 2 and the trolley 1 move, it can scan the environment above the crane in real time and obtain the three-dimensional point cloud data of the environment above. The two second three-dimensional laser radars are respectively installed at the front and rear sides or the left and right sides of the bottom of the trolley 1. The specific installation angle can be adjusted according to the actual industrial scene, which is not limited here. By setting the vertical field of view angle α2 and the horizontal field of view angle β1 to not less than 60 degrees, it can ensure that the two second three-dimensional laser radars have a larger scanning field of view, so that they can detect and identify a wider area under the crane; preferably, a single second three-dimensional laser radar can cover at least a scanning range of 8×8 square meters.
[0052] Preferably, the first displacement absolute encoder is arranged on the motor of the lifting device 3 , and can directly locate the position height of the lifting device 3 .
[0053] like Figure 3 As shown, in a specific embodiment, the sensor subsystem includes a first displacement absolute encoder arranged on the lifting device 3, a third three-dimensional laser radar arranged vertically on the trolley 1, and two first binocular cameras arranged on first opposite sides of the bottom of the trolley 1;
[0054] Among them, the horizontal field of view angle of the third three-dimensional laser radar is 360 degrees, and its vertical field of view angle α3 is not less than 25 degrees. The vertical field of view angle α4 and the horizontal field of view angle β2 of the first binocular camera are both not less than 60 degrees.
[0055] It should be noted that the horizontal field of view of the third three-dimensional laser radar is 360 degrees, and its vertical field of view angle α1 is not less than 25 degrees, which can scan a very wide area of the surrounding environment above the crane. It is vertically arranged on the trolley 1, and can scan the environment above the crane in real time as the trolley 2 and the trolley 1 move, and obtain the three-dimensional point cloud data of the environment above. The two first binocular cameras are respectively installed at the front and rear sides or the left and right sides of the bottom of the trolley 1. The specific installation angle can be adjusted according to the actual industrial scene, and is not limited here. By setting the vertical field of view angle α4 and the horizontal field of view angle β2 to not less than 60 degrees, it can ensure that the two first binocular cameras have a larger scanning field of view, so as to detect and identify a wider area under the crane; preferably, a single binocular camera can cover at least a scanning range of 8×8 square meters, and the depth or distance detected by the binocular camera is not less than 20 meters.
[0056] like Figure 4As shown, in a specific embodiment, the sensor subsystem further includes two fourth three-dimensional laser radars arranged on the second opposite sides of the bottom of the vehicle 1;
[0057] Among them, the vertical field of view angle α5 and the horizontal field of view angle β3 of the fourth three-dimensional laser radar are not less than 60 degrees.
[0058] It should be noted that, based on the previous embodiment, two fourth three-dimensional laser radars are additionally arranged at the bottom of the trolley 1 in this embodiment, that is, the sensor subsystem in this embodiment includes a first displacement absolute encoder arranged on the lifting sling 3, a third three-dimensional laser radar arranged vertically on the trolley 1, two first binocular cameras arranged on the first opposite sides of the bottom of the trolley 1, and two fourth three-dimensional laser radars arranged on the second opposite sides of the bottom of the trolley 1. The two fourth three-dimensional laser radars are respectively installed at the front and rear sides or the left and right sides of the bottom of the trolley 1, and the specific installation angle can be adjusted according to the actual industrial scene, which is not limited here. By setting the vertical field of view angle α5 and the horizontal field of view angle β3 to not less than 60 degrees, it can ensure that the two fourth three-dimensional laser radars have a larger scanning field of view, so as to detect and identify a wider area under the crane; preferably, a single fourth three-dimensional laser radar can at least cover a scanning range of 8×8 square meters.
[0059] In a specific implementation, the data processing subsystem is deployed in an industrial computer device based on a CPU+GPU framework, and two industrial computer devices are provided, and the two devices have completely identical structures, forming a one-in-use and one-in-standby working mode. The industrial computer device is used to process, analyze and calculate the data collected by the sensor subsystem, and share the analysis results with external systems, including but not limited to PLC (Programmable Logic Controller) motion control systems, WCS (Numerical Control System) systems and WMS (Warehouse Management System) systems.
[0060] like Figure 5 As shown, in the second aspect, this embodiment provides a working method of the intelligent sensing system of the bridge crane as described in any possible design of the first aspect, including but not limited to being implemented by steps S101 to S104:
[0061] Step S101. Collecting first image point cloud data of the environment above the crane, second image point cloud data of the environment below the crane, and position height data of the lifting device 3;
[0062] Among them, preferably, the first image point cloud data of this embodiment is acquired by a three-dimensional laser radar arranged above the trolley 1, and the first image point cloud data is radar point cloud data; preferably, the second image point cloud data is acquired by a three-dimensional laser radar or a binocular camera arranged below the trolley 1, and the second image point cloud data is radar point cloud data or visual point cloud data; preferably, the position height data is acquired by an absolute displacement encoder arranged on the lifting sling 3, and the position height data is displacement data directly acquired based on the encoder.
[0063] Step S102: obtaining the position information of the crane by calculating based on the positioning algorithm according to the first image point cloud data and the position height data;
[0064] In a specific implementation of step S102, the position information of the crane is calculated based on the positioning algorithm according to the first image point cloud data and the position height data, including:
[0065] Step S1021. When the crane trolley 2 and the trolley 1 are moving, the trolley 2 and the trolley 1 are positioned in real time based on the Slam positioning algorithm according to the first image point cloud data collected, and the coordinate position information of the trolley 2 and the trolley 1 in the grid map is calculated;
[0066] Wherein, preferably, the Slam positioning algorithm includes but is not limited to a Slam algorithm based on graph optimization (such as Cartographer, etc.), which is not limited here.
[0067] It should be noted that, before step S1021, the following steps are also included:
[0068] (1) The data acquisition accuracy of the three-dimensional laser radar installed above the trolley 1 can be adjusted. The three-dimensional point cloud data of the environment can be acquired by scanning multiple times in the working area of the crane to improve the acquisition accuracy of the radar point cloud data.
[0069] (2) Calibrate the relative coordinates between the installation position of the three-dimensional laser radar above the trolley 1 and the center point position of the lifting sling 3, and perform coordinate transformation on the installation position of the three-dimensional laser radar and the center point position of the lifting sling 3, which can be achieved specifically by coordinate translation.
[0070] (3) The point cloud data of the three-dimensional laser radar above the trolley 1 is registered, and the data is matched with the upper environmental position to realize the position calibration and alignment of the physical space position of the crane and the virtual grid map.
[0071] (4) The crane work and virtual grid map is established through the Slam algorithm, and the map is trimmed and divided into functional areas.
[0072] It should be noted that the above steps (1) to (4) are only required to calibrate and calibrate the system before the first image point cloud data is collected for the first time, and the above operations are no longer required for subsequent data collection.
[0073] Step S1022: When the crane lifting device 3 moves upward or downward, directly obtain the position height information of the lifting device 3 according to the collected position height data.
[0074] It should be noted that, before step S1022, the following steps are also included:
[0075] The initial zero point position of the displacement absolute encoder is calibrated according to the position of the rising stop limit position of the lifting sling 3; among them, this step only needs to calibrate the initial zero point position of the displacement absolute encoder before collecting the position height data for the first time, and this step does not need to be performed again during subsequent data collection.
[0076] Based on the above disclosed content, this embodiment uses the radar point cloud data collected by the three-dimensional laser radar, adopts the Slam algorithm to perform real-time positioning and map coordinate construction of the environment above the crane, and calculates the position information of the trolley 2 and the car 1; uses the displacement data collected by the displacement absolute encoder and directly reads the position height data, thereby realizing real-time positioning of the crane trolley 2, trolley 1 and lifting device 3, which is convenient for monitoring the working status of the crane.
[0077] Step S103. According to the second image point cloud data, target recognition is performed on the target object under the crane based on a target detection algorithm;
[0078] Among them, preferably, the target detection algorithm adopts a 3D point cloud target detection algorithm, including but not limited to PointNet++, PointCRNN and Yolo3D algorithms, which are not limited here.
[0079] It should be noted that, based on the target detection algorithm, the target object under the crane is identified, and the target identification result obtained includes the target type and local position information, wherein the local position information refers to the local position information of the target object within the field of view of the second image acquisition sensor, that is, the relative position information of the target object and the second image acquisition sensor; the second image acquisition sensor includes but is not limited to two three-dimensional laser radars arranged on opposite sides of the bottom of the trolley and / or two binocular cameras arranged on first opposite sides of the bottom of the trolley.
[0080] Preferably, before step S103, the method further includes:
[0081] (1) Synchronize the information time and space of all sensors (including radar sensors, visual sensors and displacement sensors) and calibrate the reference time.
[0082] (ii) According to the virtual grid map obtained by the perception and positioning module, a calibration position coordinate is selected, the crane is manipulated to the calibration position coordinate, and a calibration object is placed at the calibration position.
[0083] (3) When a three-dimensional laser radar is arranged above the car 1 and two three-dimensional laser radars or two binocular cameras are arranged below the car 1, the point cloud data of the two three-dimensional laser radars or two binocular cameras below the car 1 are respectively aligned, and then the coordinates of multiple point cloud data are transformed and unified into the same coordinate system; when a three-dimensional laser radar is arranged above the car 1 and two three-dimensional laser radars and two binocular cameras are arranged below the car 1, the three-dimensional laser radar point cloud data and the binocular camera point cloud data are aligned at the same time, and the coordinates are calibrated according to the relative posture position of the three-dimensional laser radar and the binocular camera, and finally the radar point cloud data and the visual point cloud data are unified into the coordinate system.
[0084] It should be noted that the above steps (i) to (iii) only need to be set before the first acquisition of the second image data, and there is no need to operate steps (i) to (iii) when the second image data is subsequently acquired.
[0085] In a specific implementation of step S103, target recognition is performed on the target object under the crane based on the target detection algorithm according to the second image point cloud data, including:
[0086] Step S1031. Based on the 3D point cloud target detection algorithm, first feature data is extracted separately from each second image point cloud data collected by the second image acquisition sensor, and target recognition is performed on each first feature data to obtain respective initial target recognition results;
[0087] Specifically, when a three-dimensional laser radar and / or a binocular camera is arranged under the car 1, feature data extraction is performed separately for each single radar point cloud data collected by each three-dimensional laser radar, and / or feature data extraction is performed separately for each single visual point cloud data collected by the binocular camera, and then target recognition detection is performed on each single radar point cloud data and / or each single visual point cloud data to obtain respective initial target recognition results. At this time, there are multiple initial target recognition results.
[0088] Step S1032: Perform multimodal recognition fusion on all initial target recognition results, and obtain the first target recognition result through recognition decision.
[0089] In order to enrich the collected point cloud data and make the target detection results obtained subsequently more accurate, in another specific implementation of step S103, after performing target recognition on each first feature data and obtaining respective initial target recognition results, the following is further included:
[0090] Step S1031a. Performing data fusion on the second image point cloud data collected by all second image acquisition sensors, extracting second feature data from the fused image point cloud data, performing target recognition on the second feature data, and obtaining a target fusion recognition result;
[0091] Specifically, when a three-dimensional laser radar and / or a binocular camera is arranged under the car 1, data fusion is performed on the single radar point cloud data and / or the single visual point cloud data collected by all the three-dimensional laser radars and / or binocular cameras, and feature data is extracted from the fused point cloud data to obtain the target fusion recognition result.
[0092] Step S1032a. Perform hybrid multimodal recognition fusion on the initial target recognition result and the target fusion recognition result, and obtain a second target recognition result through recognition decision.
[0093] Step S104: Perform feature fusion on the position information of the crane and the target result of the target object to calculate the position information of the target object.
[0094] In a specific implementation of step S104, feature fusion is performed on the position information of the crane and the target result of the target object to calculate the position information of the target object, including:
[0095] The coordinate position information of the trolley 2 and the trolley 1 and the position height information of the lifting device 3 are feature fused with the first target recognition result or the second target recognition result to calculate the position information of the target object in the grid map.
[0096] It should be noted that the target objects include static objects, such as fixed equipment or cargo, and dynamic objects, such as mobile equipment or people. By identifying the target object and calculating the position information of the target object in the grid map, the crane can automatically avoid obstacles to the target object to prevent safety accidents.
[0097] It should be noted that the position information of the target object in the grid map calculated here corresponds to the global position information of the target object in the actual physical space, so that the crane can perform correct path planning based on the global position information and achieve reasonable obstacle avoidance.
[0098] Beneficial effects:
[0099] In this embodiment, an image acquisition sensor is arranged on the crane to respectively collect image point cloud data above and below the crane, and a displacement sensor is arranged to collect position height data of the lifting sling 3. Then, the crane is positioned in real time by using a perception positioning module, and the target object below the crane is identified by using a perception recognition module. The position information of the crane and the target recognition result are feature-fused by using a perception fusion module to calculate the position information of the target object, so that automatic obstacle avoidance can be performed according to the position information of the target object to avoid accidents. In addition, the hardware setting of the present invention is simple, and the sensor will not be worn during use, thereby extending the service life of the sensor.
[0100] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. An intelligent sensing system for a bridge crane, characterized in that: Includes sensor subsystem and data processing subsystem; The sensor subsystem includes a displacement sensor disposed on the lifting device of the crane, and is used to collect position height data of the lifting device; A first image acquisition sensor disposed above the crane trolley, for acquiring first image point cloud data of the environment above the crane; A second image acquisition sensor disposed below the crane trolley, for acquiring second image point cloud data of the environment below the crane; The data processing subsystem includes a perception positioning module, a perception identification module and a perception fusion module; the perception positioning module is used to calculate the position information of the crane based on the positioning algorithm according to the position height data and the first image point cloud data; the perception identification module is used to identify the target under the crane based on the target detection algorithm according to the second image point cloud data; the perception fusion module is used to perform feature fusion on the position information of the crane and the target identification result of the target object to calculate the position information of the target object.
2. The intelligent sensing system for bridge crane according to claim 1 is characterized in that: The first image acquisition sensor and the second image acquisition sensor both include one or a combination of two-dimensional laser radar, three-dimensional laser radar, monocular camera, binocular camera and depth camera; the displacement sensor includes a displacement absolute encoder.
3. The intelligent sensing system for bridge crane according to claim 2 is characterized in that: The sensor subsystem includes a first displacement absolute encoder arranged on the lifting device, a first three-dimensional laser radar arranged vertically on the trolley, and two second three-dimensional laser radars arranged on opposite sides of the bottom of the trolley; Among them, the horizontal field of view angle of the first three-dimensional laser radar is 360 degrees, and its vertical field of view angle α1 is not less than 25 degrees. The vertical field of view angle α2 and horizontal field of view angle β1 of the second three-dimensional laser radar are both not less than 60 degrees.
4. The intelligent sensing system for bridge crane according to claim 2, characterized in that: The sensor subsystem includes a second displacement absolute encoder arranged on the lifting device, a third three-dimensional laser radar arranged vertically on the trolley, and two first binocular cameras arranged on first opposite sides of the bottom of the trolley; Among them, the horizontal field of view angle of the third three-dimensional laser radar is 360 degrees, and its vertical field of view angle α3 is not less than 25 degrees. The vertical field of view angle α4 and the horizontal field of view angle β2 of the first binocular camera are both not less than 60 degrees.
5. The intelligent sensing system for bridge crane according to claim 4 is characterized in that: The sensor subsystem also includes two fourth three-dimensional laser radars disposed on second opposite sides of the bottom of the vehicle; Among them, the vertical field of view angle α5 and the horizontal field of view angle β3 of the fourth three-dimensional laser radar are not less than 60 degrees.
6. A working method of the intelligent sensing system of a bridge crane according to any one of claims 1 to 5, characterized in that: include: Collecting first image point cloud data of the environment above the crane, second image point cloud data of the environment below the crane, and position height data of the lifting device; Calculate the position information of the crane based on the positioning algorithm according to the first image point cloud data and the position height data; According to the second image point cloud data, target recognition is performed on the target object under the crane based on a target detection algorithm; The position information of the crane and the target result of the target object are feature fused to calculate the position information of the target object.
7. The working method according to claim 6, characterized in that: According to the first image point cloud data and the position height data, the position information of the crane is calculated based on a positioning algorithm, including: When the crane trolley and the trolley are moving, the trolley and the trolley are positioned in real time based on the Slam positioning algorithm according to the first image point cloud data collected, and the coordinate position information of the trolley and the trolley in the grid map is calculated; When the lifting device of the crane is moving upward or downward, the position height information of the lifting device is directly acquired according to the collected position height data.
8. The working method according to claim 7, characterized in that: According to the second image point cloud data, target recognition is performed on the target object under the crane based on a target detection algorithm, including: Based on the 3D point cloud target detection algorithm, first feature data is extracted separately from the second image point cloud data collected by each second image acquisition sensor, and target recognition is performed on each first feature data to obtain respective initial target recognition results; Multimodal recognition fusion is performed on all initial target recognition results, and the first target recognition result is obtained through recognition decision.
9. The working method according to claim 8, characterized in that: After performing target recognition on each first feature data to obtain respective initial target recognition results, the method further includes: Performing data fusion on the second image point cloud data acquired by all second image acquisition sensors, extracting second feature data from the fused image point cloud data, performing target recognition on the second feature data, and obtaining a target fusion recognition result; The initial target recognition result and the target fusion recognition result are subjected to hybrid multimodal recognition fusion, and a second target recognition result is obtained through recognition decision.
10. The working method according to claim 9, characterized in that: The position information of the crane and the target result of the target are fused to obtain the position information of the target object, including: The coordinate position information of the trolley and the cart and the position height information of the lifting device are feature fused with the first target recognition result or the second target recognition result to calculate the position information of the target object in the grid map.
