Arm-mounted holder, arm-mounted follow-up sensing device and operation machine

Abstract

The utility model relates to the field of operation machinery, and discloses an arm-mounted holder, an arm-mounted follow-up sensing device and operation machinery. The arm-mounted holder can be mounted on an arm frame and comprises a rotating mechanism, a servo-actuated sensing device and a control mechanism; the rotating mechanism is connected with the swinging mechanism and can drive the swinging mechanism to rotate around the rotating axis, and the swinging mechanism is used for installing the scene detection unit and can drive the scene detection unit to swing around the swinging axis so as to adjust the swinging angle between the optical axis of the scene detection unit and the rotating axis; the transverse rolling mechanism can drive the rotating mechanism to rotate around the central axis of the arm frame when the arm frame is twisted; and the pitching mechanism can drive the rotating mechanism to do up-down pitching motion when the boom changes the amplitude. According to the arm-mounted holder, the arm-mounted follow-up sensing device and the operation machine, large-field-of-view monitoring in a large operation range can be achieved.

Description

Technical Field

[0001] The present application belongs to the field of operating machinery, and specifically relates to an arm-mounted pan / tilt platform, an arm-mounted follow-up sensing device and an operating machinery. Background Art

[0002] During the operation of boom-type operating machinery, it is usually necessary to obtain environmental information around the boom to ensure operational safety. Taking the operating machinery as a crane, the crane has a large hoisting operation area. For example, a large crane can have a maximum arm length of hundreds of meters, and a maximum amplitude and maximum operating radius of more than 60 meters. The operator is in the operating room, and the distance from the hoisting target can be up to more than 60 meters. The operator's operating field of vision is limited during hoisting, which brings hidden dangers to operational safety. Utility Model Content

[0003] The purpose of this application is to provide an arm-mounted pan / tilt platform, an arm-mounted follow-up sensing device and an operating machine, which can realize wide-field monitoring of a large operating range.

[0004] In order to achieve the above-mentioned object, the present application provides an arm-mounted gimbal, which can be installed on an arm frame and includes:

[0005] Rotating mechanism;

[0006] a swing mechanism, the rotating mechanism being connected to the swing mechanism and capable of driving the swing mechanism to rotate around a rotation axis, the swing mechanism being used to mount a scene detection unit and capable of driving the scene detection unit to swing around the swing axis to adjust a swing angle between an optical axis of the scene detection unit and the rotation axis; and

[0007] The pitch mechanism is connected between the arm and the rotating mechanism and can drive the rotating mechanism to perform up and down pitch motion around the pitch axis.

[0008] In some specific implementations, the arm-mounted gimbal further includes:

[0009] A roll mechanism is connected between the arm and the rotation mechanism and can drive the rotation mechanism to rotate around a roll axis, and the roll axis is perpendicular to the pitch axis.

[0010] In some specific implementations, the arm-mounted gimbal further includes:

[0011] A pitch mechanism can drive the rotating mechanism to perform up and down pitch movements when the boom is luffed.

[0012] In some specific implementations, the pitch mechanism, the roll mechanism, the rotation mechanism, and the swing mechanism are connected in sequence.

[0013] In some specific embodiments, the swing mechanism includes a swing drive and a swing bracket, the rotation mechanism includes a rotation drive and a rotation bracket, the roll mechanism includes a roll drive and a roll bracket, the pitch mechanism includes a pitch drive and a pitch bracket, the swing drive is arranged on the swing bracket and the drive shaft of the swing drive can be driven and connected to the scene detection unit, the rotation drive is arranged on the rotation bracket and the drive shaft of the rotation drive is driven and connected to the swing bracket, the roll drive is arranged on the roll bracket and the drive shaft of the roll drive is driven and connected to the rotation bracket, and the pitch drive is arranged on the pitch bracket and the drive shaft of the pitch drive is driven and connected to the roll bracket.

[0014] In some specific embodiments, the swing mechanism includes a swing drive and a swing bracket, the rotation mechanism includes a rotation drive and a rotation bracket, the roll mechanism includes a roll drive and a roll bracket, the pitch mechanism includes a pitch drive and a pitch bracket, the swing drive is arranged on the swing bracket and the drive shaft of the swing drive can be driven and connected to the scene detection unit, the rotation drive is arranged on the rotation bracket and the drive shaft of the rotation drive is driven and connected to the swing bracket, the pitch drive is arranged on the pitch bracket and the drive shaft of the pitch drive is driven and connected to the rotating bracket, and the roll drive is arranged on the roll bracket and the drive shaft of the roll drive is driven and connected to the pitch bracket.

[0015] In some specific implementations, the arm-mounted gimbal further includes:

[0016] A gimbal fixing bracket, wherein the gimbal fixing bracket can be connected to the arm, and the rotating mechanism is arranged on the gimbal fixing bracket through the rolling mechanism and the pitching mechanism.

[0017] A second aspect of the present application provides an arm-mounted follow-up sensing device, the arm-mounted follow-up sensing device comprising:

[0018] The arm-mounted gimbal as described above; and

[0019] The scene detection unit is arranged on the arm-mounted gimbal and is used to obtain scanning data of the working scene.

[0020] In some specific embodiments, the scene detection unit includes an image acquisition component and / or a radar.

[0021] In some specific embodiments, the arm-mounted tracking sensing device further includes:

[0022] A first angle sensor, disposed on the swing mechanism and used to detect a swing angle of the swing mechanism; and

[0023] The second angle sensor is arranged on the rotating mechanism and is used to detect the rotation angle of the rotating mechanism.

[0024] A third aspect of the present application provides a working machine, the working machine comprising:

[0025] Jib; and

[0026] The above-mentioned arm-mounted follow-up sensing device is arranged on the arm support.

[0027] In some specific embodiments, the arm-mounted tracking sensing device is disposed on the tip of the arm of the arm support.

[0028] Through the above technical solution, the rotating mechanism is arranged on the boom through the pitch mechanism. When the boom is extended or the amplitude is changed, the pitch mechanism can adjust the rotation axis to be perpendicular to the ground plane by driving the pitch action of the rotating mechanism, so as to facilitate the scene detection unit to perform rotational scanning of various directions of the working area. The scene detection unit is arranged on the rotating mechanism through the swing mechanism. The swing mechanism can drive the scene detection unit to swing around the swing axis to adjust the swing angle between the optical axis of the scene detection unit and the rotation axis. In this way, the action of the arm-mounted gimbal can enable the scene detection unit to perform rotational scanning at different inclination angles, thereby expanding the scanning coverage area of ​​the scene detection unit, thereby realizing large-field monitoring of a large working range, which is conducive to eliminating safety hazards in the operation process.

[0029] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the embodiments of the present application, but do not constitute a limitation on the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without creative work. In the drawings:

[0031] Figure 1 A schematic diagram showing the structure of an arm-mounted follow-up sensing device according to a specific embodiment of the present application is shown;

[0032] Figure 2 Shows Figure 1 A schematic diagram of the structure of the arm-mounted tracking sensing device in another perspective;

[0033] Figure 3 Shows Figure 1 A partial structural perspective view of the arm-mounted follow-up sensing device;

[0034] Figure 4 A schematic diagram of the operation of a crane according to a specific embodiment of the present application is shown;

[0035] Figure 5 A schematic diagram showing a scanning state of the arm-mounted follow-up sensing device of the present application when the boom of the crane is not extended;

[0036] Figure 6 A schematic diagram showing the scanning state of the arm-mounted follow-up sensing device of the present application when the boom of the crane is extended;

[0037] Figure 7 A schematic diagram of the scanning state of the arm-mounted follow-up sensing device of the present application is shown when the optical axis of the scene detection unit is inclined to the rotation axis;

[0038] Figure 8 A control logic flow chart of an arm-mounted follow-up sensing device according to a specific implementation of the present application is shown.

[0039] Description of Reference Numerals

[0040] 1 Arm-mounted follow-up sensing device 11 Swing mechanism

[0041] 111 Swing drive 112 Swing bracket

[0042] 12 Rotating mechanism 121 Rotating drive member

[0043] 122 Rotating bracket 13 Rolling mechanism

[0044] 131 Roll drive 132 Roll bracket

[0045] 14 Pitch mechanism 141 Pitch drive member

[0046] 142 Pitch bracket 151 Laser radar

[0047] 152 image acquisition component 17 processing device

[0048] 2 Cranes 21 Booms

[0049] 22 Human-computer interaction device DETAILED DESCRIPTION

[0050] The specific implementation of the present application is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present application, and is not used to limit the present application.

[0051] To improve the safety and intelligence of large-scale operating machinery, such as Figure 1 and Figure 2As shown, the first aspect of the present application provides an arm-mounted gimbal, which can be installed on the arm 21 of the operating machine and is used to install a scene detection unit, which is used to obtain scanning data of the operating scene. In this way, it is conducive to the online reconstruction of the hoisting operation scene, the identification and positioning of targets in large spaces and long distances, and the dynamic detection of spatial obstacles, thereby eliminating the operating field limitation of the operator, greatly reducing the difficulty of operating the operating machine, and also making the operating machine more suitable for automatic or unmanned operations in dangerous operating scenes or repetitive operating requirements.

[0052] The arm-mounted gimbal of the present application includes a pitch mechanism 14, a rotation mechanism 12 and a swing mechanism 11, wherein the rotation mechanism 12 is connected to the swing mechanism 11, and the scene detection unit can be installed on the swing mechanism 11. The rotation mechanism 12 can drive the swing mechanism 11 to rotate around the rotation axis to drive the scene detection unit to perform 360° rotation scanning, and the swing mechanism 11 can drive the scene detection unit to swing around the swing axis to adjust the swing angle between the optical axis of the scene detection unit and the rotation axis, and the swing axis can be parallel to the ground plane.

[0053] In the embodiment of the present application, the operating machinery may be an operating equipment with an arm, such as a crane, a pump truck, a high-rise jet fire truck, an aerial operating machinery or an arm-type robot. Taking the operating machinery as a crane as an example, since the maximum amplitude and maximum operating radius of a large crane operation can reach more than 60 meters, and the minimum amplitude and minimum operating radius can be as low as a few meters, the difference between the maximum amplitude and the minimum amplitude is relatively large, and the difference between the maximum operating radius and the minimum operating radius is also relatively large, which can reach tens of meters. In this way, the arm-mounted pan-tilt platform of the present application can also realize large-viewing field monitoring and three-dimensional reconstruction of a large operating range for a scene detection unit with a small field of view and relatively low equipment cost, and can also effectively reduce the manufacturing cost of large cranes. In addition, since the arm-mounted pan-tilt platform is additionally provided with a swing mechanism 11 that can adjust the swing angle during scanning, the arm-mounted follow-up sensing device 1 that is used in conjunction with a scene detection unit with a small field of view angle can also be applied to operating machinery of different specifications and models, especially large operating machinery, and the versatility is greatly improved.

[0054] In addition, when the boom 21 is raised or the boom is changed, the working posture of the boom 21 will change, and the posture of the rotating mechanism 12 will also change, resulting in a change in the vertical relationship between the rotation axis and the ground plane. Figure 1 and Figure 2As shown, the arm-mounted gimbal further includes a pitch mechanism 14, which can drive the rotating mechanism 12 to perform up and down pitch motion around the pitch axis when the arm 21 changes its amplitude. In this way, when the arm 21 is raised (changed), the rotation axis of the scene detection unit can be adjusted to be perpendicular to the ground plane by adjusting the pitch angle through the pitch mechanism 14, so that the scene detection unit can perform rotational scanning of various directions of the working area, and can also perform comprehensive and effective scanning of the environment below and around the scene detection unit, which helps to reduce the control difficulty of the arm-mounted gimbal and simplify the subsequent scanning data processing steps.

[0055] The pitch axis is perpendicular to the amplitude changing direction of the boom 21 , and the amplitude changing direction of the boom 21 is the extending direction of the projection of the central axis of the boom 21 on the ground plane.

[0056] In order to better reflect the technical effects that can be brought about by the embodiments of the present application, the functional implementation method of the arm-mounted gimbal is described below in combination with some application examples. In the arm-mounted gimbal of the present application, the swing mechanism 11, the rotation mechanism 12 and the pitch mechanism 14 can belong to two different control modules. The swing mechanism 11 and the rotation mechanism 12 are rotation scanning operation control modules, and the pitch mechanism 14 is a posture adjustment control module. By setting the pitch mechanism 14, it is possible to dynamically and real-time control the action of the pitch mechanism 14 when the operating posture of the arm 21 changes, thereby changing the posture of the rotation mechanism 12 so that the rotation axis always remains perpendicular to the ground plane. As for the rotary scanning operation module of the swing mechanism 11 and the rotating mechanism 12, the scene detection unit is mounted on the swing mechanism 11 and has an azimuth calibration relationship. The swing mechanism 11 and the rotating mechanism 12 can be independently controlled to scan the operation scene in steps during the rotary scanning operation. According to the calibration relationship of the scene detection unit, it helps to improve the post-processing efficiency of the scanned data. For example, when the scanned data is used for scene reconstruction, the efficiency of scene reconstruction and the real-time performance of the algorithm can be improved; or, when the scanned data is used for scene obstacle detection, the obstacle detection efficiency and positioning accuracy can be effectively improved. Of course, it is easy to understand that the above are some examples of some applicable ways of scanning data. In actual applications, the scanning data can also be used for other purposes, such as pedestrian detection, etc., which will not be given one by one here.

[0057] Furthermore, taking the application of scanning data to scene reconstruction as an example, in order to improve the efficiency of scene reconstruction and the real-time performance of the algorithm, the arm-mounted follow-up sensing system of the present application can be provided with two control coordinate systems. Specifically, the pitch axis of the pitch mechanism 14 is a dynamically changing axis, and the pitch axis of the pitch mechanism 14 is set in one of the independent control coordinate systems; while the swing axis of the swing mechanism 11 and the rotation axis of the rotating mechanism 12 driving the scene detection unit to rotate are controllable and adjustable axes according to the working conditions, and the swing axis of the swing mechanism 11 and the rotation axis of the rotating mechanism 12 driving the scene detection unit to rotate are separately set in another coordinate system.

[0058] Moreover, different from the existing common three-axis gimbal, since the arm-mounted gimbal of the present application is provided with a pitch mechanism 14 between the rotating mechanism 12 and the arm 21, the arm-mounted gimbal can be installed on the arm 21 in any orientation. When the scene detection unit rotates and scans, the rotation axis can be automatically controlled to be perpendicular to the ground plane, so that the scene detection unit below and the surrounding environment can be fully and effectively scanned, and it helps to simplify the subsequent scanning data processing steps of the processing device. Obviously, for the existing common three-axis gimbal, since the camera is set on the pitch structure or the roll structure, the pitch structure or the roll structure is set on the rotating structure, and the rotating structure is set on the mounting platform, this common three-axis gimbal cannot ensure that the camera performs a comprehensive and effective scan of the below and the surrounding environment. If the existing common three-axis gimbal is installed on the arm, once the arm is tilted, the rotation axis of the camera is also tilted in a completely decoupled manner, resulting in that some orientations cannot be effectively scanned, and the shape of the scanning area will also change, resulting in that the subsequent processing of the scanning data will inevitably be very complicated. Moreover, if one wants to obtain the all-round scanning area in this case through the existing ordinary three-axis gimbal, at least two degrees of freedom coupled motion are required, and the difficulty of gimbal control will be significantly increased.

[0059] In addition, if Figure 3 As shown, the scene detection unit includes detection components such as a laser radar 151 and / or an image acquisition component 152. The field of view angles of laser radars 151 and image acquisition components 152 of different models and specifications are different. The swing angle of the scene detection unit can be flexibly adjusted by setting a swing mechanism 11, so that the versatility of the arm-mounted gimbal is greatly improved. Among them, the scene detection unit performs a rotational scan of the working area, which can facilitate the subsequent generation of a global map of the working scene, and can also facilitate monitoring, tracking, and obstacle identification during operation.

[0060] Optionally, the structures of the rotating mechanism 12, the swinging mechanism 11 and the pitching mechanism 14 may be various. The swinging mechanism 11 may include a swinging driving member 111 and a swinging bracket 112, and the rotating mechanism 12 may include a rotating driving member 121 and a rotating bracket 122. The swinging driving member 111 is arranged on the swinging bracket 112, and the driving shaft of the swinging driving member 111 can be connected to the scene detection unit to drive the scene detection unit to swing. The rotating driving member 121 is arranged on the rotating bracket 122, and the driving shaft of the rotating driving member 121 is connected to the swinging bracket 112 to drive the swinging mechanism 11 to rotate. Among them, the swinging driving member 111 may include a servo motor, a stepping motor or an electric cylinder, etc., and the shape of the swinging bracket 112 may be a U-shaped bracket or an irregular bracket, etc. The scene detection unit may be directly connected to the driving shaft of the swinging driving member 111, or indirectly connected to the driving shaft of the swinging driving member 111 through a connecting frame, an intermediate gear assembly or a connecting rod assembly. Similarly, the rotating drive member 121 may include a servo motor or a stepper motor, etc., the rotating bracket 122 may be an L-shaped or irregular bracket, etc., and the swing bracket 112 may be directly connected to the driving shaft of the rotating drive member 121, or indirectly connected to the driving shaft of the rotating drive member 121 through a connecting frame, an intermediate gear assembly, etc. The pitch mechanism 14 may include a pitch driving member 141 and a pitch bracket 142, the pitch driving member 141 may include a servo motor, a stepper motor or an electric cylinder, etc., and the shape of the pitch bracket 142 may be a U-shaped bracket or an irregular bracket, etc.

[0061] Since the scene detection unit of the present application needs to perform rotation scanning, in order to facilitate the subsequent online reconstruction of the three-dimensional scene map, it is necessary to ensure that the rotation axis is perpendicular to the ground plane. However, in actual operation, factors such as the swing of the load on the boom 21 or the tilt of the vehicle body will cause the end of the boom 21 to twist. The twisting of the boom 21 will cause the rotating mechanism 12 to tilt, and the vertical relationship between the rotation axis and the ground plane will also change.

[0062] To this end, in some specific embodiments, such as Figure 1 and Figure 2 As shown, the arm-mounted gimbal of the present application also includes a roll mechanism 13 connected to the pitch mechanism 14, and the roll mechanism 13 and the pitch mechanism 14 are connected between the arm 21 and the rotation mechanism 12. The roll mechanism 13 can drive the rotation mechanism 12 to rotate around the roll axis when the arm 21 is twisted, and the roll axis is parallel to the amplitude change direction of the arm 21 and perpendicular to the pitch axis. In this way, when the arm 21 is twisted, the rotation axis of the scene detection unit can adjust the roll angle through the roll mechanism 13 to ensure that it is perpendicular to the ground plane. Among them, the roll mechanism 13 may include a roll drive member 131 and a roll bracket 132, the roll drive member 131 may include a servo motor, a stepper motor or an electric cylinder, etc., and the shape of the roll bracket 132 may be a U-shaped bracket or an irregular bracket, etc.

[0063] Furthermore, the rolling mechanism 13 and the pitching mechanism 14 may belong to the same control module, that is, the rolling mechanism 13 and the pitching mechanism 14 belong to the same posture adjustment control module. By setting the rolling mechanism 13 and the pitching mechanism 14, when the working posture of the boom 21 changes, the actions of the rolling mechanism 13 and the pitching mechanism 14 can be dynamically and real-time controlled, thereby changing the posture of the rotating mechanism 12 so that the rotation axis is always kept perpendicular to the ground plane.

[0064] Furthermore, taking the application of scanning data in scene reconstruction as an example, in order to improve the efficiency of scene reconstruction and the real-time performance of the algorithm, when the arm-mounted follow-up sensing system of the present application is provided with two control coordinate systems, the roll axis of the roll mechanism 13 is also a dynamically changing axis, and the pitch axis of the pitch mechanism 14 and the roll axis of the roll mechanism 13 are arranged in the same control coordinate system.

[0065] Since the arm-mounted gimbal of the present application is provided with a roll mechanism 13 and a pitch mechanism 14 between the rotation mechanism 13 and the arm 21, when the arm 21 changes amplitude and tilts, it can also perform a comprehensive and effective scan of the area below the scene detection unit and the surrounding environment, while helping to simplify subsequent scanning data processing steps and reduce the difficulty of controlling the arm-mounted gimbal.

[0066] Optionally, the swing angle of the swing mechanism 11 is adjusted in the range of -32° to 32°. The pitch angle of the pitch mechanism 14 is adjusted in the range of -10° to 90°. The roll angle of the roll mechanism 13 is adjusted in the range of -15° to 15°. The rotation angle of the rotation mechanism 12 is controlled in the range of -20° to 360° in the forward direction or -20° to 360° in the reverse direction.

[0067] In some specific embodiments, Figure 1 and Figure 2 As shown, the swing mechanism 11, the rotation mechanism 12, the roll mechanism 13 and the pitch mechanism 14 are connected in sequence. Specifically, the swing drive member 111 can be arranged on the swing bracket 112 and the drive shaft of the swing drive member 111 can be driven and connected to the scene detection unit, the rotation drive member 121 is arranged on the rotation bracket 122 and the drive shaft of the rotation drive member 121 is driven and connected to the swing bracket 112, the roll drive member 131 is arranged on the roll bracket 132 and the drive shaft of the roll drive member 131 is driven and connected to the rotation bracket 122, and the pitch drive member 141 is arranged on the pitch bracket 142 and the drive shaft of the pitch drive member 141 is driven and connected to the roll bracket 132. Among them, the rotation bracket 122 can be directly connected to the drive shaft of the roll drive member 131, and can also be indirectly connected to the drive shaft of the roll drive member 131 through a connecting frame, an intermediate gear assembly, etc. The roll bracket 132 may be directly connected to the driving shaft of the pitch driving component 141 , or may be indirectly connected to the driving shaft of the pitch driving component 141 via a connecting frame, an intermediate gear assembly, etc.

[0068] Alternatively, in some other specific embodiments, the swing mechanism 11, the rotation mechanism 12, the pitch mechanism 14 and the roll mechanism 13 are connected in sequence (not shown in the figure). Specifically, the swing drive member 111 can be arranged on the swing bracket 112 and the drive shaft of the swing drive member 111 can be driven and connected to the scene detection unit, the rotation drive member 121 is arranged on the rotation bracket 122 and the drive shaft of the rotation drive member 121 is driven and connected to the swing bracket 112, the pitch drive member 141 is arranged on the pitch bracket 142 and the drive shaft of the pitch drive member 141 is driven and connected to the rotation bracket 122, and the roll drive member 131 is arranged on the roll bracket 132 and the drive shaft of the roll drive member 131 is driven and connected to the pitch bracket 142. Among them, the rotation bracket 122 can be directly connected to the drive shaft of the pitch drive member 141, and can also be indirectly connected to the drive shaft of the pitch drive member 141 through a connecting frame, an intermediate gear assembly, etc. The pitch bracket 142 may be directly connected to the driving shaft of the roll driving component 131 , or may be indirectly connected to the driving shaft of the roll driving component 131 via a connecting frame, an intermediate gear assembly, etc.

[0069] Optionally, the arm-mounted gimbal may further include a gimbal fixing bracket (not shown in the figure), which can be connected to the arm 21, and the rotating mechanism 12 is arranged on the gimbal fixing bracket through the pitch mechanism 14 and the rolling mechanism 13. When the swing mechanism 11, the rotating mechanism 12, the rolling mechanism 13 and the pitch mechanism 14 are connected in sequence, the pitch bracket 142 is connected to the gimbal fixing bracket; when the swing mechanism 11, the rotating mechanism 12, the pitch mechanism 14 and the rolling mechanism 13 are connected in sequence, the rolling bracket 132 is connected to the gimbal fixing bracket.

[0070] Optionally, the gimbal fixing bracket of the arm-mounted gimbal is L-shaped and provided with reinforcing ribs, the pitch bracket 142 is formed as two vertical plates extending downward from the horizontal top plate of the gimbal fixing bracket and arranged at intervals, and the pitch driving member 141 is arranged on the pitch bracket 142 and located on the outside of the pitch bracket 142. The roll bracket 132 is cylindrical and rotatably arranged between the two vertical plates of the pitch bracket 142, and the pitch driving member 141 can drive the roll bracket 132 to pitch around the pitch axis. The rotating bracket 122 is U-shaped and located in the roll bracket 132, and the roll driving member 131 is arranged on the roll bracket 132 and can drive the rotating bracket 122 to roll and swing around the roll axis, and the pitch axis is perpendicular to the roll axis. The swing bracket 112 is U-shaped and located in the rotating bracket 122, and the rotation driving member 121 is arranged on the rotating bracket 122 and can drive the swing bracket 112 to rotate around the rotation axis. The scene detection unit is disposed in the swing bracket 112 , and the swing driving member 111 is disposed outside the swing bracket 112 and can drive the scene detection unit to swing around a swing axis.

[0071] Alternatively, the gimbal fixing bracket of the arm-mounted gimbal is L-shaped and provided with reinforcing ribs, one outer panel of the gimbal fixing bracket is used to connect with the arm 21, the other outer panel is used to install the processing device 17, and the pitch bracket 142 is arranged between the two inner panels. The roll bracket 132, the rotation bracket 122 and the swing bracket 112 are all L-shaped. The arm-mounted gimbal is also provided with a scene detection unit mounting bracket, which is used to install the scene detection unit and is connected to the swing drive 111, and the swing drive 111 can drive the scene detection unit mounting bracket to swing around the swing axis.

[0072] The second aspect of the present application also provides an arm-mounted follow-up sensing device 1, which includes the above-mentioned arm-mounted gimbal and a scene detection unit, and the scene detection unit is arranged on the arm-mounted gimbal and is used to obtain scanning data of the working scene. Since the arm-mounted follow-up sensing device 1 includes the above-mentioned arm-mounted gimbal, it also has all the technical effects brought by the arm-mounted gimbal, so it will not be repeated.

[0073] The third aspect of the present application also provides an arm-mounted tracking sensing system, which includes the above-mentioned arm-mounted tracking sensing device 1. Since the arm-mounted tracking sensing system includes the above-mentioned arm-mounted tracking sensing device 1, it also has all the technical effects brought by the arm-mounted tracking sensing device 1, so it will not be repeated.

[0074] Optionally, the arm-mounted tracking sensing device 1 may further include a first angle sensor, a second angle sensor, a third angle sensor, and a fourth angle sensor (not shown in the figure). The first angle sensor is arranged on the swing mechanism 11 and is used to detect the swing angle of the swing mechanism 11, the second angle sensor is arranged on the rotation mechanism 12 and is used to detect the rotation angle of the rotation mechanism 12, the third angle sensor is arranged on the roll mechanism 13 and is used to detect a first tilt angle value in a variable amplitude direction perpendicular to the boom 21, and the fourth angle sensor is arranged on the pitch mechanism 14 and is used to detect a second tilt angle value in a variable amplitude direction of the boom 21.

[0075] Among them, the first angle sensor, the second angle sensor, the third angle sensor and the fourth angle sensor can be independently set angle sensors, or they can be partially integrated in the IMU attitude detection device. The IMU attitude detection device can have two control modes: automatic and manual. The automatic control mode is mainly controlled by the processing device receiving the IMU attitude angle by itself; the manual control mode is mainly controlled by the user collecting the IMU attitude angle by himself, calling the SDK to control the axial angle rotation of the arm-mounted follow-up sensing device 1 by himself.

[0076] Optionally, the scene detection unit may include an image acquisition component 152, or the scene detection unit may include a radar, or alternatively, the scene detection unit may include both the image acquisition component 152 and the radar. The radar may be a laser radar 151, and the scene detection unit of the present application may use a multi-line laser radar with fewer line beams, which can also obtain a large range of scanning space and reduce manufacturing costs. The image acquisition component 152 may be a camera or a camera, and the camera may be, for example, a variable focus monocular camera. The optical axis of the scene detection unit may be the central axis of the field of view of the image acquisition component 152 or the radar, and the central axis of the field of view of the image acquisition component 152 and the radar is parallel.

[0077] Specifically, when the arm-mounted tracking sensing device 1 is installed at the arm tip of the boom 21, the detection range of the radar is a conical area with the sling / hook as the axis. By controlling the radar scan, the three-dimensional scene map within the radar scanning range can be reconstructed online. Subsequently, the position of the work target can be roughly located in the three-dimensional scene map in the form of point-selecting the work target, and the value can be passed to the camera used for precise positioning (secondary positioning), and the camera or camera can be controlled to follow. The radar can detect the spatial three-dimensional coordinates and azimuths of all objects (including target points and obstacles) in the conical area with the sling / hook as the axis in real time, and fuse the images to distinguish between targets and obstacles. It can also realize automatic obstacle avoidance of the boom 21 according to the movement trend of the hook / load and the dynamic path correction algorithm.

[0078] In addition, according to the rough positioning value of the three-dimensional scene map, the arm-mounted follow-up sensing device 1 can control the arm-mounted pan-tilt to make the camera automatically follow the working target position, extract the high-definition image area block of the working target by focal length stretching, and extract the features of the working target image block. According to the rough positioning value of the working target, the arm 21 rotation / amplitude change action is calculated, and then the automatic lifting is started, and the pan-tilt is controlled to make the camera lens face vertically downward to track the hook and the vertical projection point of the hook. When the camera field of view overlaps with the precise positioning rough judgment area or the working target enters the camera field of view, the camera is started to detect the target features and accurately locate the target.

[0079] In some specific embodiments, the arm-mounted follow-up sensing system may further include a processing device 17, which is used to control the action of the arm-mounted gimbal so that the scene detection unit can rotate and scan the working area at a preset swing angle. The processing device 17 communicates with the arm-mounted gimbal and the scene detection unit respectively, and can perform data exchange communication in the form of a network port. Specifically, the processing device can be configured as follows:

[0080] Obtaining the working posture of the boom 21;

[0081] The pitch mechanism 14 is controlled to move according to the working posture of the boom 21 so that the rotation axis is adjusted to be perpendicular to the ground plane;

[0082] The swing mechanism 11 and the rotation mechanism 12 are controlled to operate so that the scene detection unit performs rotation scanning around the rotation axis at a preset swing angle.

[0083] Since the field of view of the camera and the laser radar 151 is small, the detection range of the camera and the laser radar 151 can be expanded by adjusting the optical axis swing direction of the laser radar 151 and the camera in real time by using the swing mechanism 11 and the rotation mechanism 12. The processing device can automatically obtain the data of the working posture of the arm 21 by communicating with the motion control module of the arm 21; or, it can also be manually input into the processing device.

[0084] Optionally, the processing device 17 is further configured to:

[0085] A global map of the working scene is generated based on the scanning data of the scene detection unit, the motion data of the rotating mechanism 12 , and the motion data of the swing mechanism 11 .

[0086] When the arm 21 is stationary, the rotating mechanism 12 is controlled to perform a rotation action and the swing mechanism 11 is controlled to gradually adjust the scanning area of ​​the camera and the radar, so as to obtain the actual scene point cloud data and image of the entire working area, realize online reconstruction of the working scene and generate a global map of the working scene.

[0087] Optionally, the processing device 17 is further configured to:

[0088] The total number of rotations of the scene detection unit around the rotation axis and the swing angle corresponding to each rotation number are determined according to the working scene size of the boom.

[0089] When determining the total number of rotations of the scene detection unit around the rotation axis and the swing angle corresponding to each rotation according to the working scene size of the boom, the processing device determines the maximum swing angle by combining the working scene size of the boom and the field angle of the scene detection unit, and then determines the rotation scanning parameters of the scene detection unit by combining the maximum swing angle and the field angle of the scene detection unit. The rotation scanning parameters of the scene detection unit include the total number of rotations of the scene detection unit around the rotation axis and the swing angle corresponding to each rotation.

[0090] In addition, in order to balance work efficiency and equipment cost, the total number of rotations of the scene detection unit around the rotation axis can be set to be less than or equal to 5. Figure 6 As shown, when the optical axis of the scene detection unit is downward and perpendicular to the ground plane, the scene detection unit can obtain scanning data of a conical scanning area with the vertical intersection of the optical axis on the ground plane as the center; Figure 7As shown, when the swing angle is adjusted so that the optical axis of the scene detection unit is inclined to the ground plane, the rotation scanning of the scene detection unit can obtain scanning data of a double-conical annular scanning area arranged around the rotation axis; the scanning data of the scanning areas of all rotation cycles at different swing angles can be spliced ​​together to obtain scanning data of the entire working area.

[0091] In addition, the step of generating a global map of the operation scene according to the scanning data of the scene detection unit, the motion data of the rotating mechanism 12, and the motion data of the swing mechanism 11 includes:

[0092] Acquire scanning data of the scene detection unit, motion data of the rotating mechanism 12, and motion data of the swinging mechanism 11;

[0093] The scanned data is processed and spliced ​​in combination with the motion data of the rotating mechanism 12 and the motion data of the swinging mechanism 11 to generate a global map of the working scene.

[0094] Among them, the scanning data of the scene detection unit includes the point cloud data obtained by the radar. The point cloud data is converted and spliced ​​to construct a global map of the work scene of the entire work area. After the construction of the global map of the work scene is completed, the rotating mechanism 12 and the swing mechanism 11 are controlled to return to the original position so that the optical axis of the scene detection unit is perpendicular to the ground plane, and the construction quality of the global map of the work scene can be used to determine whether the global map of the work scene needs to be rebuilt. If not necessary, the arm-mounted gimbal can be directly controlled to make the next work target enter the field of view of the scene detection unit, and locate the work target for subsequent path planning and hoisting operations; if necessary, the arm-mounted follow-up sensing device 1 can be controlled to restart the scanning operation.

[0095] In addition, the step of controlling the swing mechanism 11 and the rotation mechanism 12 to enable the scene detection unit to perform rotation scanning around the rotation axis at a preset swing angle may include:

[0096] Determine the current rotation frequency of the scene detection unit, and control the swing mechanism 11 to adjust the swing angle corresponding to the current rotation frequency;

[0097] Controlling the rotating mechanism 12 to drive the scene detection unit to rotate at a preset rotation angle interval and a preset time period interval until the scene detection unit rotates one circle in the current rotation cycle;

[0098] It is determined that the scene detection unit has completed the last rotation scan, and the scene detection unit is controlled to stop scanning.

[0099] Among them, in the sub-step of controlling the rotating mechanism 12 to drive the scene detection unit to rotate at preset rotation angle intervals and preset time period intervals until the scene detection unit rotates one circle in the rotation cycle, the processing device controls the rotating mechanism 12 to rotate the preset rotation angle at each preset time period, and then obtains the scanning data of the scene detection unit until the rotating mechanism 12 drives the scene detection unit to rotate 360° in the current rotation cycle to complete a circle. For example, the preset rotation angle can be 15° to 30°, preferably 20°; the preset time period can be 0.5 seconds to 3 seconds, preferably 1 second to 2 seconds. Specifically, in any rotation cycle, the rotating mechanism 12 can be controlled to pause for a preset time period after rotating the preset rotation angle, and then continue to rotate until the 360° rotation scan of the rotation cycle is completed. In this way, the point cloud data can be made denser, which is more conducive to improving the construction quality of the global map of the operation scene.

[0100] Specifically, Figure 4 As shown, the axis parallel to the amplitude change direction of the boom 21 is the Y axis, the axis perpendicular to the amplitude change direction of the boom 21 and parallel to the ground plane is the X axis, and the axis perpendicular to the ground plane is the Z axis. When the boom 21 is extended or the amplitude is changed and the boom 21 is twisted, the tilt angles of the laser radar and the camera in the X-axis direction and the Y-axis direction change. By obtaining the tilt angle values ​​of the third angle sensor and the fourth angle sensor in the X-axis and Y-axis directions in real time, and then controlling the action of the arm-mounted gimbal, the optical axis angles of the laser radar and the camera can be adjusted.

[0101] like Figure 5 As shown, in the initial state of the boom 21, the boom 21 is fully retracted (not unfolded), the angle sensor of the arm-mounted gimbal remains parallel to the ground plane, the initial state angles of the third angle sensor and the fourth angle sensor are both 0°, and the Y axis remains parallel to the amplitude change direction.

[0102] When the operating machine starts the gimbal self-adjustment control thread and enters the arm extension step, the first inclination angle value and the second inclination angle value of the third angle sensor and the fourth angle sensor in the X and Y axes and the height of the arm tip to the ground are read in real time, and the control angle is calculated according to the initial attitude angle of the arm-mounted gimbal. The SDK of the arm-mounted gimbal is used to control the self-adjustment of the pitch angle, roll angle and camera focal length, so that the central axis of the field of view of the radar and camera is always perpendicular to the ground plane.

[0103] The laser radar and the camera field of view center axis are perpendicular to the downward control method: when the arm 21 length (variable amplitude or retracted / extended arm) changes, the camera lens angle is tilted accordingly, and at the same time, the third angle sensor and the fourth angle sensor detect the first tilt angle value ∝ in the X-axis direction in real time x and the second inclination angle value of the Y axis ∝ y , ∝ xis the first inclination angle value of the rolling mechanism 13 in the X-axis direction when the angle between the arm and the ground plane is α, ∝ y is the second tilt angle value of the pitch mechanism 14 in the Y-axis direction when the angle between the boom and the ground plane is α. The processing device obtains the angle value from the vehicle-mounted controller in real time through the CAN bus. When the boom 21 is in the initial state (not raised), ∝ x =0°, ∝ y ≠0°. When the boom 21 is extended (extended and luffed) and only ∝ y When the change occurs, the rotation angle P control value and pitch angle T control value of the gimbal on the control arm are: P = 0°, T = ∝ y If the arm 21 is twisted in the X-axis direction, then ∝ x ≠0°, the roll angle R of the control arm gimbal is: R=∝ x , the radar and camera field of view central axis can be controlled vertically downward, and the image plane coordinate system of the camera and radar is consistent with the axis direction of the coordinate system OXYZ set by the boom. When the boom 21 is fully extended, the arm-mounted gimbal self-adjustment control thread is exited, and the gimbal control mode is switched to make the arm-mounted gimbal rotate automatically, control the radar to scan the scene, and obtain the three-dimensional point cloud data of the operation scene.

[0104] When the radar is controlled to scan the scene and obtain the three-dimensional point cloud data of the working scene, when the arm 21 is stationary, the central axis of the radar and camera field of view is controlled to be perpendicular to the ground plane, and the radar point cloud data and camera image data of the scene at that moment are obtained, such as Figure 6 As shown in the figure, the arm-mounted gimbal moves so that the swing angle θ of the camera / radar rotation for the i-th week is the preset swing angle θi (set according to the size of the scan scene), and the gimbal is gradually controlled to rotate 360° horizontally at equal angle intervals, with an interval period of T1, as shown in the figure. Figure 7 Then, by converting the coordinates of the acquired point cloud data, a hoisting scene map of the entire operation area can be obtained.

[0105] Optionally, the working scene size of the boom can be determined according to the length of the boom 21, and the processing device can automatically obtain the length data of the boom 21 and convert it according to a built-in algorithm to obtain the corresponding working scene size of the boom; or, it can also be determined based on manual measurement and manually input into the processing device, and the processing device can then convert it according to the built-in algorithm to obtain the corresponding working scene size of the boom.

[0106] Optionally, the processing device 17 is further configured to:

[0107] When the boom 21 is extended or luffed, the swing mechanism 11, the rolling mechanism 13 and the pitch mechanism 14 are controlled to move so that the optical axis of the scene detection unit is perpendicular to the ground plane.

[0108] The fourth aspect of the present application further provides an operating machine, which includes an arm 21 and the above-mentioned arm-mounted follow-up sensing system, and an arm-mounted follow-up sensing device 1 of the arm-mounted follow-up sensing system is arranged on the arm 21. Since the operating machine includes the above-mentioned arm-mounted follow-up sensing system, it also has all the technical effects brought by the arm-mounted follow-up sensing system, so it will not be repeated.

[0109] The operating machinery may be a crane, such as a truck crane, a crawler crane or a boom tower crane, or other operating machinery such as a pump truck, an aerial work vehicle, a fire truck, a boom-type operating robot, etc. The power of the operating machinery may be provided by a new energy battery (such as a lithium battery or a hydrogen fuel cell), a fuel engine or a hybrid power device, or may be provided by the mains, etc., which is not specifically limited here.

[0110] Optionally, the operating machine may be provided with two arm-mounted follow-up sensing devices 1, wherein the scene detection unit of one arm-mounted follow-up sensing device 1 includes a camera or a camera head, and the scene detection unit of the other arm-mounted follow-up sensing device 1 includes a radar. Alternatively, the operating machine may be provided with only one arm-mounted follow-up sensing device 1, wherein the scene detection unit of the arm-mounted follow-up sensing device 1 includes a radar and an image acquisition component 152, that is, the radar and the image acquisition component 152 share an arm-mounted gimbal, so that the structure is simpler and more compact, and the equipment cost is lower. Among them, the image acquisition component 152 may be a camera or a camera head.

[0111] Alternatively, if Figure 4 As shown, the arm-mounted tracking sensing device 1 is arranged on the arm tip of the arm frame 21 .

[0112] According to the structural characteristics of the top of the boom 21, the movement form of the boom 21 (amplitude change, rotation, winch), the inclination of the vehicle body, the torsion of the boom 21, etc., the arm-mounted follow-up sensing device has changes in the three axes of rotation angle, pitch angle, and roll angle. For this reason, the arm-mounted gimbal of the present application is a four-axis gimbal having a swing mechanism 11, a rotation mechanism 12, a roll mechanism 13, and a pitch mechanism 14, and the laser radar 151 and the camera are integrated on the arm-mounted gimbal. When the boom 21 is extended or amplitude changed, the processing device can collect the angle values ​​of each angle sensor, and control the self-adjustment of the radar and camera posture by controlling the movement of the swing mechanism 11, the roll mechanism 13, and the pitch mechanism 14, so that the optical axis of the scene detection unit is perpendicular to the ground plane. When the boom 21 moves, ensure that the optical axis of the scene detection unit is always perpendicular to the ground plane, so that obstacles around the load and hook can be monitored in real time and avoided, and the camera can be used to accurately identify and detect the position of the operating machinery's positioning target, thereby providing support for the precise positioning of intelligent lifting based on the current posture of the boom 21 of the operating machinery and the deviation of the target position.

[0113] Alternatively, if Figure 4 As shown, the working machine also includes a human-machine interaction device 22, which is used to receive and display the image collected by the scene detection unit and obtain the user's target selection operation for the image. The processing device communicates with the human-machine interaction device 22 and is also configured to:

[0114] Control the rotation mechanism 12 and the swing mechanism 11 to move so that the work target enters the field of view collected by the scene detection unit;

[0115] Receive a target selection operation and determine the position of the work target in the global map of the work scene according to the target selection operation.

[0116] Among them, the human-computer interaction device 22 may include a display screen, which can display the image collected by the scene detection unit. The display screen may be a touch display screen, so that the user can perform a target selection operation by clicking the position of the operating target in the panoramic image. It is understandable that the target selection operation can also be implemented in other ways, for example, the position of the operating target in the image can also be selected by a peripheral device connected to the display screen. The processing device 17 communicates with the human-computer interaction device 22 and has an image processing function. The processing device 17 can receive the collected image from the scene detection unit and transmit it to the human-computer interaction device 22 for display, and receive the user's target selection operation for the image to determine the position of the operating target in the global map of the operating scene according to the target selection operation. In this way, the operator of the operating machine can intuitively view the actual lifting scene through the human-computer interaction device 22, thereby eliminating the need for coordinated command by the commander, reducing labor costs. And by automatically determining the position of the operating target according to the user's target selection operation on the image, the automatic operation positioning of the operating machine can be achieved, thereby reducing the operating intensity and proficiency of the operator and improving work efficiency.

[0117] Furthermore, the processing device 17 may also be configured to:

[0118] The operation path planning and safety calculation are performed according to the current state of the boom 21 and the spatial position of the operation target in the global map of the operation scene.

[0119] Furthermore, after the intelligent lifting function is turned on, the boom 21 moves according to the operation path planning. When the boom 21 moves close to the operation target, the processing device can also be configured as follows:

[0120] The arm-mounted gimbal is controlled to move so that the optical axis of the scene detection unit is perpendicular to the ground plane, and when the operation target enters the field of view collected by the scene detection unit, the operation target is identified and re-positioned.

[0121] Among them, secondary positioning is precise positioning, which is conducive to the realization of automation and intelligent operation of operating machinery.

[0122] In addition, when the hook approaches the work target, the distance between the hook and the work target is within the set threshold range, which can be regarded as the completion of the current follow-up sensing operation, and then the next follow-up sensing operation can be entered according to the working conditions. At the beginning of the next follow-up sensing operation, it can be determined whether it is necessary to rebuild the global map of the work scene. If it is necessary to rebuild the global map of the work scene, the arm-mounted follow-up sensing device 1 can be controlled to restart the scanning operation after the arm is extended; if it is not necessary to rebuild the global map of the work scene, the arm-mounted gimbal can be directly controlled to allow the next work target to enter the field of view of the scene detection unit, and locate the work target for subsequent path planning and lifting operations.

[0123] It should be noted that the structural principles of the components such as the arm 21, radar, camera, swing mechanism 11, rotation mechanism 12, roll mechanism 13, and pitch mechanism 14, as well as the coordinate conversion and splicing method principles of point cloud data are well known to technical personnel in this field, and do not belong to the core improvement part of this application, and will not be elaborated here.

[0124] The fifth aspect of the present application also provides a new energy vehicle, which includes a vehicle body, an arm 21 and the above-mentioned arm-mounted follow-up sensing system, the arm 21 is arranged on the vehicle body, and the arm-mounted gimbal is arranged on the arm 21. Since the new energy vehicle includes the above-mentioned arm-mounted follow-up sensing system, it also has all the technical effects brought by the arm-mounted follow-up sensing system, so it will not be repeated. The new energy vehicle can be a pure electric vehicle, an extended-range electric vehicle, a hybrid vehicle, a fuel cell electric vehicle, a hydrogen engine vehicle, etc.

[0125] The sixth aspect of the present application also provides a boom operation method, such as Figure 1 and Figure 2 As shown, the arm 21 is provided with an arm-mounted gimbal, which includes a pitch mechanism 14, a rotation mechanism 12 and a swing mechanism 11 provided on the rotation mechanism 12, the rotation mechanism 12 can drive the swing mechanism 11 to rotate around a rotation axis, a scene detection unit is provided on the swing mechanism 11, the swing mechanism 11 can drive the scene detection unit to swing to adjust the swing angle between the optical axis of the scene detection unit and the rotation axis, the pitch mechanism 14 is connected between the arm 21 and the rotation mechanism 12 and can drive the rotation mechanism 12 to perform up and down pitch movement around the pitch axis, the pitch axis is perpendicular to the amplitude change direction of the arm 21, and the arm operation method includes:

[0126] Obtaining the working posture of the boom 21;

[0127] The pitch mechanism 14 is controlled to move according to the working posture of the boom 21 so that the rotation axis is adjusted to be perpendicular to the ground plane;

[0128] The swing mechanism 11 and the rotation mechanism 12 are controlled to operate so that the scene detection unit performs rotation scanning around the rotation axis at a preset swing angle.

[0129] Optionally, the boom operation method further includes:

[0130] A global map of the working scene is generated based on the scanning data of the scene detection unit, the motion data of the rotating mechanism 12 , and the motion data of the swing mechanism 11 .

[0131] Optionally, the boom operation method further includes:

[0132] The total number of rotations of the scene detection unit around the rotation axis and the swing angle corresponding to each rotation number are determined according to the working scene size of the arm 21.

[0133] Optionally, the boom operation method further includes:

[0134] The working scene size of the boom 21 is determined according to the length of the boom 21 .

[0135] Optionally, controlling the swing mechanism 11 and the rotation mechanism 12 to operate so that the scene detection unit performs rotation scanning around the rotation axis at a preset swing angle includes:

[0136] Determine the current rotation frequency of the scene detection unit, and control the swing mechanism 11 to adjust the swing angle corresponding to the current rotation frequency;

[0137] Controlling the rotating mechanism 12 to drive the scene detection unit to rotate at a preset rotation angle interval and a preset time period interval until the scene detection unit rotates one circle in the current rotation cycle;

[0138] It is determined that the scene detection unit has completed the last rotation scan, and the scene detection unit is controlled to stop scanning.

[0139] Optionally, a global map of the operation scene is generated according to the scanning data of the scene detection unit, the motion data of the rotating mechanism 12, and the motion data of the swing mechanism 11, including:

[0140] Acquire scanning data of the scene detection unit, motion data of the rotating mechanism 12, and motion data of the swinging mechanism 11;

[0141] The scanned data is processed and spliced ​​in combination with the motion data of the rotating mechanism 12 and the motion data of the swinging mechanism 11 to generate a global map of the working scene.

[0142] Optionally, the arm-mounted gimbal further includes a rolling mechanism 13, which can drive the rotating mechanism 12 to rotate around a rolling axis, and the rolling axis is perpendicular to the pitch axis. Before the scene detection unit rotates and scans, the arm operation method further includes:

[0143] The rolling mechanism 13 is controlled to operate so that the rotation axis can be perpendicular to the ground plane when the boom 21 is twisted or tilted.

[0144] Optionally, the boom operation method further includes:

[0145] Detecting the swing angle, a first tilt angle value of the roll mechanism 13 in the amplitude changing direction perpendicular to the boom 21, and a second tilt angle value of the pitch mechanism 14 in the amplitude changing direction of the boom 21;

[0146] The swing mechanism 11, the roll mechanism 13 and the pitch mechanism 14 are controlled to operate according to the swing angle, the first tilt angle value and the second tilt angle value, so that the optical axis of the scene detection unit is perpendicular to the ground plane.

[0147] Optionally, the boom operation method further includes:

[0148] Control the rotation mechanism 12 and the swing mechanism 11 to move so that the work target enters the field of view collected by the scene detection unit;

[0149] Display the image captured by the scene detection unit and obtain the user's target selection operation on the image;

[0150] Receive a target selection operation and determine the position of the work target in the global map of the work scene according to the target selection operation.

[0151] Optionally, the boom operation method further includes:

[0152] The operation path planning and safety calculation are performed according to the current state of the boom 21 and the spatial position of the operation target in the global map of the operation scene.

[0153] Optionally, the boom operation method further includes:

[0154] When the arm 21 moves according to the operation path planning, the arm-mounted gimbal is controlled to move so that the optical axis of the scene detection unit is perpendicular to the ground plane, and when the operation target enters the field of view collected by the scene detection unit, the operation target is identified and re-positioned.

[0155] Optionally, the scene detection unit includes an image acquisition component 152 and / or a radar.

[0156] like Figure 8 As shown, the control process of the arm-mounted follow-up sensing device 1 of the present application will be briefly described below using a crane as an example:

[0157] 1. If Figure 5As shown, before the boom 21 of the crane is extended, the boom 21 of the crane is parallel to the ground plane. The arm-mounted follow-up sensing device 1 is powered on to obtain the initial state of the arm-mounted gimbal. The initial state of the arm-mounted gimbal includes a rotation angle, a swing angle, a pitch angle, and a roll angle.

[0158] 2. Crane arm extension steps: Figure 6 As shown, the swing mechanism 11, the rolling mechanism 13 and the pitching mechanism 14 are controlled in real time to adjust the swing angle, the pitch angle and the rolling angle so that the optical axis of the scene detection unit is perpendicular to the ground plane. After the arm is extended, the boom 21 of the crane is in the initial state of operation, at which time the optical axis of the scene detection unit is perpendicular to the ground plane.

[0159] 3. Scene scanning step: determine the current rotation number of the scene detection unit, control the swing mechanism 11 to adjust the swing angle corresponding to the current rotation number, control the rotation mechanism 12 to drive the scene detection unit to rotate at a preset rotation angle interval and a preset time period interval, until the scene detection unit rotates one circle at the current rotation number, determine that the scene detection unit has completed the last rotation scan, and control the scene detection unit to stop scanning.

[0160] 4. Based on the radar point cloud and camera images obtained by scanning, three-dimensional reconstruction of the operation scene is performed to generate a global map of the operation scene.

[0161] 5. After the global map of the operation scene is generated, the swing mechanism 11 and the rotation mechanism 12 are controlled to adjust the swing angle and the rotation angle so that the optical axes of the radar and the camera are perpendicular to the ground plane. At this time, it can be determined whether the global map of the operation scene needs to be rebuilt. If necessary, the scene scanning step is re-entered; if not, the operation target initial positioning step is entered.

[0162] 6. Initial positioning step of the work target: control the swing mechanism 11 and the rotation mechanism 12 to adjust the swing angle and the rotation angle so that the work target (lifting object or installation point) enters the camera field of view, selects the work target in the added image, and calculates and obtains the spatial position of the work target in the global map of the work scene.

[0163] 7. After the positioning of the work target is completed, the swing mechanism 11, the rolling mechanism 13, the pitching mechanism 14 and the rotating mechanism 12 are controlled to adjust the swing angle, the rolling angle, the pitch angle and the rotation angle to correct the radar and camera field of view and make the optical axis of the radar and camera perpendicular to the ground plane. In addition, according to the current state of the crane boom 21 and the spatial position of the work target, the work path planning and safety calculation are automatically performed.

[0164] 8. Turn on the intelligent lifting function and control the boom 21 to move along the planned operation path. During this process, the swing mechanism 11, the roll mechanism 13 and the pitch mechanism 14 need to be controlled in real time to adjust the swing angle, roll angle and pitch angle so that the optical axes of the radar and the camera are perpendicular to the ground plane.

[0165] 9. When the boom 21 moves close to the work target and the work target enters the field of view of the radar and camera, the work target is automatically identified and re-positioned (precisely positioned) based on the radar and camera. In this process, the swing mechanism 11, the rolling mechanism 13 and the pitch mechanism 14 need to be controlled in real time to adjust the swing angle, the rolling angle and the pitch angle so that the optical axis of the radar and the camera is perpendicular to the ground plane.

[0166] 10. When the hook approaches the work target and the distance between the hook and the work target is within the set threshold range, the work is considered completed.

[0167] 11. After the previous operation is completed and the next operation process is entered, it can be determined whether it is necessary to rebuild the global map of the operation scene. If necessary, re-enter the scene scanning step; if not, enter the operation target initial positioning step. If the previous operation is completed and there is no need to enter the next operation process, the operation can be ended at this time.

[0168] In summary, the arm-mounted follow-up sensing device 1, the arm-mounted follow-up sensing system and the operating machinery of the present application have an arm-mounted gimbal capable of four-axis motion control. Sensors such as cameras and radars are installed on the arm-mounted gimbal, which can realize 360-degree rotation of the camera radar in the horizontal direction, amplitude change direction and tilt control in the left and right directions perpendicular to the amplitude change direction, and can keep the optical axes of the camera and radar vertically downward during the amplitude change and rotation dynamic process of the boom 21. In addition, the arm-mounted gimbal, the arm-mounted follow-up sensing device, the operating machinery and the new energy vehicle of the present application can realize rotational scanning of the camera radar at different swing angles, thereby realizing wide-view monitoring and three-dimensional reconstruction of a large operating range; it can also keep the camera radar in a vertical downward posture regardless of whether the boom 21 has a roll deformation during the movement of the boom 21, thereby realizing the monitoring of the hook load and the real-time detection of surrounding obstacles during the movement of the hook load, and helping to ensure the coverage of the scene detection area. In addition, when the arm-mounted pan / tilt head, arm-mounted follow-up sensing device, and arm-mounted follow-up sensing system of the present application are applied to operating machinery such as cranes, arm-mounted follow-up three-dimensional reconstruction, obstacle detection, target tracking, etc. can be realized, providing a technical basis for the crane to realize automatic lifting and intelligent lifting, thereby ensuring the "eye" function of the crane.

[0169] In the description of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present application, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0170] In this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or communication with each other; it can be a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0171] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0172] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. An arm-mounted gimbal, characterized in that: Capable of being mounted on an arm (21) and comprising: Rotating mechanism (12); a swing mechanism (11), the rotating mechanism (12) being connected to the swing mechanism (11) and capable of driving the swing mechanism (11) to rotate around a rotation axis, the swing mechanism (11) being used to mount a scene detection unit and capable of driving the scene detection unit to swing around the swing axis to adjust a swing angle between an optical axis of the scene detection unit and the rotation axis; and The pitch mechanism (14) is connected between the arm (21) and the rotating mechanism (12) and is capable of driving the rotating mechanism (12) to perform up and down pitch movements around a pitch axis.

2. The arm-mounted gimbal according to claim 1, characterized in that: The arm-mounted gimbal further comprises: A rolling mechanism (13), wherein the rolling mechanism (13) is connected between the arm (21) and the rotating mechanism (12) and is capable of driving the rotating mechanism (12) to rotate around a rolling axis, wherein the rolling axis is perpendicular to the pitch axis.

3. The arm-mounted gimbal according to claim 2, characterized in that: The pitch mechanism (14), the roll mechanism (13), the rotation mechanism (12) and the swing mechanism (11) are connected in sequence.

4. The arm-mounted gimbal according to claim 3, characterized in that: The swing mechanism (11) comprises a swing drive member (111) and a swing bracket (112); the rotation mechanism (12) comprises a rotation drive member (121) and a rotation bracket (122); the roll mechanism (13) comprises a roll drive member (131) and a roll bracket (132); the pitch mechanism (14) comprises a pitch drive member (141) and a pitch bracket (142); the swing drive member (111) is arranged on the swing bracket (112); and the drive shaft of the swing drive member (111) can be connected to the scene detection unit The rotating drive member (121) is arranged on the rotating bracket (122) and the driving shaft of the rotating drive member (121) is drivingly connected to the swing bracket (112); the rolling drive member (131) is arranged on the rolling bracket (132) and the driving shaft of the rolling drive member (131) is drivingly connected to the rotating bracket (122); the pitching drive member (141) is arranged on the pitching bracket (142) and the driving shaft of the pitching drive member (141) is drivingly connected to the rolling bracket (132).

5. The arm-mounted gimbal according to claim 2, characterized in that: The rolling mechanism (13), the pitching mechanism (14), the rotating mechanism (12) and the swinging mechanism (11) are connected in sequence.

6. The arm-mounted pan / tilt platform according to claim 5, characterized in that: The swing mechanism (11) comprises a swing drive member (111) and a swing bracket (112); the rotation mechanism (12) comprises a rotation drive member (121) and a rotation bracket (122); the roll mechanism (13) comprises a roll drive member (131) and a roll bracket (132); the pitch mechanism (14) comprises a pitch drive member (141) and a pitch bracket (142); the swing drive member (111) is arranged on the swing bracket (112); and the drive shaft of the swing drive member (111) can be connected to the scene detection unit The rotating drive member (121) is arranged on the rotating bracket (122) and the driving shaft of the rotating drive member (121) is drivingly connected to the swing bracket (112); the pitch drive member (141) is arranged on the pitch bracket (142) and the driving shaft of the pitch drive member (141) is drivingly connected to the rotating bracket (122); the roll drive member (131) is arranged on the roll bracket (132) and the driving shaft of the roll drive member (131) is drivingly connected to the pitch bracket (142).

7. The arm-mounted gimbal according to claim 2, characterized in that: The arm-mounted gimbal further comprises: A pan / tilt fixing bracket, wherein the pan / tilt fixing bracket can be connected to the arm (21), and the rotating mechanism (12) is arranged on the pan / tilt fixing bracket via the rolling mechanism (13) and the pitching mechanism (14).

8. An arm-mounted follow-up sensing device, characterized in that: include: An arm-mounted gimbal according to any one of claims 1 to 7; and The scene detection unit is arranged on the arm-mounted gimbal and is used to obtain scanning data of the working scene.

9. The arm-mounted follow-up sensing device according to claim 8, characterized in that: The scene detection unit includes an image acquisition component (152) and / or a radar.

10. The arm-mounted follow-up sensing device according to claim 8, characterized in that: The arm-mounted follow-up sensing device (1) further comprises: a first angle sensor, arranged on the swing mechanism (11) and used to detect the swing angle of the swing mechanism (11); and A second angle sensor is arranged on the rotating mechanism (12) and is used to detect the rotation angle of the rotating mechanism (12).

11. A working machine, characterized in that: include: Arm (21); and According to the arm-borne tracking sensing device according to any one of claims 8 to 10, the arm-borne tracking sensing device (1) is arranged on the arm support (21).

12. The working machine according to claim 11, characterized in that: The arm-mounted follow-up sensing device (1) is arranged on the arm tip of the arm frame (21).

Images