Unmanned construction tower crane software and hardware platform based on machine vision and Internet of Things

Abstract

The invention discloses an unmanned construction tower crane software and hardware platform based on machine vision and Internet of Things, the unmanned construction tower crane software and hardware platform comprises a sensing device, a camera and a tower crane intelligent controller which are arranged on a tower crane, and the working process is as follows: 1) the tower crane intelligent controller plans a tower crane operation path according to the relative position of goods and controls the tower crane to operate; the tower crane moves to enable the lifting hook to reach the relative position of the cargo, and the cargo is carried by the cargo boom; and (2) the tower crane intelligent controller plans a tower crane operation path according to the relative position of the terminal point and controls the tower crane to operate, so that the tower crane moves to enable a lifting hook to reach the relative position of the terminal point, and then the cargo boom is controlled to unload carried cargoes to the terminal point. The tower crane is controlled by adopting an intelligent program, so that the safety and the reliability of the operation of the tower crane are improved.

Description

Technical Field

[0001] The present invention relates to the field of tower cranes, and in particular to a software and hardware platform for an unmanned construction tower crane based on machine vision and the Internet of Things. Background Art

[0002] Tower cranes are commonly used equipment on construction sites. Their efficient and precise lifting and handling capabilities, as well as their ability to handle high-altitude and heavy materials, make them play an important role in various industries, improve construction and logistics efficiency, and make important contributions to the smooth progress of engineering projects. However, improper operation of tower cranes can easily lead to collisions between cranes or between cranes and buildings, causing serious casualties and economic losses.

[0003] The existing tower crane operation relies on experienced operators, requiring operators to have a high level of proficiency. Due to the particularity and safety requirements of tower crane operation, operators need to maintain a high degree of concentration during operation. Under long-term operation, there is a risk of operating errors. At the same time, in the case of more complex modern logistics and construction environments, tower cranes need to be constantly adjusted and positioned, which adds a lot of time and cost. In the actual operation process, sudden obstacles may occur, and manual operation may not be able to detect or avoid obstacles immediately. If there is no immediate obstacle avoidance, it may cause unpredictable losses. At the same time, for management, the operating status of the tower crane cannot be monitored in real time, which brings higher risks to its operational decision-making. Summary of the invention

[0004] The purpose of the present invention is to provide a software and hardware platform for unmanned construction tower crane based on machine vision and the Internet of Things, including a sensor device, a camera, and a tower crane intelligent controller arranged on the tower crane;

[0005] The sensing equipment includes a gyroscope, a satellite locator, a radar, a strain gauge, and an encoder;

[0006] The satellite locator obtains the tower crane hook positioning data and transmits it to the tower crane intelligent controller;

[0007] The strain gauge is arranged on the tower crane boom to monitor the stress state of the boom and transmit it to the tower crane intelligent controller;

[0008] The encoders respectively collect the relative displacement of the hooks and transmit them to the tower crane intelligent controller;

[0009] The gyroscope obtains the tower crane boom direction data and transmits it to the tower crane intelligent controller;

[0010] The tower crane intelligent controller obtains the working position status of the tower crane through various sensors to control the tower crane;

[0011] The radar detects that an obstacle appears within the warning radius, and the tower crane intelligent controller controls the tower crane to slow down;

[0012] The radar detects that an obstacle appears within the braking radius, and the tower crane intelligent controller controls the tower crane to brake urgently;

[0013] The camera is used to obtain image information of the cargo and the destination, and transmit it to the tower crane intelligent controller;

[0014] The tower crane intelligent controller drives the tower crane to move based on the received data to complete the loading and unloading of goods.

[0015] Furthermore, the workflow of the unmanned construction tower crane software and hardware platform is as follows:

[0016] 1) Integrate sensing and communication equipment, cameras, and controllers on the tower crane;

[0017] 2) Arrange the tower crane in the construction area and limit the movable area of ​​the tower crane;

[0018] 3) The customer terminal obtains and processes the satellite positioning absolute position information of the goods and the destination based on the map product API, and transmits it to the tower crane intelligent controller;

[0019] 4) The tower crane intelligent controller plans the tower crane operation path based on the input cargo satellite positioning and the tower crane center satellite positioning information after the tower crane is installed, so that the tower crane moves so that the hook reaches the absolute satellite positioning position of the cargo;

[0020] 5) Use the camera to obtain cargo image information and transmit it to the tower crane intelligent controller; the tower crane intelligent controller processes the cargo image information in real time to obtain the relative position of the cargo;

[0021] 6) The tower crane intelligent controller plans the tower crane operation path according to the relative position of the cargo, and controls the tower crane operation, so that the tower crane moves so that the hook reaches the relative position of the cargo, and uses the crane arm to carry the cargo;

[0022] Then, the tower crane intelligent controller plans the tower crane operation path according to the current position of the tower crane and the satellite positioning absolute position information of the end point, and controls the tower crane operation so that the tower crane moves so that the hook reaches the satellite positioning absolute position of the end point;

[0023] 7) Use the camera to obtain the end point image information and transmit it to the tower crane intelligent controller; the tower crane intelligent controller processes the cargo image information to obtain the relative position of the end point;

[0024] 8) The tower crane intelligent controller plans the tower crane operation path according to the relative position of the end point, and controls the tower crane operation, so that the tower crane moves so that the hook reaches the relative position of the end point, and then controls the crane arm to unload the carried cargo to the end point.

[0025] Furthermore, during the operation of the tower crane, radar is used to detect obstacles. If there is an obstacle within the warning radius, the crane will slow down; if there is an obstacle within the braking radius, the crane will brake; after braking, the tower crane's operating path will be replanned.

[0026] Furthermore, the satellite positioning absolute position information of the goods and the destination is obtained by using map marking to obtain the satellite positioning absolute position information of the goods and the destination through an online or offline map.

[0027] Furthermore, the steps of processing the cargo image information by the tower crane intelligent controller include: firstly, marking the cargo image information by selecting a frame, removing background interference, and obtaining a cargo identification frame; then, processing the cargo identification frame by using a machine learning image recognition algorithm, establishing a cargo identification model, transmitting the cargo identification model to the tower crane intelligent controller for comparison and search, performing image recognition by a camera, and obtaining the relative position of the cargo;

[0028] The steps for the tower crane intelligent controller to process the endpoint image information include: first, box-marking the endpoint image information to remove background interference and obtain an endpoint identification frame; then, using a machine learning image recognition algorithm to process the endpoint identification frame, establish an endpoint target model, transmit the target model to the tower crane intelligent controller for comparison and search, perform image recognition through the camera, and obtain the relative position of the endpoint.

[0029] Furthermore, during the operation of the tower crane, the tower crane intelligent controller controls the tower crane operation error through the PID algorithm to compensate for the tower crane motor output;

[0030] The error value I output by the PID algorithm is as follows:

[0031]

[0032] In the formula, K p represents the proportional gain, e(t) represents the operating error value at time t, K i represents the integration time constant, K d Represents the differential time constant.

[0033] Furthermore, when controlling the operation of the tower crane, the tower crane operation parameters at the next moment are the average values ​​of the tower crane operation parameters at the current moment and the tower crane operation parameters at the past T moments.

[0034] Furthermore, before the tower crane reaches the relative position of the cargo and / or the relative position of the destination, the cargo and the destination are tracked by visual recognition or color recognition; the visual recognition is achieved by a camera; the color recognition is achieved by an optical flow vector V;

[0035] The optical flow vector V is as follows:

[0036] V=∑ x,y∈Ω W 2 (x)(l x u+l y v+l t ) 2 =(A T W 2 A) -1 A T W 2 b(2)

[0037] Where W 2 (x) is the window weight coefficient; vector Weight vector W = diag(W(X1),…W(X n )), bias vector Coordinate vector I x ,I y is the horizontal and vertical coordinates of the image pixel.

[0038] Further, it also includes a virtual cockpit;

[0039] The user can remotely control the tower crane by operating the virtual cockpit, making the unmanned construction tower crane move synchronously.

[0040] The technical effect of the present invention is undoubted, and the beneficial effects of the present invention are as follows:

[0041] 1) The use of intelligent program to control the tower crane increases the safety and reliability of tower crane operation;

[0042] 2) The system provides path planning and intelligent obstacle avoidance systems, which can plan the most reasonable and efficient transportation route in a short time, and instantly detect obstacles and plan avoidance routes;

[0043] 3) The use of the Internet of Things and a variety of sensor technologies, based on the domestic advanced network communication technology, has enabled the possibility of uploading the status of the tower crane. Using the construction concept of the intelligent manufacturing factory, the terminal carried by the tower crane can first perform edge computing, and upload the processed data (such as stress, speed, power-on time, positioning and other status information) to the server, and then send it to the client. Users can initiate management and control of the tower crane through the client, and at the same time, it can be equipped with twin driving equipment to realize remote simulated driving of the tower crane. The unmanned driving task of the tower crane is issued by the client. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is the overall flow chart of the unmanned construction tower crane method based on machine vision and the Internet of Things.

[0045] Figure 2Design structures for client terminals. DETAILED DESCRIPTION

[0046] The present invention is further described below in conjunction with the embodiments, but it should not be understood that the above subject matter of the present invention is limited to the following embodiments. Without departing from the above technical ideas of the present invention, various substitutions and changes are made according to the common technical knowledge and customary means in the art, which should all be included in the protection scope of the present invention.

[0047] Embodiment 1:

[0048] See also Figure 1 to Figure 2 , a software and hardware platform for unmanned construction tower crane based on machine vision and the Internet of Things, including sensor equipment, cameras, and tower crane intelligent controllers arranged on the tower crane;

[0049] The sensing equipment includes a gyroscope, a satellite locator, a radar, a strain gauge, and an encoder;

[0050] The satellite locator obtains the tower crane hook positioning data and transmits it to the tower crane intelligent controller;

[0051] The strain gauge is arranged on the tower crane boom to monitor the stress state of the boom and transmit it to the tower crane intelligent controller;

[0052] The encoders respectively collect the relative displacement of the hooks and transmit them to the tower crane intelligent controller;

[0053] The gyroscope obtains the tower crane boom direction data and transmits it to the tower crane intelligent controller;

[0054] The tower crane intelligent controller obtains the working position status of the tower crane through various sensors to control the tower crane;

[0055] The radar detects that an obstacle appears within the warning radius, and the tower crane intelligent controller controls the tower crane to slow down;

[0056] The radar detects that an obstacle appears within the braking radius, and the tower crane intelligent controller controls the tower crane to brake urgently;

[0057] The camera is used to obtain image information of the cargo and the destination, and transmit it to the tower crane intelligent controller;

[0058] The tower crane intelligent controller drives the tower crane to move based on the received data to complete the loading and unloading of goods.

[0059] The workflow of the unmanned construction tower crane software and hardware platform is as follows:

[0060] 1) Integrate sensing and communication equipment, cameras, and controllers on the tower crane;

[0061] 2) Arrange the tower crane in the construction area and limit the movable area of ​​the tower crane;

[0062] 3) The customer terminal obtains and processes the satellite positioning absolute position information of the goods and the destination based on the map product API, and transmits it to the tower crane intelligent controller;

[0063] 4) The tower crane intelligent controller plans the tower crane operation path based on the input cargo satellite positioning and the tower crane center satellite positioning information after the tower crane is installed, so that the tower crane moves so that the hook reaches the absolute satellite positioning position of the cargo;

[0064] 5) Use the camera to obtain cargo image information and transmit it to the tower crane intelligent controller; the tower crane intelligent controller processes the cargo image information in real time to obtain the relative position of the cargo;

[0065] 6) The tower crane intelligent controller plans the tower crane operation path according to the relative position of the cargo, and controls the tower crane operation, so that the tower crane moves so that the hook reaches the relative position of the cargo, and uses the crane arm to carry the cargo;

[0066] Then, the tower crane intelligent controller plans the tower crane operation path according to the current position of the tower crane and the satellite positioning absolute position information of the end point, and controls the tower crane operation so that the tower crane moves so that the hook reaches the satellite positioning absolute position of the end point;

[0067] 7) Use the camera to obtain the end point image information and transmit it to the tower crane intelligent controller; the tower crane intelligent controller processes the cargo image information to obtain the relative position of the end point;

[0068] 8) The tower crane intelligent controller plans the tower crane operation path according to the relative position of the end point, and controls the tower crane operation, so that the tower crane moves so that the hook reaches the relative position of the end point, and then controls the crane arm to unload the carried cargo to the end point.

[0069] During the operation of the tower crane, radar is used to detect obstacles. If there is an obstacle within the warning radius, the crane will slow down. If there is an obstacle within the braking radius, the crane will brake. After braking, the tower crane's operation path will be replanned.

[0070] The method for obtaining the satellite positioning absolute position information of the goods and the destination is: using map marking to obtain the satellite positioning absolute position information of the goods and the destination through an online or offline map.

[0071] The steps of processing cargo image information by the tower crane intelligent controller include: firstly, marking the cargo image information by selecting and marking, removing background interference, and obtaining a cargo identification frame; then, processing the cargo identification frame by using a machine learning image recognition algorithm, establishing a cargo identification model, transmitting the cargo identification model to the tower crane intelligent controller for comparison and search, and obtaining the relative position of the cargo by performing image recognition through a camera;

[0072] The steps for the tower crane intelligent controller to process the endpoint image information include: first, box-marking the endpoint image information to remove background interference and obtain an endpoint identification frame; then, using a machine learning image recognition algorithm to process the endpoint identification frame, establish an endpoint target model, transmit the target model to the tower crane intelligent controller for comparison and search, perform image recognition through the camera, and obtain the relative position of the endpoint.

[0073] During the operation of the tower crane, the tower crane intelligent controller controls the tower crane operation error through the PID algorithm to compensate for the tower crane motor output;

[0074] The error value I output by the PID algorithm is as follows:

[0075]

[0076] In the formula, K p represents the proportional gain, e(t) represents the operating error value at time t, K i represents the integration time constant, K d Represents the differential time constant.

[0077] When controlling the tower crane operation, the tower crane operation parameters at the next moment are the average of the tower crane operation parameters at the current moment and the tower crane operation parameters at the past T moments.

[0078] Before the tower crane reaches the relative position of the cargo and / or the relative position of the destination, the cargo and the destination are tracked by visual recognition or color recognition; the visual recognition is achieved by a camera; the color recognition is achieved by an optical flow vector V;

[0079] The optical flow vector V is as follows:

[0080] V=∑ x,y∈Ω W 2 (x)(l x u+l y v+l t ) 2 =(A T W 2 A) -1 A T W 2 b(2)

[0081] Where W 2 (x) is the window weight coefficient; vector Weight vector W = diag(W(X1),…W(X n )), bias vector Coordinate vector I x ,I y is the horizontal and vertical coordinates of the image pixel.

[0082] Also included is a virtual cockpit;

[0083] The user can remotely control the tower crane by operating the virtual cockpit, making the unmanned construction tower crane move synchronously.

[0084] Embodiment 2:

[0085] A software and hardware platform for unmanned construction tower crane based on machine vision and the Internet of Things, including sensor equipment, cameras, and tower crane intelligent controllers arranged on the tower crane;

[0086] The sensing equipment includes a gyroscope, a satellite locator, a radar, a strain gauge, and an encoder;

[0087] The satellite locator obtains the tower crane hook positioning data and transmits it to the tower crane intelligent controller;

[0088] The strain gauge is arranged on the tower crane boom to monitor the stress state of the boom and transmit it to the tower crane intelligent controller;

[0089] The encoders respectively collect the relative displacement of the hooks and transmit them to the tower crane intelligent controller;

[0090] The gyroscope obtains the tower crane boom direction data and transmits it to the tower crane intelligent controller;

[0091] The tower crane intelligent controller obtains the working position status of the tower crane through various sensors to control the tower crane;

[0092] The radar detects that an obstacle appears within the warning radius, and the tower crane intelligent controller controls the tower crane to slow down;

[0093] The radar detects that an obstacle appears within the braking radius, and the tower crane intelligent controller controls the tower crane to brake urgently;

[0094] The camera is used to obtain image information of the cargo and the destination, and transmit it to the tower crane intelligent controller;

[0095] The tower crane intelligent controller drives the tower crane to move based on the received data to complete the loading and unloading of goods.

[0096] Embodiment 3:

[0097] A software and hardware platform for an unmanned construction tower crane based on machine vision and the Internet of Things, the technical content is the same as that of Example 2, and further, the workflow of the software and hardware platform for an unmanned construction tower crane is as follows:

[0098] 1) Integrate sensing and communication equipment, cameras, and controllers on the tower crane;

[0099] 2) Arrange the tower crane in the construction area and limit the movable area of ​​the tower crane;

[0100] 3) The customer terminal obtains and processes the satellite positioning absolute position information of the goods and the destination based on the map product API, and transmits it to the tower crane intelligent controller;

[0101] 4) The tower crane intelligent controller plans the tower crane operation path based on the input cargo satellite positioning and the tower crane center satellite positioning information after the tower crane is installed, so that the tower crane moves so that the hook reaches the absolute satellite positioning position of the cargo;

[0102] 5) Use the camera to obtain cargo image information and transmit it to the tower crane intelligent controller; the tower crane intelligent controller processes the cargo image information in real time to obtain the relative position of the cargo;

[0103] 6) The tower crane intelligent controller plans the tower crane operation path according to the relative position of the cargo, and controls the tower crane operation, so that the tower crane moves so that the hook reaches the relative position of the cargo, and uses the crane arm to carry the cargo;

[0104] Then, the tower crane intelligent controller plans the tower crane operation path according to the current position of the tower crane and the satellite positioning absolute position information of the end point, and controls the tower crane operation so that the tower crane moves so that the hook reaches the satellite positioning absolute position of the end point;

[0105] 7) Use the camera to obtain the end point image information and transmit it to the tower crane intelligent controller; the tower crane intelligent controller processes the cargo image information to obtain the relative position of the end point;

[0106] 8) The tower crane intelligent controller plans the tower crane operation path according to the relative position of the end point, and controls the tower crane operation, so that the tower crane moves so that the hook reaches the relative position of the end point, and then controls the crane arm to unload the carried cargo to the end point.

[0107] Embodiment 4:

[0108] A software and hardware platform for an unmanned construction tower crane based on machine vision and the Internet of Things, the technical content of which is the same as any one of Examples 2-3. Furthermore, during the operation of the tower crane, radar is used to detect obstacles. If there is an obstacle within the warning radius, the crane is decelerated; if there is an obstacle within the braking radius, the crane is braked; after braking, the operation path of the tower crane is replanned.

[0109] Embodiment 5:

[0110] A software and hardware platform for unmanned construction tower cranes based on machine vision and the Internet of Things, the technical content of which is the same as any one of Examples 2-4. Furthermore, the satellite positioning absolute position information of the cargo and the destination is obtained by using map marking to obtain the satellite positioning absolute position information of the cargo and the destination through an online or offline map.

[0111] Embodiment 6:

[0112] A software and hardware platform for unmanned construction tower crane based on machine vision and the Internet of Things, the technical content of which is the same as any one of Embodiments 2-5, and further, the step of processing cargo image information by the tower crane intelligent controller includes: firstly, box-marking the cargo image information, removing background interference, and obtaining a cargo identification frame; then, processing the cargo identification frame using a machine learning image recognition algorithm, establishing a cargo identification model, transmitting the cargo identification model to the tower crane intelligent controller for comparison and search, and performing image recognition through a camera to obtain the relative position of the cargo;

[0113] The steps for the tower crane intelligent controller to process the endpoint image information include: first, box-marking the endpoint image information to remove background interference and obtain an endpoint identification frame; then, using a machine learning image recognition algorithm to process the endpoint identification frame, establish an endpoint target model, transmit the target model to the tower crane intelligent controller for comparison and search, perform image recognition through the camera, and obtain the relative position of the endpoint.

[0114] Embodiment 7:

[0115] A software and hardware platform for unmanned construction tower crane based on machine vision and the Internet of Things, the technical content of which is the same as any one of Embodiments 2-6, and further, during the operation of the tower crane, the tower crane intelligent controller controls the tower crane operation error through a PID algorithm to compensate for the output of the tower crane motor;

[0116] The error value I output by the PID algorithm is as follows:

[0117]

[0118] In the formula, K p represents the proportional gain, e(t) represents the operating error value at time t, K i represents the integration time constant, K d Represents the differential time constant.

[0119] Embodiment 8:

[0120] A software and hardware platform for unmanned construction tower crane based on machine vision and the Internet of Things, the technical content of which is the same as any one of Examples 2-7. Furthermore, when controlling the operation of the tower crane, the tower crane operation parameters at the next moment are the average of the tower crane operation parameters at the current moment and the tower crane operation parameters at the past T moments.

[0121] Embodiment 9:

[0122] A software and hardware platform for unmanned construction tower crane based on machine vision and the Internet of Things, the technical content is the same as any one of embodiments 2-8, further, the cargo and the destination are tracked by color recognition; the visual recognition is achieved by a camera; the color recognition is achieved by an optical flow vector V;

[0123] The optical flow vector V is as follows:

[0124] V=∑ x,y∈Ω W 2 (x)(l x u+l y v+l t ) 2 =(A T W 2 A) -1 A T W 2 b(2)

[0125] Where W 2 (x) is the window weight coefficient; vector Weight vector W = diag(W(X1),…W(X n )), bias vector Coordinate vector I x ,I y is the horizontal and vertical coordinates of the image pixel.

[0126] Embodiment 10:

[0127] A software and hardware platform for unmanned construction tower crane based on machine vision and the Internet of Things, the technical content of which is the same as any one of Embodiments 2-9, and further includes a virtual cockpit;

[0128] The user can remotely control the tower crane by operating the virtual cockpit, making the unmanned construction tower crane move synchronously.

[0129] Embodiment 11:

[0130] A software and hardware platform for an unmanned construction tower crane based on machine vision and the Internet of Things, the technical content of which is the same as any one of Examples 2-10, and further, communication between the sensor equipment, camera, tower crane intelligent controller, virtual cockpit, and client is achieved through the Internet of Things.

[0131] Embodiment 12:

[0132] An unmanned tower crane system based on machine vision and the Internet of Things is implemented in the following steps:

[0133] S1: If it is based on the transformation of ordinary tower crane, the hardware facilities must be installed first, including gyroscopes, encoders, strain gauges, ESP8266 and other sensors and communication equipment and STM32 and other controllers, and the coordination parameters between related components must be debugged to ensure that the components can operate in coordination;

[0134] S2: Personalized customized tower crane intelligent controller, which communicates with the tower crane intelligent controller (i.e. STM32 microcontroller) through the host computer to control the movement of the tower crane;

[0135] S3: Design and optimize PID algorithm to enable the tower crane to move accurately and stably;

[0136] S4: Complete the construction of the camera module, design the corresponding camera tracking algorithm, and design the automatic obstacle avoidance algorithm in conjunction with the radar;

[0137] S5: Place the set identification object at the preset location, and let the tower crane move the object to the required position input by the host computer;

[0138] S6: Develop the front-end and back-end required for the program to interact with users.

[0139] For the use of tower crane, the steps are as follows:

[0140] S1: The tower crane is installed in the construction area, and the relevant basic configuration is completed, and the movable area of ​​the tower crane is set;

[0141] S2: Users upload pictures of the goods they need to carry and the destination and the absolute location information of satellite positioning through a client in the form of a mobile app or web page. The pictures need to be marked with boxes through the built-in module of the client, and the approximate location is analyzed by clicking on the online / offline map to obtain relative coordinate information to achieve macro positioning; micro positioning is achieved through precise comparison and recognition of the camera module with the tasks uploaded by the user.

[0142] S3: During the operation of the tower crane, the crane will slow down when the radar and camera detect obstacles within the warning radius, and brake when obstacles are detected within the braking radius. The tower crane can plan the route through radar sensing and preset geographic information (such as the existence of fixed obstacles such as buildings) to improve transportation efficiency. The crane can customize personalized algorithms to achieve higher performance route planning.

[0143] S4: The sensors on the tower crane monitor the status of the tower crane and the surrounding environment in real time, and upload them to the server after pre-processing. The server can send them to the client.

[0144] S5: Users can use the client to control the unmanned tower crane, and at the same time perform big data analysis on the tower crane operation data, optimize the algorithm, and further improve efficiency.

[0145] Next, the above-mentioned PID algorithm and camera tracking algorithm are explained in detail.

[0146] In the PID control algorithm, the formula used is: Where K p represents the proportional gain, e(t) represents the error value at time t, K i represents the integration time constant, K d Represents the differential time constant. The result calculated by the above formula is used to adjust the tower crane movement. Among them, K p ,K i ,K d Corresponding adjustments were made through the actual model of the tower crane so that the tower crane can operate smoothly and accurately.

[0147] In order to further improve the stability of the tower crane, the following optimizations were carried out:

[0148] Taking into account that some parameters may exceed the PID output limit, the PID output range is limited here. When the PID exceeds the output range, it is set to the preset value to prevent uncontrollable situations.

[0149] Integral separation was performed and an integral controller was adopted to separate the integral term from PID control to better suppress steady-state errors.

[0150] The idea is similar to the spatiotemporal prediction model in machine learning. In each calculation process, the error is not only compared with the previous result, but also compared with the previous results, and the final result is averaged. This approach makes the tower crane move more smoothly and prevents large rotations that may cause accidents.

[0151] In order to further improve the PID algorithm, we hope to introduce machine learning for training and prediction in subsequent work. p ,K i ,K d At the same time, we hope to access the spatiotemporal prediction model instead of simply comparing with the results of previous times.

[0152] In terms of camera tracking algorithms, the optical flow method can be used for tracking. The target image is binarized. Here, the three-dimensional Otsu segmentation method is used to improve the accuracy of the binary image. The light loss ∑ is estimated by minimizingx,y∈Ω W 2 (x)(l x u+l y v+l t ) 2 , W 2 (x) is the window weight coefficient. Let V = (u, v) T , W=diag(W(X1),…W(X n )), Finally got an A T W 2 AV=A T W 2 b, that is, V = (A T W 2 A) -1 A T W 2 b This is the required optical flow vector.

[0153] After obtaining the optical flow vector, the direction of image movement is obtained, and the camera can track the movement of the object.

Claims

1. A software and hardware platform for unmanned construction tower crane based on machine vision and the Internet of Things, characterized in that: It includes sensor equipment, cameras and tower crane intelligent controller arranged on the tower crane; The sensing equipment includes a gyroscope, a satellite locator, a radar, a strain gauge, and an encoder; The satellite locator obtains the tower crane hook positioning data and transmits it to the tower crane intelligent controller; The strain gauge is arranged on the tower crane boom to monitor the stress state of the boom and transmit it to the tower crane intelligent controller; The encoders respectively collect the relative displacement of the hooks and transmit them to the tower crane intelligent controller; The gyroscope obtains the tower crane boom direction data and transmits it to the tower crane intelligent controller; The tower crane intelligent controller obtains the working position status of the tower crane through various sensors to control the tower crane; When the radar detects that an obstacle is within the warning radius, the tower crane intelligent controller controls the tower crane to slow down; The radar detects that an obstacle appears within the braking radius, and the tower crane intelligent controller controls the tower crane to brake urgently; The camera is used to obtain image information of the cargo and the destination, and transmit it to the tower crane intelligent controller; The tower crane intelligent controller drives the tower crane to move based on the received data to complete the loading and unloading of goods.

2. According to claim 1, a software and hardware platform for unmanned construction tower crane based on machine vision and Internet of Things is characterized in that: The workflow of the unmanned construction tower crane software and hardware platform is as follows: 1) Integrate sensing and communication equipment, cameras, and controllers on the tower crane; 2) Arrange the tower crane in the construction area and limit the movable area of ​​the tower crane; 3) The customer terminal obtains and processes the satellite positioning absolute position information of the goods and the destination based on the map product API, and transmits it to the tower crane intelligent controller; 4) The tower crane intelligent controller plans the tower crane operation path based on the input cargo satellite positioning and the tower crane center satellite positioning information after the tower crane is installed, so that the tower crane moves so that the hook reaches the satellite positioning absolute position of the cargo; 5) Use the camera to obtain cargo image information and transmit it to the tower crane intelligent controller; the tower crane intelligent controller processes the cargo image information in real time to obtain the relative position of the cargo; 6) The tower crane intelligent controller plans the tower crane operation path according to the relative position of the cargo, and controls the tower crane operation, so that the tower crane moves so that the hook reaches the relative position of the cargo, and uses the crane arm to carry the cargo; Then, the tower crane intelligent controller plans the tower crane operation path according to the current position of the tower crane and the satellite positioning absolute position information of the end point, and controls the tower crane operation so that the tower crane moves so that the hook reaches the satellite positioning absolute position of the end point; 7) Use the camera to obtain the end point image information and transmit it to the tower crane intelligent controller; the tower crane intelligent controller processes the cargo image information to obtain the relative position of the end point; 8) The tower crane intelligent controller plans the tower crane operation path according to the relative position of the end point, and controls the tower crane operation, so that the tower crane moves so that the hook reaches the relative position of the end point, and then controls the crane arm to unload the carried cargo to the end point.

3. According to claim 1, the unmanned construction tower crane software and hardware platform based on machine vision and the Internet of Things is characterized in that: During the operation of the tower crane, radar is used to detect obstacles. If there is an obstacle within the warning radius, the crane will slow down. If there is an obstacle within the braking radius, the crane will brake. After braking, the tower crane's operation path will be replanned.

4. According to claim 1, the unmanned construction tower crane software and hardware platform based on machine vision and the Internet of Things is characterized in that: The method for obtaining the satellite positioning absolute position information of the goods and the destination is: using map marking to obtain the satellite positioning absolute position information of the goods and the destination through an online or offline map.

5. According to claim 1, the unmanned construction tower crane software and hardware platform based on machine vision and the Internet of Things is characterized in that: The steps of processing cargo image information by the tower crane intelligent controller include: firstly, marking the cargo image information by selecting and marking, removing background interference, and obtaining a cargo identification frame; then, processing the cargo identification frame by using a machine learning image recognition algorithm, establishing a cargo identification model, transmitting the cargo identification model to the tower crane intelligent controller for comparison and search, and obtaining the relative position of the cargo by performing image recognition through a camera; The steps for the tower crane intelligent controller to process the endpoint image information include: first, box-marking the endpoint image information to remove background interference and obtain an endpoint identification frame; then, using a machine learning image recognition algorithm to process the endpoint identification frame, establish an endpoint target model, transmit the target model to the tower crane intelligent controller for comparison and search, perform image recognition through the camera, and obtain the relative position of the endpoint.

6. The unmanned construction tower crane software and hardware platform based on machine vision and the Internet of Things according to claim 1 is characterized in that: During the operation of the tower crane, the tower crane intelligent controller controls the tower crane operation error through the PID algorithm to compensate for the tower crane motor output; The error value I output by the PID algorithm is as follows: In the formula, K p represents the proportional gain, e(t) represents the operating error value at time t, K i represents the integration time constant, K d Represents the differential time constant.

7. The unmanned construction tower crane software and hardware platform based on machine vision and the Internet of Things according to claim 6 is characterized in that: When controlling the tower crane operation, the tower crane operation parameters at the next moment are the average of the tower crane operation parameters at the current moment and the tower crane operation parameters at the past T moments.

8. The unmanned construction tower crane software and hardware platform based on machine vision and the Internet of Things according to claim 1 is characterized in that: Before the tower crane reaches the relative position of the cargo and / or the relative position of the destination, the cargo and the destination are tracked by visual recognition or color recognition; the visual recognition is achieved by a camera; the color recognition is achieved by an optical flow vector V; The optical flow vector V is as follows: V=∑ x，y∈Ω W 2 (x)(l x u+l y v+l t ) 2 =(A T W 2 A) -1 A T W 2 b (2) Where W 2 (x) is the window weight coefficient; vector Weight vector W = diag(W(X1), ...W(X n ), bias vector Coordinate vector I x , I y is the horizontal and vertical coordinates of the image pixel.

9. The unmanned construction tower crane software and hardware platform based on machine vision and the Internet of Things according to claim 1 is characterized by: Also included is a virtual cockpit; The user can remotely control the tower crane by operating the virtual cockpit, making the unmanned construction tower crane move synchronously.

Images